unidata/netcdf4-python

netcdf4-python: python/numpy interface to the netCDF C library

Structural Verdict

How Data Flows Through the System

Data flows from Python numpy arrays through netCDF4 Python objects to the underlying netCDF C library, which handles file I/O and data transformations.

1. Array Creation — User creates numpy arrays with scientific data
2. Dataset Creation — Create netCDF4.Dataset object with file format and options
3. Variable Definition — Define variables with dimensions, data types, and compression settings
4. Data Writing — Write numpy arrays to variables through C library calls
5. File Storage — netCDF C library handles actual file format and compression
6. Data Reading — Read operations return numpy arrays from stored netCDF data

System Behavior

How the system actually operates at runtime — where data accumulates, what loops, what waits, and what controls what.

Data Pools

Feedback Loops

• Compression Optimization (convergence, balancing) — Trigger: Variable creation with compression settings. Action: Adjust compression level based on data characteristics. Exit: Optimal compression ratio achieved.
• Parallel I/O Coordination (polling, balancing) — Trigger: MPI collective operations. Action: Coordinate writes across parallel processes. Exit: All processes complete operation.

Delays & Async Processing

• Disk I/O Operations (async-processing, ~varies by file size) — GIL released during C library calls allowing concurrent operations
• Compression Processing (batch-window, ~depends on data size) — Data compressed before writing to disk

Control Points

• File Format Selection (runtime-toggle) — Controls: netCDF format version and features. Default: NETCDF4
• Compression Settings (threshold) — Controls: zlib compression level and shuffle filter. Default: complevel=6
• Parallel I/O Mode (feature-flag) — Controls: MPI collective vs independent operations. Default: False
• Auto Complex Number Handling (feature-flag) — Controls: Automatic conversion of complex data structures. Default: False

Technology Stack

Key Components

• Dataset (class) — Main interface for creating/opening netCDF files with support for groups, dimensions, and variables src/netCDF4/__init__.py
• Variable (class) — Represents netCDF variables with array data access and compression options src/netCDF4/__init__.py
• backend.py (module) — Custom build backend that conditionally adds mpi4py dependency based on netCDF C library features _build/backend.py
• netcdf4_has_parallel_support (function) — Detects whether underlying netCDF C library supports parallel I/O _build/utils.py
• utils module (module) — Utility functions for format conversion and data manipulation src/netCDF4/utils.py
• stringtochar (function) — Converts arrays of fixed-length strings to character arrays with extra dimension src/netCDF4/__init__.py
• chartostring (function) — Converts character arrays back to fixed-length string arrays src/netCDF4/__init__.py
• nc_complex plugin (plugin) — Provides complex number support as compound types or dimension-based storage external/nc_complex/

Science Pipeline

1. Array Creation — numpy.random.uniform or user data [varies by example → (n1dim, n2dim, n3dim, n4dim)] examples/bench.py
2. Compression Processing — zlib/shuffle compression with optional least_significant_digit quantization [(n1dim, n2dim, n3dim, n4dim) → compressed binary] src/netCDF4/__init__.py
3. NetCDF Storage — netCDF C library file I/O with HDF5 backend [compressed binary → hierarchical file format] src/netCDF4/__init__.py
4. Array Reading — C library read through numpy array interface [hierarchical file format → (n1dim, n2dim, n3dim, n4dim)] src/netCDF4/__init__.py

Assumptions & Constraints

• [warning] Assumes 4D array with fixed dimensions (n1dim, n2dim, n3dim, n4dim) but no validation of input array shape (shape)
• [info] Assumes numpy dtypes map directly to netCDF types but some conversions may be lossy (dtype)
• [warning] Assumes fixed-length strings have consistent encoding but doesn't validate character encoding (format)
• [info] Complex numbers stored as compound types assume specific field names ('r', 'i') without validation (format)

Related Library Repositories

Frequently Asked Questions

Analyzed on March 31, 2026 by CodeSea. Written by Karolina Sarna.