Hidden Assumptions in dask
12 assumptions this code never checks · 5 critical · spanning Shape, Contract, Domain, Resource, Ordering, Scale, Temporal, Environment
Every codebase relies on things it never checks. Most of them are routine. CodeSea looked at dask/dask and picked out the few most likely to cause trouble. The full list is just below.
Most of what this code assumes is routine. These 3 are the ones most likely to cause trouble here. The rest are minor; they're under "Show everything".
If out_ind declares 3 dimensions but the function returns 2D arrays, or vice versa, chunk assembly will concatenate results with wrong axes, producing arrays with incorrect final shapes that pass validation but contain scrambled data
If layer tasks return arrays with different shapes than declared in chunks (e.g., chunks=((10, 10),) but tasks return 15-element arrays), array indexing and concatenation will fail with cryptic shape mismatch errors during compute()
Operations mixing arrays with different units (e.g., pixels vs. meters) or coordinate systems (e.g., row-major vs. column-major indexing) will execute without error but produce results in wrong coordinate spaces or with incorrect physical meanings
Show everything (9 more)
Converting dask arrays to numpy via __array__ assumes the result fits in memory but never checks available RAM against computed array size before triggering computation
If this fails: Large dask arrays that exceed available memory will trigger OOM kills or swap thrashing when converted to numpy arrays, with no warning until the system becomes unresponsive
dask/array/_array_expr/_collection.py:Array.__array__
TaskRef dependencies form a directed acyclic graph but never validates for cycles during task graph construction - circular dependencies will cause infinite loops during execution
If this fails: If task A references task B which references task A (directly or through a chain), the scheduler will loop indefinitely trying to resolve dependencies, hanging the computation with no error message
dask/_task_spec.py:Task
The func parameter is callable with the signature expected by the blockwise operation pattern but never validates function signatures match the declared input/output structure
If this fails: If func expects 2 arguments but blockwise provides 3, or func returns tuples when scalars are expected, tasks will fail at runtime with TypeError or ValueError after graph compilation and partial execution
dask/array/_array_expr/_blockwise.py:Blockwise._layer
Maximum chunk size calculation assumes chunk dimensions fit in standard integer types but uses max() on chunk tuples that could contain numpy.nan or extremely large values
If this fails: Arrays with chunks containing NaN sizes or dimensions exceeding integer limits will cause max() to return NaN or overflow, breaking downstream size calculations and memory allocation
dask/array/_array_expr/_expr.py:ArrayExpr.chunksize
Expression cache entries remain valid for the lifetime of the process but never invalidates cache when underlying data sources change or global configuration affecting computation changes
If this fails: If source arrays are modified in-place or dask configuration changes (like scheduler or chunk size), cached expressions may produce stale results that don't reflect current data or settings
dask/_expr.py:SingletonExpr._cache
The threads scheduler is available and functional but never checks if threading is disabled (e.g., in some embedded environments) or if thread pool creation will succeed
If this fails: In environments where threading is restricted or thread pool creation fails, array computations will fall back to synchronous execution without warning, causing severe performance degradation
dask/array/_array_expr/_collection.py:Array.__dask_scheduler__
Array dtype represents the numpy dtype of computed results but assumes all chunk operations preserve dtype consistency without validating intermediate results maintain the expected type
If this fails: Operations that change dtype during computation (e.g., integer overflow, precision loss in floating point) may produce final arrays with different dtypes than declared, breaking downstream code that assumes specific numeric types
dask/array/_array_expr/_expr.py:ArrayExpr.dtype
Graph layer tasks are compatible with the array slicing and indexing protocols but never validates that task functions support the getter interfaces used by array operations
If this fails: Custom tasks that don't support numpy-style indexing will fail when array slicing operations try to call getter functions on them, causing opaque AttributeError exceptions
dask/array/_array_expr/_io.py:FromGraph._layer
Task reference keys correspond to tasks that will exist in the same execution context but never validates that referenced tasks are actually present in the graph before execution begins
If this fails: TaskRefs pointing to non-existent keys will cause KeyError during execution rather than at graph construction time, making debugging harder as errors appear during parallel execution rather than at graph build time
dask/_task_spec.py:TaskRef
See the full structural analysis of dask: the pipeline, data models, and system behavior that put these assumptions in context.
Full analysis of dask/dask →Compare dask
Frequently Asked Questions
What does dask assume that could break in production?
The one most likely to cause trouble: The out_ind parameter correctly specifies output dimensions that align with the actual function output shape but never validates this alignment - a mismatch between declared output indices and actual function results will cause silent shape corruption If this fails, If out_ind declares 3 dimensions but the function returns 2D arrays, or vice versa, chunk assembly will concatenate results with wrong axes, producing arrays with incorrect final shapes that pass validation but contain scrambled data
How many hidden assumptions does dask have?
CodeSea found 12 assumptions dask relies on but never validates, 5 of them critical, spanning Shape, Contract, Domain, Resource, Ordering, Scale, Temporal, Environment. Most are routine — the analysis flags the two or three most likely to actually bite.
What is a hidden assumption?
Something the code depends on but never checks: a data shape, an ordering, an environment condition, a scale limit, or a contract with another service. It holds until the world it runs in changes, then fails silently.