scipy/scipy
SciPy library main repository
12 hidden assumptions · 4-stage pipeline · 6 components
Like any codebase, this library makes assumptions it never checks — most are routine. The ones worth your attention are below.
Scientific computing library providing algorithms for optimization, linear algebra, integration, and statistics
Data enters SciPy through domain-specific module APIs where NumPy arrays are validated and optionally converted between array backends. The data flows through optimized algorithms implemented in C/Cython/Fortran that leverage BLAS/LAPACK for linear algebra operations. Results are packaged into standardized result objects (OptimizeResult, etc.) with metadata about the computation performed. Throughout this flow, the ccallback system enables Python functions to be embedded as callbacks within native code execution.
Under the hood, the system uses 3 feedback loops, 2 data pools, 3 control points to manage its runtime behavior.
A 6-component library. 1610 files analyzed. Data flows through 4 distinct pipeline stages.
Hidden Assumptions
Most of what this code assumes is routine. These 3 are the ones most likely to cause trouble here. The rest are minor; they're under "Show everything".
If HAVE_BLAS_ILP64 is incorrectly set during build (e.g., linking against 32-bit BLAS with ILP64 flag set), function calls will use wrong symbol names causing undefined symbol errors or silent data corruption when array indices exceed 32-bit limits
If MKL changes its symbol naming convention or if the fix flag is applied to non-MKL libraries, linker errors will occur or wrong functions will be called silently
Direct calls to 'w'-prefixed wrappers from Cython/F2PY code will cause segfaults on x86 machines because return value ABI is incompatible - the calling code receives garbage data instead of proper complex values
Show everything (9 more)
This header must be included after fortran_defs.h because it redefines the F_FUNC macro, but there's no enforcement of this include order dependency
If this fails: If included in wrong order, F_FUNC macro will have wrong definition leading to incorrect symbol name mangling and linker errors for all BLAS/LAPACK function calls
scipy/_build_utils/src/_blas64_defines.h:include order
The NumPy C API import_array() function has been called in the current scope before including this header, as stated in the comment that 'version dependency will misbehave' otherwise
If this fails: Runtime version detection will fail silently, potentially causing crashes when NumPy 2.0 features are used with NumPy 1.x or vice versa
scipy/_build_utils/src/npy_2_compat.h:import_array requirement
Version strings follow PEP 440 format with optional epoch, release numbers, pre/post/dev suffixes, and local version identifiers, but the VERSION_PATTERN regex is the only validation
If this fails: Malformed version strings that don't match expected patterns will raise InvalidVersion exceptions, but edge cases in the regex could allow through invalid versions that cause incorrect version comparisons
scipy/_external/packaging_version/src/version.py:version string format
The operating system cache directory selected by pooch.os_cache() is writable and has sufficient disk space for dataset files, with no validation of disk space or write permissions
If this fails: Dataset downloads will fail with unclear error messages if cache directory is read-only, full, or on a filesystem that doesn't support the file sizes being downloaded
scipy/datasets/_fetchers.py:pooch.os_cache
The scipy module version string in sys.modules['scipy'].__version__ is available and properly formatted for HTTP User-Agent header construction
If this fails: If scipy version is malformed or missing, HTTP requests could fail due to invalid User-Agent header, causing dataset downloads to be rejected by servers that validate headers
scipy/datasets/_fetchers.py:HTTPDownloader with User-Agent
Python callback functions provided by users match the expected C function signature defined in ccallback_signature_t, but signature validation only happens at callback registration time
If this fails: If user provides callback with wrong argument types or count, the callback will crash during execution within C/Fortran code, potentially corrupting the Python interpreter state
scipy/_lib/src/ccallback.h:callback signature matching
When ACCELERATE_NEW_LAPACK is defined, the build environment has macOS SDK 13.3 or later (__MAC_OS_X_VERSION_MAX_ALLOWED >= 130300), but this is only checked at compile time
If this fails: Building with older macOS SDK while ACCELERATE_NEW_LAPACK is enabled will cause compilation errors, and the error only appears during build rather than being caught by configure/setup
scipy/_build_utils/src/scipy_blas_defines.h:macOS SDK version
Version comparison operations will never encounter arithmetic overflow when comparing against Infinity/NegativeInfinity objects, and that Python's object comparison protocol handles mixed types correctly
If this fails: If version components become extremely large numbers that cause comparison overflow, or if unexpected object types are compared against Infinity, version ordering could become inconsistent leading to wrong dependency resolution
scipy/_external/packaging_version/src/_structures.py:InfinityType comparison
The registry and registry_urls imported from _registry follow expected format with dataset names as keys and properly formatted URLs and hashes as values
If this fails: Malformed registry entries could cause pooch to attempt downloads with invalid URLs or fail hash verification with cryptic error messages that don't indicate the source of the problem
scipy/datasets/_fetchers.py:registry format
Open the standalone hidden-assumptions report for scipy →
How Data Flows Through the System
Data enters SciPy through domain-specific module APIs where NumPy arrays are validated and optionally converted between array backends. The data flows through optimized algorithms implemented in C/Cython/Fortran that leverage BLAS/LAPACK for linear algebra operations. Results are packaged into standardized result objects (OptimizeResult, etc.) with metadata about the computation performed. Throughout this flow, the ccallback system enables Python functions to be embedded as callbacks within native code execution.
- Array Input Processing — User arrays are validated for shape, dtype, and memory layout requirements, then potentially converted between array backends using the array API compatibility layer [NumPy Array → NumPy Array]
- Python Function Wrapping — When user provides Python callables (objective functions, constraints), the ccallback system creates C-compatible function pointers that can invoke Python code from native algorithms
- Algorithm Computation — Core algorithms execute in C/Cython/Fortran code, calling BLAS/LAPACK through the wrapper system for linear algebra operations and invoking user callbacks through ccallback infrastructure [NumPy Array → NumPy Array]
- Result Packaging — Algorithm outputs are wrapped in domain-specific result objects that standardize return values and include metadata like convergence status, iteration counts, and error estimates [NumPy Array → OptimizeResult]
Data Models
The data structures that flow between stages — the contracts that hold the system together.
numpy (external dependency)ndarray with dtype-specific data (float64, complex128, etc.), shape tuple, and optional masks/metadata
Created from user input, passed through domain-specific algorithms that operate in-place or create new arrays, returned as computation results
scipy/_lib/src/ccallback.hC struct containing function pointer, signature metadata, Python object references, and error handling state for low-level callbacks
Created when Python functions need to be called from C/Fortran code, holds references during computation, cleaned up after callback completion
scipy/optimize/_optimize.pydict-like object with standardized fields: x (solution array), fun (objective value), success (bool), message (str), nit (iterations)
Constructed by optimization algorithms to package results in a consistent format, returned to users with solver-specific additional fields
scipy/integrate/_cubature.pydataclass with estimate: Array, error: Array, status: str, regions: list[CubatureRegion], subdivisions: int, atol: float, rtol: float
Built incrementally during adaptive integration as regions are subdivided and error estimates refined, finalized when tolerance met or max subdivisions reached
scipy/stats/_hypotests.pydataclass with statistic: float, pvalue: float, and optional method-specific fields like degrees of freedom or confidence intervals
Computed by statistical test functions from input data, packages test statistic and p-value with metadata about the test performed
System Behavior
How the system operates at runtime — where data accumulates, what loops, what waits, and what controls what.
Data Pools
Local file cache managed by Pooch that stores downloaded example datasets with SHA256 verification to avoid repeated network requests
Cached detection results for BLAS libraries, compiler capabilities, and platform-specific build settings used during compilation
Feedback Loops
- Optimization iterations (convergence, balancing) — Trigger: Gradient/objective value computation. Action: Update parameter estimates using algorithm-specific rules (gradient descent, Newton methods, etc.). Exit: Convergence criteria met or maximum iterations reached.
- Adaptive integration subdivision (recursive, reinforcing) — Trigger: Error estimate exceeds tolerance. Action: Split integration region into smaller subregions and compute estimates for each. Exit: Global error estimate within tolerance or maximum subdivisions reached.
- Dataset download retry (retry, balancing) — Trigger: Network failure or hash mismatch. Action: Reattempt download with exponential backoff. Exit: Successful download with verified hash or max retries exceeded.
Delays
- JIT compilation warmup (warmup, ~first call only) — Initial function calls slower due to code generation and optimization
- Dataset download (async-processing, ~varies by file size) — First access to each dataset requires network download and caching
- BLAS library loading (compilation, ~import time) — Dynamic library loading and symbol resolution happens at module import
Control Points
- BLAS library selection (architecture-switch) — Controls: Which BLAS implementation (OpenBLAS, MKL, ATLAS) is used for linear algebra operations. Default: build-time detection
- Integer precision mode (precision-mode) — Controls: Whether to use ILP64 (64-bit integers) or LP64 (32-bit integers) for BLAS/LAPACK calls. Default: HAVE_BLAS_ILP64 compile flag
- Array backend selection (runtime-toggle) — Controls: Which array library (NumPy, CuPy, JAX) is used for computations. Default: automatic detection from input arrays
Technology Stack
Build system that handles cross-platform compilation, dependency detection, and integration with BLAS/LAPACK libraries
Compiles Python-like code to C extensions, providing performance while maintaining Python compatibility for algorithm implementations
Provides optimized linear algebra operations through vendor-specific implementations (OpenBLAS, MKL, ATLAS) with ABI compatibility handling
Foundational array library that provides the core data structures and basic operations for all SciPy algorithms
Handles dataset downloading, caching, and verification with SHA256 hashes for the scipy.datasets module
Provides C++ to Python bindings for components that need to interface with C++ libraries or use C++ features
Key Components
- ccallback system (adapter) — Enables Python functions to be called from C/Fortran code by creating compatible function pointers and managing the Python interpreter state during callbacks
scipy/_lib/src/ccallback.h - BLAS/LAPACK wrapper system (adapter) — Handles Fortran name mangling, ABI compatibility issues, and ILP64 vs LP64 integer size differences across different BLAS/LAPACK implementations
scipy/_build_utils/src/ - Array API compatibility layer (adapter) — Provides a unified interface for different array backends (NumPy, CuPy, JAX) allowing SciPy algorithms to work with various array implementations
scipy/_lib/_array_api.py - Dataset fetcher (loader) — Downloads and caches example datasets from remote repositories using Pooch, with SHA256 verification and local caching to avoid repeated downloads
scipy/datasets/_fetchers.py - Build utilities (factory) — Generates platform-specific build configurations, detects available BLAS libraries, and creates compatibility shims for different Fortran compilers
scipy/_build_utils/ - Version compatibility system (validator) — Handles version parsing and comparison for dependency management, supporting complex version schemes with epochs, pre-releases, and local versions
scipy/_external/packaging_version/
Explore the interactive analysis
See the full architecture map, data flow, and code patterns visualization.
Analyze on CodeSeaCompare scipy
Related Library Repositories
Frequently Asked Questions
What is scipy used for?
Scientific computing library providing algorithms for optimization, linear algebra, integration, and statistics scipy/scipy is a 6-component library written in Python. Data flows through 4 distinct pipeline stages. The codebase contains 1610 files.
How is scipy architected?
scipy is organized into 4 architecture layers: Domain Modules, Core Infrastructure, External Dependencies, Build System. Data flows through 4 distinct pipeline stages. This layered structure keeps concerns separated and modules independent.
How does data flow through scipy?
Data moves through 4 stages: Array Input Processing → Python Function Wrapping → Algorithm Computation → Result Packaging. Data enters SciPy through domain-specific module APIs where NumPy arrays are validated and optionally converted between array backends. The data flows through optimized algorithms implemented in C/Cython/Fortran that leverage BLAS/LAPACK for linear algebra operations. Results are packaged into standardized result objects (OptimizeResult, etc.) with metadata about the computation performed. Throughout this flow, the ccallback system enables Python functions to be embedded as callbacks within native code execution. This pipeline design keeps the data transformation process straightforward.
What technologies does scipy use?
The core stack includes Meson (Build system that handles cross-platform compilation, dependency detection, and integration with BLAS/LAPACK libraries), Cython (Compiles Python-like code to C extensions, providing performance while maintaining Python compatibility for algorithm implementations), BLAS/LAPACK (Provides optimized linear algebra operations through vendor-specific implementations (OpenBLAS, MKL, ATLAS) with ABI compatibility handling), NumPy (Foundational array library that provides the core data structures and basic operations for all SciPy algorithms), Pooch (Handles dataset downloading, caching, and verification with SHA256 hashes for the scipy.datasets module), pybind11 (Provides C++ to Python bindings for components that need to interface with C++ libraries or use C++ features). A focused set of dependencies that keeps the build manageable.
What system dynamics does scipy have?
scipy exhibits 2 data pools (Dataset Cache, Build Configuration Cache), 3 feedback loops, 3 control points, 3 delays. The feedback loops handle convergence and recursive. These runtime behaviors shape how the system responds to load, failures, and configuration changes.
What design patterns does scipy use?
4 design patterns detected: Fortran ABI Compatibility Wrappers, Build-time Feature Detection, Standardized Result Objects, Callback Thunks.
How does scipy compare to alternatives?
CodeSea has side-by-side architecture comparisons of scipy with scikit-learn. These comparisons show tech stack differences, pipeline design, system behavior, and code patterns. See the comparison pages above for detailed analysis.
Analyzed on April 20, 2026 by CodeSea. Written by Karolina Sarna.