Hidden Assumptions in Olive
13 assumptions this code never checks · 5 critical · spanning Environment, Resource, Temporal, Contract, Domain, Scale, Ordering
Every codebase relies on things it never checks. Most of them are routine. CodeSea looked at microsoft/olive and picked out the few most likely to cause trouble. The full list is just below.
Most of what this code assumes is routine. These 3 are the ones most likely to cause trouble here. The rest are minor; they're under "Show everything".
If the current Python interpreter moves or becomes unavailable after job starts, venv creation fails silently and jobs crash with obscure errors about missing Python
Large models (>1GB) running 3 concurrent optimizations can exhaust system memory, causing OOM kills or swap thrashing that makes all jobs fail
Long-running optimizations (large model quantization can take 2-4 hours) get their results purged before users can retrieve them, losing hours of computation
Show everything (10 more)
The mapping from implementation names to pass types stays synchronized with olive/cli/quantize.py selection logic
If this fails: When olive/cli/quantize.py adds new quantization methods or changes selection logic, MCP routes to wrong passes, causing optimization failures or suboptimal results
mcp/src/olive_mcp/packages.py:_IMPL_TO_PASS_TYPES
User home directory (~/.olive-mcp/venvs) is writable and has sufficient disk space for multiple virtual environments
If this fails: In containerized or restricted environments where home directory is read-only or has quota limits, venv creation fails and no optimization jobs can run
olive_mcp/constants.py:VENV_BASE
olive_config.json structure matches expected schema with pass types correctly categorized
If this fails: If olive_config.json format changes or pass types get reorganized, package resolution fails silently, causing optimization jobs to fail with missing dependency errors
mcp/src/olive_mcp/packages.py:_load_olive_config
Virtual environments not used for 14 days are safe to delete and users won't need them again
If this fails: Users returning to projects after vacation or long development cycles find their optimized environments purged, forcing expensive package reinstallation
mcp/src/olive_mcp/constants.py:_VENV_MAX_AGE_DAYS
RunConfig JSON schema can be serialized to string and fits in memory without size limits
If this fails: As Olive adds more passes and configuration options, the schema grows large enough to cause memory issues or hit string size limits during documentation generation
docs/source/dump_schema.py:RunConfig.schema_json
Azure DevOps organization 'aiinfra' remains accessible and API endpoints don't change
If this fails: If organization moves, gets renamed, or API versions change, the commit bisection script fails and debugging CI issues becomes manual
.azure_pipelines/scripts/find_failed_commit.py:_organization_url
Package config pass names map correctly to module names and the import_pass_module method exists
If this fails: Documentation generation fails when pass modules can't be imported, leaving configuration docs incomplete or with broken references
docs/source/exts/auto_config_doc/__init__.py:import_class
tools module import registers MCP tools and prompts before mcp.run() is called
If this fails: If import order changes or tools registration is delayed, MCP server starts without tools available, confusing users who get empty capability lists
mcp/src/olive_mcp/__init__.py:import olive_mcp.tools
Gallery template uses MyST markdown grid syntax that remains compatible with current Sphinx version
If this fails: Documentation build fails when Sphinx or MyST versions update and change grid syntax, breaking all gallery pages
docs/source/exts/gallery_directive.py:TEMPLATE_GRID
Script runs from gh-pages branch root where _static/versions.json exists and is writable
If this fails: Version updates fail silently if run from wrong directory or in environments where _static doesn't exist, leaving documentation with stale version lists
.azure_pipelines/scripts/update_versions_json.py:versions_file
See the full structural analysis of Olive: the pipeline, data models, and system behavior that put these assumptions in context.
Full analysis of microsoft/olive →Frequently Asked Questions
What does Olive assume that could break in production?
The one most likely to cause trouble: Python executable path returned by sys.executable is stable and available for subprocess creation throughout job execution lifecycle If this fails, If the current Python interpreter moves or becomes unavailable after job starts, venv creation fails silently and jobs crash with obscure errors about missing Python
How many hidden assumptions does Olive have?
CodeSea found 13 assumptions Olive relies on but never validates, 5 of them critical, spanning Environment, Resource, Temporal, Contract, Domain, Scale, Ordering. Most are routine — the analysis flags the two or three most likely to actually bite.
What is a hidden assumption?
Something the code depends on but never checks: a data shape, an ordering, an environment condition, a scale limit, or a contract with another service. It holds until the world it runs in changes, then fails silently.