Hidden Assumptions in evidence
12 assumptions this code never checks · 4 critical · spanning Resource, Shape, Domain, Contract, Ordering, Environment, Scale, Temporal
Every codebase relies on things it never checks. Most of them are routine. CodeSea looked at evidence-dev/evidence and picked out the few most likely to cause trouble. The full list is just below.
Most of what this code assumes is routine. These 3 are the ones most likely to cause trouble here. The rest are minor; they're under "Show everything".
On systems with less RAM or with very long existing NODE_OPTIONS, Evidence projects will fail to start with out-of-memory errors or command line argument overflow
If data array is empty or data[0] is undefined, Object.keys(data[0]) throws 'Cannot convert undefined or null to object' error, crashing query execution
If later rows have additional columns not present in data[0], those columns are silently ignored in column type inference, leading to incomplete schema detection
Show everything (9 more)
All files tracked in originalFiles, createdFiles, and deletedFiles still exist and are writable when restoration occurs, and the process has not been interrupted between tracking and restoration
If this fails: If files were externally deleted or made read-only, or process crashed before restoration, test cleanup fails silently leaving orphaned files and modified content in the test environment
e2e/hmr/tests/fs-utils.js:restoreChangedFiles
Template population completes before any file watchers start monitoring the .evidence/template directory, and clearQueryCache() completes before template files are copied
If this fails: Race condition where file watchers detect template files being written during population, triggering unnecessary rebuilds or corrupting the template structure mid-copy
packages/evidence/cli.js:populateTemplate
Chokidar can successfully watch all specified file patterns and the filesystem supports inotify/FSEvents, with sufficient file descriptor limits for the number of watched files
If this fails: On systems with exhausted file descriptors or filesystems without change notification support, file watching silently fails causing hot reload to stop working without user notification
packages/evidence/cli.js:runFileWatcher
File system has sufficient disk space for file modifications and the process has write permissions to all files being tracked in the Maps
If this fails: When disk is full or permissions are revoked mid-test, file modifications fail but tracking data remains inconsistent, leading to incomplete rollback attempts
e2e/hmr/tests/fs-utils.js:editFile
JavaScript data arrays are small enough to fit in memory multiple times (original + processed output) and Object.keys() operation on data[0] completes quickly
If this fails: Large datasets cause memory exhaustion during column type inference or long blocking operations that freeze the Evidence build process
packages/datasources/javascript/index.js:getRunner
The template source directory structure at path.join(__dirname, '/template') matches the expected Evidence project structure and all template files are valid
If this fails: If template directory is corrupted or incomplete, Evidence projects will be created with missing critical files, causing mysterious runtime failures in generated sites
packages/evidence/cli.js:populateTemplate
Files marked for deletion in deletedFiles Map haven't been modified between deletion and restoration, and the content read before deletion still represents the correct state
If this fails: If external processes modify files after Evidence deletes them but before restoration, the restored content will be stale, potentially losing important changes made during the test
e2e/hmr/tests/fs-utils.js:deleteFile
Shell execution is available and secure on the target system with shell: true, and spawn operations won't exceed system process limits
If this fails: On systems without shell access or with restricted process spawning, Evidence e2e project creation fails with unclear error messages about command execution
e2e/create-e2e-project.js:globalSpawnOpts
Markdown files follow Evidence naming conventions where index.md becomes +page.md and other .md files become directory/+page.md, and path transformations don't create conflicts
If this fails: Non-standard file naming or deeply nested structures could cause path transformation to create duplicate +page.md files or invalid directory structures in the template
packages/evidence/cli.js:runFileWatcher
See the full structural analysis of evidence: the pipeline, data models, and system behavior that put these assumptions in context.
Full analysis of evidence-dev/evidence →Frequently Asked Questions
What does evidence assume that could break in production?
The one most likely to cause trouble: 4096MB of system memory is available and NODE_OPTIONS environment variable won't exceed shell command line limits when appended to If this fails, On systems with less RAM or with very long existing NODE_OPTIONS, Evidence projects will fail to start with out-of-memory errors or command line argument overflow
How many hidden assumptions does evidence have?
CodeSea found 12 assumptions evidence relies on but never validates, 4 of them critical, spanning Resource, Shape, Domain, Contract, Ordering, Environment, Scale, Temporal. Most are routine — the analysis flags the two or three most likely to actually bite.
What is a hidden assumption?
Something the code depends on but never checks: a data shape, an ordering, an environment condition, a scale limit, or a contract with another service. It holds until the world it runs in changes, then fails silently.