Hidden Assumptions in growthbook
13 assumptions this code never checks · 4 critical · spanning Domain, Shape, Temporal, Contract, Resource, Scale, Ordering, Environment
Every codebase relies on things it never checks. Most of them are routine. CodeSea looked at growthbook/growthbook and picked out the few most likely to cause trouble. The full list is just below.
Most of what this code assumes is routine. These 3 are the ones most likely to cause trouble here. The rest are minor; they're under "Show everything".
License server API calls fail with authentication errors, breaking billing operations and payment updates, with no clear error handling for invalid credentials
Large seed values cause numpy.random.seed() to fail with ValueError, crashing bandit weight calculations and breaking experiment traffic allocation
When thousands of SDK instances refresh at once, backend API gets overwhelmed, causing 503 errors and stale feature flags serving to users
Show everything (10 more)
SDK feature requests contain project context in headers or query params, but router assumes this data is always present and valid
If this fails: Missing or invalid project context causes features to be served from wrong organization or default to empty feature set, leading to incorrect experiment assignments
packages/back-end/src/api/api.router.ts:feature endpoints
External data warehouse connections (BigQuery, Snowflake) have unlimited query capacity and always return results within Express request timeout
If this fails: Long-running metric aggregation queries time out or hit rate limits, causing statistical analysis to fail with incomplete data and blocking experiment result updates
packages/back-end/src/controllers/datasources.ts:data warehouse queries
min_variation_weight of 0.01 (1%) assumes experiments never need traffic allocation below this threshold, hardcoded in BanditConfig
If this fails: Experiments with many variations (>100) cannot allocate minimum viable traffic to each arm, causing statistical power issues and invalid bandit optimization
packages/stats/gbstats/bayesian/bandits.py:min_variation_weight
useExperiments hook returns experiments in consistent order for pagination, but assumes experiments array doesn't change between page renders
If this fails: User sees duplicate experiments or missing experiments when navigating pages if new experiments are added during browsing session
packages/front-end/pages/bandits/index.tsx:experiment filtering
Express trust proxy configuration assumes specific load balancer setup (AWS ALB, CloudFlare, etc.) without validating the proxy chain
If this fails: Incorrect client IP detection in production leads to wrong user attribution in experiments and security issues with rate limiting bypasses
packages/back-end/src/app.ts:EXPRESS_TRUST_PROXY_OPTS
Hardcoded cursor positions array assumes fixed UI dimensions and screen resolution, with exact pixel coordinates for interactive elements
If this fails: Animated cursors render in wrong positions on different screen sizes or when UI layout changes, breaking the managed warehouse onboarding flow
packages/front-end/pages/datasources/index.tsx:ManagedWarehouse cursors
TrackingCallback functions are synchronous or handle errors internally, but SDK assumes they never throw exceptions
If this fails: User-provided tracking callbacks that throw errors crash the SDK experiment evaluation, causing features to fallback to default values
packages/sdk-js/src/index.ts:TrackingCallback
UNLIMITED_USAGE.lastUpdated timestamp assumes current system time represents accurate billing period boundaries
If this fails: Server clock skew or timezone issues cause billing period calculations to be wrong, potentially allowing usage overages or incorrect quota enforcement
packages/back-end/src/enterprise/billing/index.ts:UNLIMITED_USAGE
Demo datasource exports assume specific data schema matches production warehouse table structures exactly
If this fails: Users testing with demo data get different statistical results than production, leading to incorrect experiment setup and misleading A/B test outcomes
packages/shared/src/demo-datasource/index.ts:demo data format
Design system page loads all component stories simultaneously, assuming browser can handle rendering dozens of complex React components at once
If this fails: Design system page becomes unresponsive on lower-end devices or with limited memory, making it impossible to preview UI components during development
packages/front-end/pages/design-system/index.tsx:component stories
See the full structural analysis of growthbook: the pipeline, data models, and system behavior that put these assumptions in context.
Full analysis of growthbook/growthbook →Frequently Asked Questions
What does growthbook assume that could break in production?
The one most likely to cause trouble: CLOUD_SECRET environment variable contains valid credentials for license server authentication, used in payment method operations without validation If this fails, License server API calls fail with authentication errors, breaking billing operations and payment updates, with no clear error handling for invalid credentials
How many hidden assumptions does growthbook have?
CodeSea found 13 assumptions growthbook relies on but never validates, 4 of them critical, spanning Domain, Shape, Temporal, Contract, Resource, Scale, Ordering, Environment. Most are routine — the analysis flags the two or three most likely to actually bite.
What is a hidden assumption?
Something the code depends on but never checks: a data shape, an ordering, an environment condition, a scale limit, or a contract with another service. It holds until the world it runs in changes, then fails silently.