posthog/posthog
🦔 PostHog is an all-in-one developer platform for building successful products. We offer product analytics, web analytics, session replay, error tracking, feature flags, experimentation, surveys, data warehouse, a CDP, and an AI product assistant to help debug your code, ship features faster, and keep all your usage and customer data in one stack.
12 hidden assumptions · 9-stage pipeline · 8 components
Like any codebase, this fullstack makes assumptions it never checks — most are routine. The ones worth your attention are below.
Collects user events across web and mobile apps, processes them for product analytics and feature flags
Events flow from client SDKs through Rust capture services that validate and enrich them, then into Kafka queues where consumers process them into ClickHouse for analytics and PostgreSQL for operational data. The Django API serves queries by translating HogQL to ClickHouse SQL, while the React frontend provides real-time dashboards and configuration interfaces.
Under the hood, the system uses 3 feedback loops, 5 data pools, 4 control points to manage its runtime behavior.
A 8-component fullstack. 15551 files analyzed. Data flows through 9 distinct pipeline stages.
Hidden Assumptions
Most of what this code assumes is routine. These 3 are the ones most likely to cause trouble here. The rest are minor; they're under "Show everything".
If running in a browser environment or restricted Node.js context without stdin access, the plugin transpiler silently hangs waiting for input that never comes
If webpack.config.js changes its export structure or createEntry signature, Storybook build fails with property access errors on undefined objects
If events arrive out of order due to network delays or buffering, the replay timeline becomes scrambled, showing user actions in wrong sequence without any reordering logic
Show everything (9 more)
Assumes filesystem has sufficient space and permissions for webpack filesystem cache, and CI environment variable accurately indicates memory constraints
If this fails: In CI with limited disk space or wrong CI detection, either runs out of disk space with caching enabled or suffers slow builds with caching disabled unnecessarily
common/storybook/.storybook/main.ts:cache
Assumes event.type values 2,3,4,5 correspond exactly to specific transformer functions and that no new event types are introduced
If this fails: When new mobile event types are added or existing type numbers change, events silently pass through untransformed, causing replay corruption or missing functionality
common/replay-shared/src/mobile/index.ts:transformEventToWeb
Assumes 'pnpm' command is available in PATH and 'tsx' can execute TypeScript files, with shell access enabled
If this fails: In environments without pnpm (using npm/yarn) or restricted shell access, token rebuilds fail silently and CSS custom properties become stale until manual restart
packages/quill/apps/storybook/.storybook/main.ts:quillTokensWatcher
Assumes file change events arrive within reasonable intervals and 100ms debounce is sufficient to batch multiple file changes
If this fails: During rapid file changes or slow filesystem events, rebuild triggers repeatedly causing resource exhaustion, or changes get lost if debounce window is too short
packages/quill/apps/storybook/.storybook/main.ts:rebuildTimer
Assumes presets object has entries for 'site' and 'frontend' types with consistent structure containing wrapper and options properties
If this fails: If presets configuration is missing required types or has different property structure, transpilation fails with property access errors
common/plugin_transpiler/src/index.ts:presets
Assumes all valid replay events must have 'type' and 'timestamp' properties, and that these are sufficient to identify processable events
If this fails: Events with valid structure but missing these fields are discarded, causing gaps in replay data, while malformed events with only these fields may crash downstream processing
common/replay-shared/src/mobile/index.ts:couldBeEventWithTime
Assumes mobile replay data arrays fit in memory and that reduce operation can process all events synchronously without memory pressure
If this fails: For very long sessions with thousands of events, the transformation consumes excessive memory and may cause browser tab crashes or Node.js heap exhaustion
common/replay-shared/src/mobile/index.ts:transformToWeb
Assumes file watcher can monitor unlimited files in tokens directory and that filesystem events are reliable
If this fails: On systems with file watcher limits or unreliable filesystem events, token changes may not trigger rebuilds, leaving stale CSS in development
packages/quill/apps/storybook/.storybook/main.ts:server.watcher
Assumes CI environment is accurately detected by process.env.CI and that CI environments always have memory constraints requiring cache disabling
If this fails: In CI environments with ample memory but CI=true, builds run slower without filesystem cache, while memory-constrained non-CI environments may crash with caching enabled
common/storybook/.storybook/main.ts:process.env.CI
Open the standalone hidden-assumptions report for posthog →
How Data Flows Through the System
Events flow from client SDKs through Rust capture services that validate and enrich them, then into Kafka queues where consumers process them into ClickHouse for analytics and PostgreSQL for operational data. The Django API serves queries by translating HogQL to ClickHouse SQL, while the React frontend provides real-time dashboards and configuration interfaces.
- Event Capture — CaptureEndpoint receives HTTP POST requests containing event data from PostHog JavaScript SDK or server libraries, validates JSON structure and extracts team token for authentication
- Event Validation — EventsProcessor validates the captured event against team configuration, enriches it with person properties and session context, and prepares it for downstream processing [CapturedEvent → Event]
- Kafka Publishing — Validated events are published to appropriate Kafka topics (events, session_recording_events, plugin_log_entries) for asynchronous processing by specialized consumers [Event]
- Event Processing — Kafka consumers (ClickHouseEventsConsumer, SessionRecordingIngester) read events from topics, apply additional transformations, and write to final storage destinations [Event]
- Person Merging — PersonManager processes identify events to link distinct_ids to the same person, merging properties and ensuring consistent identity across sessions [Event → Person]
- Query Parsing — HogQLParser converts user-written HogQL queries into Abstract Syntax Trees, validating syntax and preparing for translation to ClickHouse SQL
- Analytics Queries — QueryExecutor translates HogQL ASTs to optimized ClickHouse SQL with automatic team filtering, executes queries via ClickHouseClient, and formats results for API response [HogQLQuery]
- Flag Evaluation — FeatureFlagMatcher evaluates feature flag conditions against user properties and cohort membership in real-time, returning which flags are active for the request [FeatureFlag]
- Session Processing — SessionRecordingIngester processes rrweb snapshots from Kafka, compresses and stores them in object storage (S3), and updates session metadata for playback indexing [SessionRecording]
Data Models
The data structures that flow between stages — the contracts that hold the system together.
posthog/models/event.pyDjango model with uuid: UUID, event: str, distinct_id: str, team_id: int, timestamp: datetime, properties: dict, elements_chain: str
Created by capture service from incoming SDK data, enriched with session and user context, then stored in ClickHouse for querying
posthog/models/person.pyDjango model with id: UUID, team_id: int, properties: dict, created_at: datetime, version: int, is_identified: bool
Created when new distinct_id encountered, merged when identities linked, properties updated through identify calls
posthog/models/session_recording.pyDjango model with id: UUID, team_id: int, session_id: str, distinct_id: str, start_time: datetime, end_time: datetime, storage_version: str
Created from rrweb snapshots, snapshots stored in object storage, metadata indexed for querying and playback
posthog/models/feature_flag.pyDjango model with key: str, team_id: int, filters: dict, rollout_percentage: int, active: bool, deleted: bool
Created via API with targeting rules, evaluated by Rust service against incoming requests, changes logged as events
rust/capture/src/events.rsRust struct with token: String, distinct_id: String, event: String, properties: Value, timestamp: DateTime
Deserialized from HTTP request JSON, validated and enriched with team metadata, then serialized to Kafka
posthog/hogql/ast.pyAST node classes representing SELECT statements with from_table: Optional[JoinExpr], select: List[Expr], where: Optional[Expr]
Parsed from HogQL string syntax into AST, translated to ClickHouse SQL with team filtering, executed and results formatted
System Behavior
How the system operates at runtime — where data accumulates, what loops, what waits, and what controls what.
Data Pools
Stores all analytics events with optimized columnar structure for fast aggregation queries, partitioned by team and date
Stores user accounts, team configuration, feature flags, and other operational data requiring ACID transactions
Buffers events between capture and processing services, providing durability and allowing independent scaling of ingestion vs processing
Caches feature flags, person properties, and query results to reduce database load and improve response times
Stores session recording snapshots, exports, and other large binary data with compression and lifecycle policies
Feedback Loops
- Person Identity Resolution (recursive, balancing) — Trigger: Identify event with new distinct_id mapping. Action: PersonManager merges person records and updates all historical events to use canonical person_id. Exit: All distinct_ids resolved to single person.
- Query Result Caching (cache-invalidation, balancing) — Trigger: New events arrive that could affect cached query results. Action: Cache keys are invalidated based on team_id and time range, forcing fresh query execution. Exit: Cache TTL expires or explicit invalidation.
- Kafka Consumer Retry (retry, balancing) — Trigger: Consumer fails to process message due to temporary error. Action: Message is retried with exponential backoff, dead letter queue for persistent failures. Exit: Successful processing or maximum retry count reached.
Delays
- Event Processing Latency (async-processing, ~1-5 seconds) — Events appear in analytics dashboards with slight delay after capture
- ClickHouse Query Compilation (compilation, ~100-500ms) — First execution of complex queries takes longer due to query plan optimization
- Session Recording Processing (batch-window, ~10-60 seconds) — Recording snapshots are batched before compression and storage to optimize throughput
Control Points
- Feature Flag Rollout Percentage (threshold) — Controls: What percentage of users see new features without code deployment. Default: Configurable per flag 0-100%
- Kafka Consumer Batch Size (env-var) — Controls: How many events are processed together, affecting latency vs throughput. Default: Environment variable
- Query Timeout Settings (threshold) — Controls: Maximum time allowed for analytics queries before cancellation. Default: Configurable per query type
- Person Properties Sync (feature-flag) — Controls: Whether person properties are synced to ClickHouse for query performance. Default: Team-level setting
Technology Stack
Web framework providing REST APIs, admin interface, and request routing for the main application backend
Frontend library building the analytics dashboard with component-based UI and real-time data updates
Columnar database optimized for analytical queries, storing billions of events with fast aggregation performance
High-performance language for event capture services handling thousands of events per second with low latency
Message broker providing reliable event streaming between capture services and processing consumers
Relational database storing user accounts, configuration, and operational data requiring ACID transactions
In-memory cache storing feature flags, session data, and query results for fast access
Type-safe JavaScript providing compile-time error checking across the large React frontend codebase
Key Components
- CaptureEndpoint (gateway) — HTTP endpoint that receives events from PostHog JavaScript SDK and other client libraries, validates the payload structure and team authentication
rust/capture/src/endpoints/capture.rs - EventsProcessor (processor) — Enriches captured events with team metadata, person properties, and session context before publishing to Kafka for downstream consumption
rust/capture/src/processor.rs - HogQLParser (transformer) — Parses HogQL query strings into Abstract Syntax Trees, providing PostHog's SQL-like interface for analytics queries with automatic team filtering
posthog/hogql/parser.py - ClickHouseClient (adapter) — Manages connections to ClickHouse database, executes queries with proper escaping and team isolation, handles connection pooling and retries
posthog/clickhouse/client.py - FeatureFlagMatcher (processor) — Evaluates feature flag conditions against user properties and cohort membership, determining which flags should be active for a given request
rust/feature-flags/src/flag_matching.rs - SessionRecordingIngester (processor) — Consumes session recording events from Kafka, processes rrweb snapshots, stores them in object storage, and updates session metadata in PostgreSQL
posthog/session_recordings/consumer.py - QueryExecutor (executor) — Executes different types of analytics queries (trends, funnels, retention) by translating them to optimized ClickHouse SQL with caching
posthog/hogql_queries/query_runner.py - PersonManager (orchestrator) — Manages person identity resolution by merging distinct_ids when identify events occur, ensuring consistent person properties across sessions
posthog/models/person/manager.py
Package Structure
React-based web application providing the main PostHog dashboard and analytics interface
Django application handling API requests, user management, and core business logic
High-performance event capture and processing services written in Rust
Modular product features (analytics, replay, experiments, etc.) with their own APIs and UIs
Component library and design tokens for consistent UI across PostHog products
Shared utilities for HogQL parsing, replay processing, and build tools
Explore the interactive analysis
See the full architecture map, data flow, and code patterns visualization.
Analyze on CodeSeaRelated Fullstack Repositories
Frequently Asked Questions
What is posthog used for?
Collects user events across web and mobile apps, processes them for product analytics and feature flags posthog/posthog is a 8-component fullstack written in Python. Data flows through 9 distinct pipeline stages. The codebase contains 15551 files.
How is posthog architected?
posthog is organized into 4 architecture layers: Ingestion Layer, API Layer, Frontend Layer, Data Layer. Data flows through 9 distinct pipeline stages. This layered structure keeps concerns separated and modules independent.
How does data flow through posthog?
Data moves through 9 stages: Event Capture → Event Validation → Kafka Publishing → Event Processing → Person Merging → .... Events flow from client SDKs through Rust capture services that validate and enrich them, then into Kafka queues where consumers process them into ClickHouse for analytics and PostgreSQL for operational data. The Django API serves queries by translating HogQL to ClickHouse SQL, while the React frontend provides real-time dashboards and configuration interfaces. This pipeline design reflects a complex multi-stage processing system.
What technologies does posthog use?
The core stack includes Django (Web framework providing REST APIs, admin interface, and request routing for the main application backend), React (Frontend library building the analytics dashboard with component-based UI and real-time data updates), ClickHouse (Columnar database optimized for analytical queries, storing billions of events with fast aggregation performance), Rust (High-performance language for event capture services handling thousands of events per second with low latency), Kafka (Message broker providing reliable event streaming between capture services and processing consumers), PostgreSQL (Relational database storing user accounts, configuration, and operational data requiring ACID transactions), and 2 more. A focused set of dependencies that keeps the build manageable.
What system dynamics does posthog have?
posthog exhibits 5 data pools (ClickHouse Events Table, PostgreSQL Operational DB), 3 feedback loops, 4 control points, 3 delays. The feedback loops handle recursive and cache-invalidation. These runtime behaviors shape how the system responds to load, failures, and configuration changes.
What design patterns does posthog use?
4 design patterns detected: Event Sourcing, CQRS (Command Query Responsibility Segregation), Plugin Architecture, Multi-tenant Data Isolation.
Analyzed on April 20, 2026 by CodeSea. Written by Karolina Sarna.