openreplay/openreplay
Session replay, cobrowsing and product analytics you can self-host. Best for reproducing issues and iterating on your product.
13 hidden assumptions · 6-stage pipeline · 8 components
Like any codebase, this fullstack makes assumptions it never checks — most are routine. The ones worth your attention are below.
Records user sessions to replay their exact browser interactions for debugging
User interactions in browsers are captured by JavaScript trackers that serialize events into binary messages. These messages flow through Kafka queues to Go services that deserialize, batch, and insert them into ClickHouse time-series tables. The API layer queries this stored data to reconstruct session timelines, which the React frontend renders as playable videos. Real-time assistance bypasses storage, routing live events directly through WebSockets.
Under the hood, the system uses 3 feedback loops, 4 data pools, 4 control points to manage its runtime behavior.
A 8-component fullstack. 3209 files analyzed. Data flows through 6 distinct pipeline stages.
Hidden Assumptions
Most of what this code assumes is routine. These 3 are the ones most likely to cause trouble here. The rest are minor; they're under "Show everything".
If Kafka partitioning fails or messages for a session arrive across multiple partitions, the session timeout logic will see events out of order, potentially ending sessions prematurely or extending them incorrectly, corrupting session duration metrics
Sessions for mobile apps with background behavior or slow networks get artificially terminated early, while bot traffic or automated tests create artificially long sessions, skewing analytics and potentially losing real user data
Asset caching grows unbounded over time, eventually exhausting disk space and causing the entire asset service to fail, breaking session replay functionality when screenshots and source maps can't be retrieved
Show everything (10 more)
The MessageSizeLimit configuration adequately handles the largest possible user interactions, but there's no validation that message producers respect this limit or that downstream consumers can handle messages approaching this size
If this fails: Large DOM mutations or massive JavaScript payloads exceed the limit, causing message rejection at Kafka ingestion but silent data loss since the application continues processing without the oversized events, resulting in incomplete session replays
backend/cmd/db/main.go:cfg.MessageSizeLimit
The process.env.maxHttpBufferSize environment variable, if present, contains a valid numeric value that can be parsed as a float and converted to bytes by multiplying by 1e6
If this fails: If maxHttpBufferSize contains non-numeric data, parseFloat returns NaN, causing WebSocket connections to use NaN * 1e6 as buffer size, which becomes 0 bytes, immediately dropping all real-time assistance data and breaking live support features
assist/utils/wsServer.js:createSocketIOServer
ClickHouse event records always contain the expected performance timing properties (dom_content_loaded_event_time, first_contentful_paint_time, etc.) as valid numeric strings that can be cast to Float32
If this fails: If browser events omit timing data or send non-numeric values (like 'undefined' or null), the CAST operations produce null values that NULLIF removes, causing performance alert thresholds to evaluate against incomplete data and miss critical performance regressions
api/chalicelib/core/alerts/alerts_processor.py:LeftToDb
All binary message data passed to the canvas image iterator contains valid message headers with correctly formatted message types and session IDs that can be deserialized by messages.ReadMessage()
If this fails: Malformed binary data from network corruption or version mismatches causes ReadMessage to panic, crashing the entire canvas service and preventing screenshot generation for all sessions until the service restarts
backend/cmd/canvases/main.go:messages.NewImagesMessageIterator
Message timestamps in the stream represent the actual time when user events occurred in the browser, not when they were processed by the backend, allowing accurate session timeout calculations
If this fails: Network delays or message queue backlogs cause timestamps to represent processing time rather than event time, making session timeouts fire based on system lag rather than user inactivity, incorrectly ending active sessions during high load
backend/cmd/ender/main.go:sessionEndGenerator.UpdateSession
A 20-minute cleanup interval is sufficient to prevent memory leaks from accumulated asset cache entries, and the system can handle all cleanup operations within a single 20-minute window
If this fails: During high traffic periods with many unique URLs, memory usage grows faster than the 20-minute cleanup can process, eventually causing out-of-memory crashes in the assets service and losing cached data needed for session replay
backend/cmd/assets/main.go:tick = time.Tick(20 * time.Minute)
All Message structs deserialized from Kafka contain session IDs that exist as valid sessions in the sessions manager and can be looked up without error
If this fails: Race conditions where messages arrive after a session is deleted from Redis cause datasaver to fail session lookups, dropping events for recently ended sessions and creating gaps in session replay data near session boundaries
backend/internal/db/datasaver/datasaver.go:saver processing
The JWT secret key is provided through configuration and contains sufficient entropy to prevent brute force attacks, but there's no validation of key strength or rotation requirements
If this fails: Weak or default JWT secrets allow attackers to forge authentication tokens, gaining unauthorized access to session data and user recordings from any project in the system
backend/cmd/api/main.go:cfg.JWTSecret
The hardcoded msgFilter array contains all message types that the datasaver needs to process, and this list remains synchronized with message types generated by the tracker without any automated verification
If this fails: When new message types are added to the tracker, they won't be filtered into the database consumer, causing new event data to be silently ignored and creating incomplete session replays missing the newest interaction types
backend/cmd/db/main.go:queue.NewConsumer msgFilter
The number of concurrent WebSocket connections per room stays within Socket.IO's default memory limits, and room-based operations complete within reasonable time bounds
If this fails: Large assistance sessions with many concurrent agents cause fetchSockets() operations to timeout or consume excessive memory, degrading performance for all assistance sessions and potentially causing the entire assist service to become unresponsive
assist/utils/wsServer.js:io.in(roomID).fetchSockets()
Open the standalone hidden-assumptions report for openreplay →
How Data Flows Through the System
User interactions in browsers are captured by JavaScript trackers that serialize events into binary messages. These messages flow through Kafka queues to Go services that deserialize, batch, and insert them into ClickHouse time-series tables. The API layer queries this stored data to reconstruct session timelines, which the React frontend renders as playable videos. Real-time assistance bypasses storage, routing live events directly through WebSockets.
- Capture browser events — JavaScript tracker observes DOM mutations, mouse clicks, keyboard inputs, network requests, console logs, and performance metrics, serializing each event with timestamp and session context into binary Message format [Browser events → Message]
- Stream to message queues — Tracker batches binary messages and sends them to Kafka topics (TopicRawWeb, TopicRawMobile) partitioned by session ID to maintain event ordering [Message → Message]
- Process message streams — MessageIterator in db service consumes Kafka streams, deserializes binary data back into Go Message structs, and filters by configured message types [Message → Message]
- Batch and store events — DataSaver accumulates messages into batches, transforms them to ClickHouse-specific schemas, and performs bulk inserts into time-partitioned tables for fast querying [Message → ClickHouse Event Record]
- Query session data — API services receive frontend requests with session IDs and time ranges, query ClickHouse for matching events, and reconstruct chronological event sequences [API Request/Response → ClickHouse Event Record]
- Render session playback — SessionPlayer component fetches event sequences from API, builds virtual DOM timeline, and renders events as synchronized video playback with timeline controls [ClickHouse Event Record → Rendered session playback]
Data Models
The data structures that flow between stages — the contracts that hold the system together.
backend/pkg/messages/Go structs with message type (uint), timestamp (uint64), session ID (uint64), and type-specific payload (various fields like URL, element selector, mouse coordinates)
Created by JavaScript tracker when user interactions occur, serialized to binary format, sent through Kafka queues, deserialized by Go services, and stored in ClickHouse
assist/utils/assistHelper.jsJavaScript object with pageTitle: string, active: boolean, sessionID: string, userID: string, projectKey: string, userOs/userBrowser/userDevice: string, location data, timestamp: number
Built by tracker from browser environment, sent via WebSocket handshake, used to route assistance requests to appropriate agents
backend/internal/db/datasaver/ClickHouse table rows with session_id: UInt64, timestamp: DateTime64, message_type: UInt8, project_id: UInt32, plus event-specific columns for URLs, selectors, values, errors
Generated from Message structs by datasaver, bulk-inserted into ClickHouse time-series tables, queried by API to reconstruct session timelines
api/schemas/Pydantic models with typed fields like project_id: int, session_id: str, filters: dict, pagination: page/limit integers, timestamps as ISO strings
Created by frontend user interactions, validated by Pydantic schemas, transformed to database queries, results formatted back to JSON responses
System Behavior
How the system operates at runtime — where data accumulates, what loops, what waits, and what controls what.
Data Pools
Partitioned message queues that buffer binary-serialized browser events, ensuring ordered delivery and replay capability during service failures
Time-series database storing billions of user interaction events in compressed columnar format, partitioned by time for fast range queries
In-memory store tracking active sessions, user locations, and real-time assistance connections with TTL-based expiration
S3-compatible storage for screenshots, source maps, and cached static resources referenced during session replay
Feedback Loops
- Session Timeout Detection (polling, balancing) — Trigger: Ender service periodically scans active sessions in Redis. Action: For sessions without recent events, generates SessionEnd messages and removes from active pool. Exit: All sessions processed or service shutdown.
- Asset Caching Loop (cache-invalidation, reinforcing) — Trigger: JavaScript errors reference source map URLs not in object storage. Action: Asset service downloads, processes, and caches referenced files for future replay needs. Exit: All referenced assets cached or download failures exceed retry limit.
- Alert Processing Loop (polling, reinforcing) — Trigger: Alert processor runs ClickHouse queries on schedules. Action: Evaluates performance metrics against thresholds, sends notifications when conditions met. Exit: All alerts processed for current time window.
Delays
- Event Batching Window (batch-window, ~configured batch interval) — Messages accumulate in memory before bulk ClickHouse insert, trading latency for throughput
- Session End Timeout (scheduled-job, ~EVENTS_SESSION_END_TIMEOUT) — Sessions remain active in system until timeout expires, consuming memory and affecting metrics
- Real-time Cache Refresh (cache-ttl, ~CACHE_REFRESH_INTERVAL_SECONDS) — Assistance connections may see stale session data until next refresh cycle
Control Points
- Message Size Limit (threshold) — Controls: Maximum size of individual messages in Kafka queues, affects memory usage and network bandwidth. Default: cfg.MessageSizeLimit
- Batch Processing Size (threshold) — Controls: Number of events accumulated before ClickHouse insert, balances latency vs throughput. Default: configurable batch size
- Debug Mode (env-var) — Controls: Logging verbosity level throughout the system, affects performance and disk usage. Default: process.env.debug
- Project Key Length (env-var) — Controls: Validation of project identifiers in assistance routing, affects security. Default: PROJECT_KEY_LENGTH
Technology Stack
Message queues that buffer and partition browser event streams for ordered processing
Columnar database optimized for time-series queries of billions of user interaction events
Relational database storing user accounts, projects, alerts, and other structured metadata
In-memory cache for active session tracking, user presence, and assistance routing state
Frontend framework rendering session replay videos and developer debugging interfaces
WebSocket library enabling real-time communication for assistance features
Backend language providing high-performance message processing and API services
API layer using Chalice framework for REST endpoints and data validation
Browser API enabling peer-to-peer voice/video calls between users and support agents
Key Components
- MessageIterator (processor) — Deserializes binary message streams from Kafka, identifying message types and routing them to appropriate handlers based on configured filters
backend/pkg/messages/ - DataSaver (transformer) — Batches incoming messages and converts them to ClickHouse insert statements, handling different event types with type-specific table schemas
backend/internal/db/datasaver/ - SessionManager (orchestrator) — Tracks active sessions in Redis, manages session lifecycle from creation to expiration, and coordinates between different backend services
backend/pkg/sessions/ - Router (gateway) — HTTP request router that handles authentication, rate limiting, and routes API calls to appropriate service handlers with middleware
backend/pkg/server/api/ - AssistSocketHandler (adapter) — WebSocket server managing real-time connections between user browsers and support agents, handling room creation and message routing
assist/utils/wsServer.js - Tracker (encoder) — Browser-side JavaScript that observes DOM mutations, captures user events, records network activity, and serializes everything into binary messages
tracker/tracker/ - SessionPlayer (decoder) — React component that fetches session event sequences from the API and reconstructs them as a playable video timeline with controls
frontend/app/components/Session_/Player/ - AlertsProcessor (monitor) — Evaluates ClickHouse queries against performance and error thresholds to trigger notifications when sessions match alert conditions
api/chalicelib/core/alerts/alerts_processor.py
Explore the interactive analysis
See the full architecture map, data flow, and code patterns visualization.
Analyze on CodeSeaRelated Fullstack Repositories
Frequently Asked Questions
What is openreplay used for?
Records user sessions to replay their exact browser interactions for debugging openreplay/openreplay is a 8-component fullstack written in TypeScript. Data flows through 6 distinct pipeline stages. The codebase contains 3209 files.
How is openreplay architected?
openreplay is organized into 6 architecture layers: Browser Tracker, Message Processing, Data Storage, API Services, and 2 more. Data flows through 6 distinct pipeline stages. This layered structure keeps concerns separated and modules independent.
How does data flow through openreplay?
Data moves through 6 stages: Capture browser events → Stream to message queues → Process message streams → Batch and store events → Query session data → .... User interactions in browsers are captured by JavaScript trackers that serialize events into binary messages. These messages flow through Kafka queues to Go services that deserialize, batch, and insert them into ClickHouse time-series tables. The API layer queries this stored data to reconstruct session timelines, which the React frontend renders as playable videos. Real-time assistance bypasses storage, routing live events directly through WebSockets. This pipeline design reflects a complex multi-stage processing system.
What technologies does openreplay use?
The core stack includes Kafka (Message queues that buffer and partition browser event streams for ordered processing), ClickHouse (Columnar database optimized for time-series queries of billions of user interaction events), PostgreSQL (Relational database storing user accounts, projects, alerts, and other structured metadata), Redis (In-memory cache for active session tracking, user presence, and assistance routing state), React (Frontend framework rendering session replay videos and developer debugging interfaces), Socket.io (WebSocket library enabling real-time communication for assistance features), and 3 more. This broad technology surface reflects a mature project with many integration points.
What system dynamics does openreplay have?
openreplay exhibits 4 data pools (Kafka Event Streams, ClickHouse Event Store), 3 feedback loops, 4 control points, 3 delays. The feedback loops handle polling and cache-invalidation. These runtime behaviors shape how the system responds to load, failures, and configuration changes.
What design patterns does openreplay use?
5 design patterns detected: Event Sourcing, CQRS (Command Query Responsibility Segregation), Circuit Breaker, Pub/Sub Messaging, WebRTC Signaling.
Analyzed on April 20, 2026 by CodeSea. Written by Karolina Sarna.