kestra-io/kestra
Event Driven Orchestration & Scheduling Platform for Mission Critical Applications
Event-driven orchestration platform for mission-critical workflow automation
Workflows are defined in YAML, stored in repositories, queued for execution, processed by workers, with results indexed and exposed via API/UI
Under the hood, the system uses 2 feedback loops, 3 data pools, 3 control points to manage its runtime behavior.
A 8-component data pipeline with 3 connections. 2285 files analyzed. Data flows through 5 distinct pipeline stages.
How Data Flows Through the System
Workflows are defined in YAML, stored in repositories, queued for execution, processed by workers, with results indexed and exposed via API/UI
- Flow Definition — YAML workflow definitions are validated and stored in flow repositories
- Execution Queue — Flow executions are queued and dispatched to available workers
- Worker Processing — Workers execute flow tasks and report status back through controllers
- Result Indexing — Execution results and logs are indexed for searchability and monitoring
- API/UI Access — Results are exposed through REST API and consumed by web UI
System Behavior
How the system operates at runtime — where data accumulates, what loops, what waits, and what controls what.
Data Pools
Persisted workflow definitions and metadata
Pending and active workflow executions
Historical execution results and logs
Feedback Loops
- Worker Health Monitoring (polling, balancing) — Trigger: Worker controller checks worker status periodically. Action: Reschedule failed executions or spawn new workers. Exit: Healthy worker count maintained.
- Flow Validation Loop (retry, balancing) — Trigger: Invalid flow definition submitted. Action: Return validation errors to user for correction. Exit: Valid flow definition accepted.
Delays
- Scheduled Execution (scheduled-job, ~User-defined cron schedules) — Workflows wait for scheduled trigger time
- Queue Processing (queue-drain, ~Variable based on worker availability) — Executions wait in queue until worker capacity available
Control Points
- Worker Thread Count (env-var) — Controls: Number of concurrent worker threads for execution. Default: Math.max(3, Runtime.getRuntime().availableProcessors())
- Scheduler Enable/Disable (runtime-toggle) — Controls: Whether scheduled workflows are processed. Default: true
- Tenant ID Selection (env-var) — Controls: Multi-tenant isolation for enterprise deployments
Technology Stack
Dependency injection and HTTP framework
Command-line interface framework
Frontend web UI framework
Pluggable database backends
Build tool and dependency management
Containerization for deployment
Key Components
- Kestra (cli-command) — Main CLI orchestrator that routes to subcommands for flows, servers, plugins, and system operations
cli/src/main/java/io/kestra/cli/Kestra.java - StandAloneRunner (service) — Runs the complete Kestra stack in standalone mode with workers, scheduler, indexer, and controller
cli/src/main/java/io/kestra/cli/StandAloneRunner.java - AbstractApiCommand (class) — Base class for CLI commands that interact with Kestra server API with authentication and HTTP client setup
cli/src/main/java/io/kestra/cli/AbstractApiCommand.java - FlowCommand (cli-command) — Flow management subcommands including test, export, delete, and synchronization operations
cli/src/main/java/io/kestra/cli/commands/flows/FlowCommand.java - FlowTestCommand (cli-command) — Executes flow tests in standalone mode with temporary execution environment and await completion
cli/src/main/java/io/kestra/cli/commands/flows/FlowTestCommand.java - useKvFilter (hook) — Vue composable exports for various filter configurations used across the UI
ui/src/components/filter/configurations/index.ts - flowExamples (config) — Collection of predefined workflow templates for onboarding including ETL, microservices, and automation examples
ui/src/components/onboarding/flows/index.ts - defineKestraHeyConfig (function) — Configuration builder for Kestra's API SDK plugin integration
ui/heyapi-sdk-plugin/index.ts
Explore the interactive analysis
See the full architecture map, data flow, and code patterns visualization.
Analyze on CodeSeaRelated Data Pipeline Repositories
Frequently Asked Questions
What is kestra used for?
Event-driven orchestration platform for mission-critical workflow automation kestra-io/kestra is a 8-component data pipeline written in Java. Data flows through 5 distinct pipeline stages. The codebase contains 2285 files.
How is kestra architected?
kestra is organized into 4 architecture layers: Core Engine, Execution Layer, Storage & Queue, Interface Layer. Data flows through 5 distinct pipeline stages. This layered structure keeps concerns separated and modules independent.
How does data flow through kestra?
Data moves through 5 stages: Flow Definition → Execution Queue → Worker Processing → Result Indexing → API/UI Access. Workflows are defined in YAML, stored in repositories, queued for execution, processed by workers, with results indexed and exposed via API/UI This pipeline design reflects a complex multi-stage processing system.
What technologies does kestra use?
The core stack includes Micronaut (Dependency injection and HTTP framework), Picocli (Command-line interface framework), Vue 3 (Frontend web UI framework), PostgreSQL/MySQL/H2 (Pluggable database backends), Gradle (Build tool and dependency management), Docker (Containerization for deployment). A focused set of dependencies that keeps the build manageable.
What system dynamics does kestra have?
kestra exhibits 3 data pools (Flow Repository, Execution Queue), 2 feedback loops, 3 control points, 2 delays. The feedback loops handle polling and retry. These runtime behaviors shape how the system responds to load, failures, and configuration changes.
What design patterns does kestra use?
4 design patterns detected: Command Pattern, Plugin Architecture, Tenant Multi-tenancy, Composables Pattern.
Analyzed on March 31, 2026 by CodeSea. Written by Karolina Sarna.