How DSPy Works
Prompt engineering is manual tuning — you adjust words until the output looks right. DSPy treats it as an optimization problem: define what you want (signatures), provide examples, and let the framework find the prompts and few-shot examples that actually work.
What dspy Does
Framework for programming language models using structured Python code instead of prompts
DSPy allows developers to build AI systems using declarative Python code with typed signatures and modules, then automatically optimize their prompts and weights. It provides abstractions for composing LM calls into pipelines and algorithms for improving their performance.
Architecture Overview
dspy is organized into 5 layers, with 10 components and 21 connections between them.
How Data Flows Through dspy
User creates signatures and modules, which get processed by adapters into LM calls, then parsed back into structured outputs. Optimizers can improve this pipeline automatically.
1Signature Definition
User defines typed input/output interface with field descriptions
2Module Creation
Predict, ChainOfThought, or custom modules use signatures to define behavior
3Adapter Processing
Adapter converts signature+inputs into LM-specific format (chat messages, JSON schema)
4LM Execution
Language model processes formatted input and returns response
5Response Parsing
Adapter parses LM response back into structured signature outputs
6Optimization
Teleprompt optimizers can improve prompts and demonstrations automatically
System Dynamics
Beyond the pipeline, dspy has runtime behaviors that shape how it responds to load, failures, and configuration changes.
Data Pools
DSPY_CACHE
Disk-based cache for LM responses to avoid redundant API calls
Type: cache
settings
Global configuration state for LM, adapter, and framework settings
Type: state-store
Feedback Loops
Teleprompt Optimization
Trigger: User calls optimizer with training data → Generate candidates, evaluate performance, update prompts/weights (exits when: Convergence criteria met or max iterations)
Type: training-loop
Adapter Fallback
Trigger: ChatAdapter parsing fails → Fall back to JSONAdapter with same inputs (exits when: Successful parse or both adapters fail)
Type: retry
LM Retry Logic
Trigger: LM call fails or times out → Retry with exponential backoff via tenacity (exits when: Success or max retries exceeded)
Type: retry
Control Points
settings.configure
Adapter selection
Cache enabled
Log level
Delays
LM API Calls
Duration: varies by model
Cache Lookups
Duration: until cache expires
Optimization Iterations
Duration: minutes to hours
Technology Choices
dspy is built with 8 key technologies. Each serves a specific role in the system.
Key Components
- Adapter (class): Base interface layer that transforms DSPy signatures into LM calls and parses responses back
- ChatAdapter (class): Default adapter that formats DSPy signatures using field header delimiters for most LMs
- JSONAdapter (class): Structured output adapter using JSON Schema for precise field parsing
- TwoStepAdapter (class): Two-stage adapter using natural prompts for reasoning models then structured extraction
- Signature (class): Type-safe interface defining input/output fields and descriptions for LM programs
- Type (class): Base class for custom types like Image, Audio, Code that format into LM message content
- LM (class): Language model client interface with caching, usage tracking, and adapter integration
- Module (class): Base class for composable DSPy components that can be trained and optimized
- Predict (class): Basic module for single LM calls using a signature
- settings (module): Global configuration manager for LM, adapter, and framework settings
Who Should Read This
ML researchers and engineers who want to move beyond manual prompt engineering, or teams building complex LLM pipelines.
This analysis was generated by CodeSea from the stanfordnlp/dspy source code. For the full interactive visualization — including pipeline graph, architecture diagram, and system behavior map — see the complete analysis.
Explore Further
Full Analysis
Interactive architecture map for dspy
dspy vs langchain
Side-by-side architecture comparison
dspy vs llama_index
Side-by-side architecture comparison
dspy vs guidance
Side-by-side architecture comparison
How LangChain Works
ML Inference & Agents
How LlamaIndex Works
ML Inference & Agents
How vLLM Works
ML Inference & Agents
Frequently Asked Questions
What is dspy?
Framework for programming language models using structured Python code instead of prompts
How does dspy's pipeline work?
dspy processes data through 6 stages: Signature Definition, Module Creation, Adapter Processing, LM Execution, Response Parsing, and more. User creates signatures and modules, which get processed by adapters into LM calls, then parsed back into structured outputs. Optimizers can improve this pipeline automatically.
What tech stack does dspy use?
dspy is built with litellm (Unified LM client interface), pydantic (Type validation and structured data), openai (OpenAI API client), json_repair (Robust JSON parsing for LM outputs), tenacity (Retry logic for LM calls), and 3 more technologies.
How does dspy handle errors and scaling?
dspy uses 3 feedback loops, 4 control points, 2 data pools to manage its runtime behavior. These mechanisms handle error recovery, load distribution, and configuration changes.
How does dspy compare to langchain?
CodeSea has detailed side-by-side architecture comparisons of dspy with langchain, llama_index, guidance. These cover tech stack differences, pipeline design, and system behavior.