microsoft/semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps

How Data Flows Through the System

User input flows through the Kernel orchestrator to agents, which use LLM connectors to generate responses while calling plugins for external functionality. Memory stores provide context retrieval through vector similarity search, while process frameworks coordinate multi-step workflows with state persistence and event handling across distributed services.

1. Initialize Kernel with services and plugins — Kernel.builder() registers LLM connectors (OpenAIChatCompletion), memory stores (VectorStoreRecordCollection), and plugins (KernelPlugin) through dependency injection [KernelBuilder → Kernel]
2. Create agent with kernel and instructions — ChatCompletionAgent constructor takes kernel, name, instructions, and optional execution_settings to create an agent instance with conversation state [Kernel → ChatCompletionAgent]
3. Process user message through agent — Agent.invoke_async() converts user input to ChatMessageContent, maintains conversation history in ChatHistory, and sends to LLM connector [ChatMessageContent → ChatMessageContent]
4. Execute tool calls from LLM response — If LLM response contains function_calls, agent iterates through them, looks up KernelFunction in registered plugins, and invokes with extracted arguments [FunctionCall → FunctionResult]
5. Retrieve context from vector memory — Memory search uses EmbeddingGenerator to convert query text to vectors, then VectorStoreRecordCollection.search() finds similar MemoryRecord entries [TextContent → MemoryRecord]
6. Orchestrate multi-step process workflow — ProcessStepBuilder creates workflow steps that emit events, maintain state in ProcessData, and coordinate between local functions and external services via gRPC/SignalR [ProcessData → ProcessEvent]

Data Models

The data structures that flow between stages — the contracts that hold the system together.

Hidden Assumptions

Things this code relies on but never validates. These are the things that cause silent failures when the system changes.

System Behavior

How the system operates at runtime — where data accumulates, what loops, what waits, and what controls what.

Data Pools

Feedback Loops

• Agent conversation loop (recursive, reinforcing) — Trigger: User message or function result. Action: Agent processes input, calls LLM, executes tools if needed, generates response. Exit: No more function calls in LLM response.
• Process workflow execution (training-loop, reinforcing) — Trigger: Process event emission. Action: Steps execute in sequence, emit events to trigger next steps, update state. Exit: Workflow reaches terminal step or error condition.
• Memory similarity search refinement (recursive, balancing) — Trigger: Low relevance scores in search results. Action: Adjust embedding generation parameters, re-index content, refine query. Exit: Relevance threshold met or max iterations reached.

Delays

• LLM API response time (async-processing, ~200ms-5s) — Agent waits for completion before proceeding with tool calls or response generation
• Vector embedding generation (async-processing, ~50-500ms) — Memory operations wait for text-to-vector conversion before similarity search
• Process step coordination (eventual-consistency, ~Variable) — Distributed steps may execute out of order until event correlation completes

Control Points

• Model selection (architecture-switch) — Controls: Which LLM provider and model variant agents use for completions. Default: Configured via environment variables
• Temperature and top_p (hyperparameter) — Controls: Randomness and creativity of LLM responses. Default: Default 0.7 temperature
• Vector similarity threshold (threshold) — Controls: Minimum relevance score for memory retrieval results. Default: 0.8 default
• Tool calling enablement (feature-flag) — Controls: Whether agents can execute functions during conversation. Default: Enabled by default

Technology Stack

Key Components

• Kernel (orchestrator) — Central coordinator that manages plugins, services, and execution context — handles function registration, service dependency injection, and coordinates between agents, memory, and LLM connectors python/semantic_kernel/kernel.py
• ChatCompletionAgent (agent) — Wraps LLM chat completion APIs with conversation state management, tool calling capabilities, and structured output parsing python/semantic_kernel/agents/chat_completion_agent.py
• VectorStoreRecordCollection (adapter) — Abstract interface for vector database operations — provides upsert, get, delete, and vector search methods across different vector store implementations python/semantic_kernel/data/vector_store_record_collection.py
• ProcessStepBuilder (factory) — Builds process step definitions with event routing, state management, and external integration configuration for workflow orchestration dotnet/src/Experimental/Process.Abstractions/ProcessStepBuilder.cs
• OpenAIChatCompletion (adapter) — Connector for OpenAI and Azure OpenAI chat APIs — handles authentication, request formatting, response parsing, and streaming for chat completions python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion.py
• PromptTemplate (processor) — Renders prompt templates with variable substitution, conditional logic, and formatting — supports Jinja2 and handlebars syntax for dynamic prompt generation python/semantic_kernel/prompt_template/prompt_template.py
• EmbeddingGenerator (transformer) — Abstract base for converting text into vector embeddings using various embedding models — handles batching, normalization, and caching python/semantic_kernel/connectors/ai/embeddings/embedding_generator_base.py
• KernelPlugin (registry) — Container for related functions that can be discovered and invoked by agents — manages function metadata, validation, and access control python/semantic_kernel/functions/kernel_plugin.py

Related Ml Inference Repositories

Frequently Asked Questions

Analyzed on April 20, 2026 by CodeSea. Written by Karolina Sarna.