axolotl-ai-cloud/axolotl

Go ahead and axolotl questions

Structural Verdict

How Data Flows Through the System

Configuration files drive dataset preprocessing, model initialization, training execution, and optional model serving through a linear pipeline with extensive customization points.

1. Config Loading — YAML config parsed and validated against AxolotlInputConfig schema
2. Dataset Preprocessing — Raw datasets transformed using configurable prompt strategies (config: datasets, dataset_prepared_path, prompt_strategies)
3. Model Initialization — Base model loaded with specified architecture and optimization settings (config: base_model, model_type, load_in_8bit +1)
4. Training Execution — Model fine-tuned using specified training parameters and techniques (config: learning_rate, num_epochs, batch_size +2)
5. Model Output — Trained model saved to configured output directory (config: output_dir, save_strategy)
6. Optional Serving — Model deployed for inference using vLLM or other serving backends (config: inference_engine)

System Behavior

How the system actually operates at runtime — where data accumulates, what loops, what waits, and what controls what.

Data Pools

Feedback Loops

• Training Loop (training-loop, balancing) — Trigger: Training start command. Action: Forward pass, loss calculation, backward pass, optimizer step. Exit: Max epochs or convergence reached.
• Checkpoint Saving (polling, balancing) — Trigger: Save strategy interval. Action: Save model state to disk. Exit: Training completion.
• Learning Rate Schedule (convergence, balancing) — Trigger: Each training step. Action: Adjust learning rate based on schedule. Exit: Training completion.

Delays & Async Processing

• Dataset Preprocessing (batch-window, ~varies by dataset size) — Training waits for dataset preparation
• Model Loading (async-processing, ~varies by model size) — Training startup delay
• Checkpoint Saving (async-processing, ~varies by model size) — Brief training pause during saves
• Cloud Provisioning (queue-drain, ~varies by cloud provider) — Job waits in queue before execution

Control Points

• Learning Rate (threshold) — Controls: Training convergence speed and stability. Default: configurable
• Batch Size (threshold) — Controls: Memory usage and gradient noise. Default: configurable
• LoRA Rank (threshold) — Controls: Adapter capacity and training efficiency. Default: configurable
• Save Strategy (env-var) — Controls: Checkpoint frequency and storage. Default: configurable
• GPU Memory Management (env-var) — Controls: CUDA memory allocation strategy. Default: configurable

Technology Stack

Key Components

• cli.main (cli-command) — Provides CLI commands for train, preprocess, quantize, shard, and serve operations src/axolotl/cli/main.py
• AxolotlInputConfig (type-def) — Pydantic schema defining all valid configuration parameters for training src/axolotl/utils/schemas/config.py
• handler (handler) — Runpod serverless entry point that processes training jobs from cloud requests .runpod/src/handler.py
• train (function) — Async training orchestrator that runs preprocessing and training commands .runpod/src/train.py
• AxolotlTrainer (class) — Base trainer class extending HuggingFace Trainer with Axolotl-specific features src/axolotl/core/trainers/base.py
• entropy_from_logits (function) — Triton-optimized entropy calculation for training metrics axolotl/monkeypatch/trainer/utils.py
• ScatterMoELoRA (class) — LoRA implementation for Mixture of Experts models with optimized kernels axolotl/integrations/kernels/libs/scattermoe_lora/parallel_linear_lora.py
• swanlab_profile (middleware) — Decorator for adding performance profiling to training methods axolotl/integrations/swanlab/profiling.py
• QuartoGenerator (utility) — Generates documentation from Pydantic model schemas automatically docs/scripts/generate_config_docs.py
• transform (function) — Dataset transformation function for EBFT training with code instruction data examples/ebft/ebft_opencode.py

Sub-Modules

Configuration

_quarto.yml (yaml)

• project.type (string, unknown) — default: website
• project.pre-render (array, unknown) — default: docs/scripts/generate_config_docs.py,docs/scripts/generate_examples_docs.py
• quartodoc.dir (string, unknown) — default: docs/api
• quartodoc.package (string, unknown) — default: axolotl
• quartodoc.title (string, unknown) — default: API Reference
• quartodoc.parser (string, unknown) — default: google
• quartodoc.sections (array, unknown) — default: [object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
• website.title (string, unknown) — default: Axolotl
• +22 more parameters

codecov.yml (yaml)

• codecov.require_ci_to_pass (string, unknown) — default: yes
• codecov.notify.wait_for_ci (boolean, unknown) — default: true
• coverage.precision (number, unknown) — default: 2
• coverage.round (string, unknown) — default: down
• coverage.range (string, unknown) — default: 70...100
• coverage.status.project.default.target (string, unknown) — default: auto
• coverage.status.project.default.threshold (string, unknown) — default: 1%
• coverage.status.project.default.base (string, unknown) — default: auto
• +29 more parameters

docker-compose.yaml (yaml)

• services.axolotl.build.context (string, unknown) — default: .
• services.axolotl.build.dockerfile (string, unknown) — default: ./docker/Dockerfile
• services.axolotl.volumes (array, unknown) — default: .:/workspace/axolotl,~/.cache/huggingface/:/root/.cache/huggingface/
• services.axolotl.environment (array, unknown) — default: GIT_AUTHOR_NAME=${GIT_AUTHOR_NAME},GIT_AUTHOR_EMAIL=${GIT_AUTHOR_EMAIL},GIT_COMMITTER_NAME=${GIT_COMMITTER_NAME},GIT_COMMITTER_EMAIL=${GIT_COMMITTER_EMAIL},WANDB_API_KEY=${WANDB_API_KEY}
• services.axolotl.deploy.resources.reservations.devices (array, unknown) — default: [object Object]
• services.axolotl.command (string, unknown) — default: tail -f /dev/null

src/axolotl/scripts/vllm_serve_lora.py (python-pydantic)

• n (int, unknown) — default: 1
• repetition_penalty (float, unknown) — default: 1.0
• temperature (float, unknown) — default: 1.0
• top_p (float, unknown) — default: 1.0
• top_k (int, unknown) — default: -1
• min_p (float, unknown) — default: 0.0
• max_tokens (int, unknown) — default: 16
• generation_kwargs (dict, unknown) — default: PydanticField(default_factory=dict)

Science Pipeline

1. Load Dataset — load_dataset then apply transform functions [variable (depends on dataset) → tokenized sequences of shape (batch_size, sequence_len)] src/axolotl/utils/data.py
2. Tokenize & Pad — tokenizer encode with padding to max_length [raw text strings → (batch_size, sequence_len) with attention masks] examples/ebft/ebft_pretrain.py
3. Model Forward Pass — transformer forward through attention layers and MLP/MoE [(batch_size, sequence_len) → (batch_size, sequence_len, vocab_size) logits] src/axolotl/core/trainers/base.py
4. Loss Calculation — cross entropy loss with label smoothing and entropy regularization [(batch_size, sequence_len, vocab_size) logits + labels → scalar loss value] axolotl/monkeypatch/trainer/utils.py
5. Gradient Update — backward pass through LoRA adapters or full model [loss scalar → updated model parameters] src/axolotl/core/trainers/base.py

Assumptions & Constraints

• [warning] Assumes logits tensor can be reshaped to (-1, num_classes) but no explicit shape validation (shape)
• [warning] Assumes sequence_len parameter fits within tokenizer limits without validation (value-range)
• [critical] Hardcoded DEVICE = 'cuda' assumes GPU availability without fallback (device)
• [info] Assumes specific tensor dtypes for MoE routing without explicit casting (dtype)

Related Ml Training Repositories

Frequently Asked Questions

Analyzed on March 31, 2026 by CodeSea. Written by Karolina Sarna.