Table of Contents

• State-of-the-Art Results
  • SWE-bench Lite Performance
  • The Debugging Gap
  • Repository-Specific Results
• MRR Benchmark Results
• Key Innovations
• Architecture
• Benchmarks & Evaluation
• Research Paper
• Getting Started
• Repository Structure
• Research Highlights
• Detailed Performance
• Documentation
• Contributing
• Citation
• License

Model Access Notice

🏅 State-of-the-Art Results

📈 SWE-bench Lite Performance

The Debugging Gap

Repository-Specific Results

🔬 MRR Benchmark Results

🐛 Performance by Bug Category

📏 Repository Scale Performance

💡 Key Innovations

1. Debugging-First Architecture

• Trained on 42.5M real debugging examples (not code completion)
• Specialized for root cause analysis and multi-file patches
• 89% root cause accuracy vs 15.8% best baseline
• 7-layer architecture optimized for debugging workflows

2. Persistent Debug Memory (PDM)

• Repository-specific learning from 15M+ debugging sessions
• Improves from 35% → 65% success rate over time
• Cross-session pattern recognition and learning
• 87% cache hit rate for similar bugs
• Temporal pattern learning across project lifecycles

3. Adaptive Graph-Guided Retrieval (AGR)

• O(k log d) complexity with dynamic k-hop expansion
• 92% precision, 85% recall on multi-file context
• Handles unlimited repository scale intelligently
• Multi-hop traversal with confidence-based termination
• 3.8x faster than traditional retrieval methods

4. Output-Optimized Design

• Optimized for ~3K output tokens (fixes, tests, docs)
• 47.2% output entropy density vs 12.8% for completion models
• Designed for complex patch generation
• Template-aware generation for consistency
• Confidence-guided output strategy

5. Autonomous Debugging Loop

• Average 7.8 iterations to successful fix
• Propose → Test → Analyze → Refine cycles
• 67.3% fully autonomous success rate
• Execution sandbox with real-time feedback
• Iterative refinement until validation succeeds

🏗️ Architecture

Seven-Layer System Design

┌─────────────────────────────────────────────┐
│   7. Explainability Layer                   │  Human-readable root cause analysis
├─────────────────────────────────────────────┤
│   6. Execution Sandbox                      │  Isolated test validation
├─────────────────────────────────────────────┤
│   5. Persistent Debug Memory (PDM)          │  Repository-specific learning
├─────────────────────────────────────────────┤
│   4. Orchestration Controller               │  Autonomous debugging loop
├─────────────────────────────────────────────┤
│   3. Debug-Tuned LLM Core                   │  42.5M debugging examples
├─────────────────────────────────────────────┤
│   2. Adaptive Retrieval Engine (AGR)        │  Dynamic k-hop graph traversal
├─────────────────────────────────────────────┤
│   1. Multi-Source Input Layer               │  Code, logs, traces, tests, docs
└─────────────────────────────────────────────┘

Layer Descriptions

1. Multi-Source Input Layer: Processes code, logs, traces, tests, docs simultaneously
2. Adaptive Retrieval Engine (AGR): Dynamic k-hop graph traversal (92% precision)
3. Debug-Tuned LLM Core: 42.5M debugging examples, not code completion
4. Orchestration Controller: Autonomous debugging loop management
5. Persistent Debug Memory (PDM): Repository-specific learning (35% → 65% improvement)
6. Execution Sandbox: Isolated test validation environment
7. Explainability Layer: Human-readable root cause analysis

View Detailed Architecture Documentation →

🧪 Benchmarks & Evaluation

📋 Available Benchmarks

🏆 SWE-bench Lite Evaluation Results

View Complete SWE-bench Lite Submission →

The evaluation directory contains:

• README.md: Detailed submission results and methodology
• metadata.yaml: Submission metadata and configuration
• all_preds.jsonl: All 300 instance predictions
• Kodezi Chronos-1.hybrid_eval.json: Complete evaluation metrics
• logs/: Execution logs for all instances
• results/: Per-instance results and analysis
• trajs/: Debugging trajectories and fix attempts

🎯 Multi-Random Retrieval (MRR) Benchmark

MRR simulates real-world debugging complexity:

• Spatial Distribution: Bug context scattered across 10-50 files
• Temporal Dispersion: Relevant information from 3-12 months of history
• Obfuscation Levels: Low/medium/high code complexity
• 5,000 Scenarios: Comprehensive evaluation across languages (sample dataset of 500 available now, full benchmark Q1 2026)

View Complete Benchmark Documentation →

📚 Research Paper

Published Research

Title: Kodezi Chronos: A Debugging-First Language Model for Repository-Scale Code Understanding

Authors: Ishraq Khan, Assad Chowdary, Sharoz Haseeb, Urvish Patel, Yousuf Zaii

Institution: Kodezi Inc.

Publication: arXiv:2507.12482 (2025)

Paper Resources

Key Contributions

1. Debugging-Specific Architecture: First LM trained specifically on debugging workflows (42.5M examples)
2. Adaptive Graph-Guided Retrieval (AGR): Novel multi-hop retrieval with O(k log d) complexity
3. Persistent Debug Memory (PDM): Cross-session learning system for repository-specific patterns
4. Comprehensive Evaluation: 12,500 real-world bugs across multiple benchmarks
5. State-of-the-Art Results: 80.33% on SWE-bench Lite (20pp lead over second place)

🚀 Getting Started

Prerequisites

# Python 3.8+ required
python --version

# Git for cloning
git --version

Quick Start: Running Benchmarks

# Clone the repository
git clone https://github.com/kodezi/chronos-research.git
cd chronos-research

# Install dependencies
pip install -r requirements.txt

# Run MRR benchmark on your model
python benchmarks/run_mrr_benchmark_2025.py \
  --model your_model \
  --scenarios 100  # Start with subset for testing

# Run full sample evaluation (500 scenarios available)
python benchmarks/run_mrr_benchmark_2025.py \
  --model your_model \
  --scenarios 500

# Analyze results
python benchmarks/analyze_results.py \
  --results_dir results/your_model

Model Access

The Chronos model is NOT included in this repository

This repository contains:

• Research paper and documentation
• Benchmark suite and evaluation framework
• Performance results and analysis
• Chronos model (proprietary - NOT included)

To access Chronos model:

Join Waitlist → | Contact Sales →

📁 Repository Structure

chronos-research/
│
├── benchmarks/                    # Benchmark Suite
│   ├── multi-random-retrieval/      # 5,000 scenario MRR benchmark (500 sample available)
│   ├── comprehensive_benchmarks/    # Extended test scenarios
│   ├── debug_categories/            # Bug type categorization (6 types)
│   ├── evaluation_metrics/          # Custom metrics implementation
│   ├── run_mrr_benchmark_2025.py    # Main benchmark runner
│   └── analyze_results.py           # Results analysis tools
│
├── evaluation/                    # Evaluation Results
│   └── lite/                        # SWE-bench Lite results (80.33%)
│       └── 20251111_kodezi_chronos_1/  # Official submission
│           ├── all_preds.jsonl      # All 300 predictions
│           ├── logs/                # 300+ execution logs
│           ├── results/             # Per-instance results
│           └── trajs/               # Debugging trajectories
│
├── paper/                         # Research Paper
│   ├── chronos-research.md          # Full paper (arXiv:2507.12482)
│   ├── chronos-research-2025.md     # 2025 updates
│   ├── abstract.md                  # Executive summary
│   ├── methodology.md               # Research methodology
│   └── figures/                     # Visualizations
│
├── architecture/                  # Architecture Documentation
│   ├── README.md                    # Architecture overview
│   ├── AGR_ALGORITHM.md             # Adaptive Graph-Guided Retrieval
│   ├── memory_engine.md             # Persistent Debug Memory (PDM)
│   └── debugging_loop.md            # Autonomous loop design
│
├── results/                       # Performance Data
│   ├── figures/                     # 15+ SVG visualizations
│   ├── ablation_studies/            # Component impact analysis
│   ├── case_studies/                # Real-world debugging examples
│   └── raw_data/                    # Benchmark outputs (CSV/JSON)
│
├── reference_implementations/     # Algorithm Reference Code
│   ├── algorithms/                  # AGR, PDM reference implementations
│   └── NOTICE.md                    # Proprietary notice
│
├── docs/                          # Documentation
│   ├── getting_started.md           # Quick start guide
│   ├── API_DOCUMENTATION.md         # API reference (Q1 2026)
│   ├── faq.md                       # Frequently asked questions
│   └── limitations.md               # Known constraints
│
├── LEADERBOARD.md                 # Performance rankings
├── CITATION.cff                   # Citation information (BibTeX)
├── CONTRIBUTING.md                # Contribution guidelines
├── LICENSE                        # MIT License + proprietary notice
└── requirements.txt               # Python dependencies

Key Directories:

• benchmarks/: 5,000 scenario MRR benchmark (500 sample available), multi-language support, automated evaluation
• evaluation/: SWE-bench Lite results (80.33%, 241/300 instances)
• paper/: Complete research paper and documentation (arXiv:2507.12482)
• architecture/: 7-layer system design, AGR/PDM documentation
• results/: 12,500+ bug resolutions, visualizations, statistical analysis
• reference_implementations/: Algorithm reference code (NOT the actual model)

🔬 Research Highlights

Training Dataset Composition

Total Training Data: 42.5M debugging-specific examples (not code completion)

AGR Performance by Depth

PDM Learning Curve

Key Insight: PDM enables continuous improvement through cross-session learning

📊 Detailed Performance

Language-Specific Performance

Debugging Cycle Efficiency

Context Window Efficiency

Ablation Studies

📖 Documentation

🤝 Contributing

We welcome contributions to the evaluation framework and benchmarks!

How to Contribute

# 1. Fork and clone the repository
git clone https://github.com/[your-username]/chronos-research.git
cd chronos-research

# 2. Create a feature branch
git checkout -b feature/your-contribution

# 3. Make your changes
# - Add new benchmarks
# - Improve documentation
# - Fix bugs in evaluation scripts

# 4. Run tests
python -m pytest tests/

# 5. Commit your changes
git add .
git commit -m "feat: description of your changes"

# 6. Push and create PR
git push origin feature/your-contribution

Contribution Guidelines

• Add tests for new features
• Follow existing code style
• Update documentation
• Add benchmarks for new capabilities
• Include performance analysis

See CONTRIBUTING.md for detailed guidelines.

📝 Citation

If you use this research in your work, please cite:

@article{khan2025chronos,
  title={Kodezi Chronos: A Debugging-First Language Model for
         Repository-Scale Code Understanding},
  author={Khan, Ishraq and Chowdary, Assad and
          Haseeb, Sharoz and Patel, Urvish and Zaii, Yousuf},
  journal={arXiv preprint arXiv:2507.12482},
  year={2025},
  url={https://arxiv.org/abs/2507.12482},
  note={State-of-the-art: 80.33\% on SWE-bench Lite}
}

🏢 About Kodezi

📧 Contact & Community

📄 License

© Kodezi Inc. All rights reserved. Use is subject to Kodezi's Terms of Service.

MIT License

Copyright (c) 2025 Kodezi Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

⚠️ Important Notice

This license applies ONLY to the research paper, benchmarks, evaluation frameworks, and documentation contained in this repository.

The Kodezi Chronos model itself is proprietary technology owned by Kodezi Inc. and is NOT included in this repository or covered by this license.

📦 What's Included Under MIT License

• Research Paper: arXiv publication and markdown versions
• Benchmark Suite: MRR and evaluation frameworks
• Evaluation Results: SWE-bench Lite results and analysis
• Documentation: Architecture docs, guides, and references
• Reference Implementations: Algorithm reference code (NOT the actual model)

🔒 Proprietary Components

• Chronos Model: NOT included in this repository
• Kodezi OS Integration: Proprietary platform components
• Production APIs: Enterprise deployment infrastructure

🚀 Chronos Model Access

The Chronos model is available exclusively through Kodezi OS:

• Q4 2025: Enterprise beta access
• Q1 2026: General availability
• Learn more: chronos.so
• Early access: kodezi.com/os