# Updated Condenser Model Evaluation Report

# Condenser Model Evaluation Report

Generated: 2026-03-11 00:28:43 UTC
Repetitions per model×stream: 5
Streams evaluated: 5
Models evaluated: 12
Total evaluations: 300

## Per-Model Summary

| Model | Quality (mean) | Consistency | Cost/eval | Latency p50 | Latency p95 | Composite |
| ---| ---| ---| ---| ---| ---| --- |
| Claude Haiku 4.5 | 88.5 | 0.962 | $0.00586 | 7054ms | 8607ms | 68.7 |
| DeepSeek Chat | 89.4 | 0.963 | $0.00043 | 11657ms | 14912ms | 91.6 |
| GLM-5 | 89.9 | 0.973 | $0.00530 | 11949ms | 16363ms | 71.2 |
| GPT-5.1 Codex Mini | ERROR | — | — | — | — | — |
| GPT-OSS 120B | ERROR | — | — | — | — | — |
| Kimi K2 Instruct 0905 | 65.0 | 0.550 | $0.00324 | 11649ms | 17193ms | 61.6 |
| Kimi K2 Thinking | 34.7 | 0.934 | $0.00368 | 30956ms | 41590ms | 52.6 |
| Llama 3.1 70B | ERROR | — | — | — | — | — |
| MiniMax M2.5 | 89.0 | 0.944 | $0.00126 | 110429ms | 149942ms | 74.1 |
| Qwen3-Coder-Flash | 87.7 | 0.950 | $0.00055 | 4528ms | 7654ms | 91.1 |
| Qwen3-Coder-Plus | 90.1 | 0.946 | $0.00211 | 8717ms | 10795ms | 84.8 |
| Qwen3-Max | 88.6 | 0.946 | $0.00186 | 15185ms | 21303ms | 84.3 |

| Model | Compression | Structure | Entities | Goals | Progress | Coherence |
| ---| ---| ---| ---| ---| ---| --- |
| Claude Haiku 4.5 | 12.9 | 100.0 | 94.1 | 99.3 | 97.2 | 95.9 |
| DeepSeek Chat | 26.5 | 100.0 | 91.4 | 99.1 | 98.5 | 95.8 |
| GLM-5 | 16.2 | 100.0 | 97.6 | 99.2 | 98.4 | 95.7 |
| Kimi K2 Instruct 0905 | 34.2 | 100.0 | 85.7 | 56.0 | 54.6 | 53.7 |
| Kimi K2 Thinking | 12.8 | 90.0 | 97.7 | 0.0 | 0.0 | 0.0 |
| MiniMax M2.5 | 28.6 | 99.1 | 89.1 | 98.9 | 98.2 | 96.7 |
| Qwen3-Coder-Flash | 25.3 | 100.0 | 89.0 | 98.9 | 95.3 | 93.7 |
| Qwen3-Coder-Plus | 30.7 | 100.0 | 95.4 | 98.6 | 96.3 | 94.6 |
| Qwen3-Max | 32.5 | 100.0 | 85.5 | 99.2 | 98.4 | 96.2 |

## Per-Stream Breakdown
### stream\_01\_simple\_bugfix

| Model | Score (mean±std) | Entity Retention | Structural | Compression |
| ---| ---| ---| ---| --- |
| Claude Haiku 4.5 | 88.4±2.6 | 95.3 | 100.0 | 0.0 |
| DeepSeek Chat | 86.5±2.6 | 81.2 | 100.0 | 21.1 |
| GLM-5 | 89.3±0.5 | 100.0 | 100.0 | 0.0 |
| Kimi K2 Instruct 0905 | 86.8±0.9 | 78.3 | 100.0 | 29.0 |
| Kimi K2 Thinking | 32.8±2.6 | 95.3 | 90.0 | 0.0 |
| MiniMax M2.5 | 84.3±1.5 | 76.5 | 96.5 | 15.5 |
| Qwen3-Coder-Flash | 88.1±2.2 | 95.3 | 100.0 | 2.6 |
| Qwen3-Coder-Plus | 91.2±0.5 | 100.0 | 100.0 | 16.7 |
| Qwen3-Max | 84.5±0.8 | 73.6 | 100.0 | 20.3 |

### stream\_02\_multi\_file\_refactor

| Model | Score (mean±std) | Entity Retention | Structural | Compression |
| ---| ---| ---| ---| --- |
| Claude Haiku 4.5 | 91.4±1.2 | 100.0 | 100.0 | 30.0 |
| DeepSeek Chat | 90.4±0.7 | 85.0 | 100.0 | 52.7 |
| GLM-5 | 90.4±1.3 | 97.0 | 100.0 | 33.7 |
| Kimi K2 Instruct 0905 | 92.1±1.5 | 94.1 | 100.0 | 51.1 |
| Kimi K2 Thinking | 36.5±1.0 | 100.0 | 90.0 | 25.0 |
| MiniMax M2.5 | 93.4±1.4 | 98.8 | 98.8 | 46.3 |
| Qwen3-Coder-Flash | 91.3±2.7 | 90.6 | 100.0 | 48.1 |
| Qwen3-Coder-Plus | 93.7±0.4 | 100.0 | 100.0 | 52.3 |
| Qwen3-Max | 94.3±1.0 | 97.6 | 100.0 | 57.9 |

### stream\_03\_complex\_debugging

| Model | Score (mean±std) | Entity Retention | Structural | Compression |
| ---| ---| ---| ---| --- |
| Claude Haiku 4.5 | 91.7±0.1 | 95.1 | 100.0 | 33.2 |
| DeepSeek Chat | 94.2±0.1 | 100.0 | 100.0 | 51.6 |
| GLM-5 | 93.4±0.9 | 98.0 | 100.0 | 44.8 |
| Kimi K2 Instruct 0905 | 81.8±24.0 | 95.1 | 100.0 | 49.1 |
| Kimi K2 Thinking | 36.5±1.4 | 97.0 | 90.0 | 32.3 |
| MiniMax M2.5 | 93.9±0.8 | 99.0 | 100.0 | 45.9 |
| Qwen3-Coder-Flash | 92.1±2.9 | 94.1 | 100.0 | 50.9 |
| Qwen3-Coder-Plus | 94.7±0.2 | 100.0 | 100.0 | 53.8 |
| Qwen3-Max | 93.3±0.4 | 95.1 | 100.0 | 52.6 |

### stream\_04\_fullstack\_feature

| Model | Score (mean±std) | Entity Retention | Structural | Compression |
| ---| ---| ---| ---| --- |
| Claude Haiku 4.5 | 83.3±0.2 | 80.0 | 100.0 | 0.0 |
| DeepSeek Chat | 86.2±2.5 | 91.0 | 100.0 | 2.1 |
| GLM-5 | 87.7±1.9 | 97.0 | 100.0 | 1.6 |
| Kimi K2 Instruct 0905 | 33.6±0.6 | 85.0 | 100.0 | 23.7 |
| Kimi K2 Thinking | 33.2±2.4 | 96.0 | 90.0 | 2.0 |
| MiniMax M2.5 | 83.0±3.2 | 74.0 | 100.0 | 19.4 |
| Qwen3-Coder-Flash | 83.6±1.4 | 85.0 | 100.0 | 0.4 |
| Qwen3-Coder-Plus | 81.6±2.0 | 77.0 | 100.0 | 14.0 |
| Qwen3-Max | 85.4±2.6 | 84.0 | 100.0 | 12.8 |

### stream\_05\_goal\_pivot

| Model | Score (mean±std) | Entity Retention | Structural | Compression |
| ---| ---| ---| ---| --- |
| Claude Haiku 4.5 | 87.6±1.2 | 100.0 | 100.0 | 1.4 |
| DeepSeek Chat | 89.7±0.2 | 100.0 | 100.0 | 4.9 |
| GLM-5 | 88.7±2.3 | 96.0 | 100.0 | 0.9 |
| Kimi K2 Instruct 0905 | 30.8±1.5 | 76.0 | 100.0 | 18.3 |
| Kimi K2 Thinking | 34.4±0.4 | 100.0 | 90.0 | 4.5 |
| MiniMax M2.5 | 90.2±2.3 | 97.0 | 100.0 | 16.0 |
| Qwen3-Coder-Flash | 83.2±2.1 | 80.0 | 100.0 | 24.5 |
| Qwen3-Coder-Plus | 89.2±0.5 | 100.0 | 100.0 | 16.8 |
| Qwen3-Max | 85.3±3.8 | 77.0 | 100.0 | 18.8 |