Laminar run comparison
Compare multiple runs under the same harness contract: success, token spend, carry-forward, and straightforward lower-is-better search and verification burdens.
Avg carry-forward ratio
98.4%
Avg files before first edit
18.0
Avg tests before first pass
113.5
Success versus token spend
Each point is one run, sized by total steps and colored by cache ratio. Hover for output tokens, tests before first pass, and files before first edit.
Action mix per run
Normalized action shares across all classified events, including runtime overhead.
Visible action mix per run
Same mix after removing overhead, which makes exploration, editing, testing, and finalization easier to compare.
Overhead breakdown per run
Subdivides overhead into persistence lookups, persistence writes, orchestration, and unknown internal work.
Search burden versus test burden
X-axis is files before the first edit; Y-axis is tests before the first pass or run end. Marker size tracks total steps and color tracks carry-forward ratio.
Direct burden metrics per run
Grouped bars show files before first edit, edits before first pass, tests before first pass, tests after first pass, and retry bursts. Lower is usually better.
Run focus snapshots
Top modules and files help distinguish runs that share the same benchmark pattern but work in different areas of the repo.
537db26f
first edit step: 214 · files before edit: 31 · edits before first pass: 0 · tests before first pass: 226
Top modules
- workspace23
- simpy9
- *.py1
- exceptions.py1
Top files
- /workspace/simpy68
- /workspace/simpy/src/simpy/events.py60
- /workspace/simpy/src/simpy/core.py16
- /workspace/simpy/src/simpy/rt.py14
9c186814
first edit step: 150 · files before edit: 9 · edits before first pass: 2 · tests before first pass: 48
Top modules
- workspace9
- tests/test_deprecated.py1
- classic.py1
- tests/test_sphinx.py1
Top files
- /workspace/deprecated28
- /workspace/deprecated/deprecated/sphinx.py12
- /workspace/deprecated/deprecated/classic.py6
- tests/test_deprecated.py6
39f38cb4
first edit step: 198 · files before edit: 16 · edits before first pass: 0 · tests before first pass: 48
Top modules
- workspace14
- src/jinja28
- tmp8
- extracted5
Top files
- /workspace/jinja71
- /workspace/jinja/src/jinja2/utils.py52
- /tmp/jinja2_download26
- /workspace/jinja/src/jinja2/nodes.py12
67ade002
first edit step: 152 · files before edit: 16 · edits before first pass: 17 · tests before first pass: 132
Top modules
- workspace19
- jwt11
- *.py1
- tests/test_algorithms.py1
Top files
- /workspace/pyjwt113
- /workspace/pyjwt/jwt/algorithms.py50
- /workspace/pyjwt/jwt/api_jwt.py29
- tests/test_algorithms.py18
Run summary table
Compact per-run metrics using direct lower-is-better search and verification counts.
| Label | Run ID | Success | Input tokens | Output tokens | Files before first edit | Edits before first pass | Tests before first pass | Tests after first pass | Max retry burst | Carry-forward ratio |
|---|
| 537db26f | 537db26f-50c3-7350-1a28-dfdf1a349f66 | no | 12,492,928 | 50,715 | 31 | 0 | 226 | 1016 | 0 | 98.9% |
| 9c186814 | 9c186814-4042-2cc0-eda3-a408457ebb56 | no | 956,814 | 9,994 | 9 | 2 | 48 | 351 | 0 | 96.8% |
| 39f38cb4 | 39f38cb4-a374-3834-7153-8d1443d04822 | yes | 8,517,025 | 23,694 | 16 | 0 | 48 | 2725 | 1 | 98.8% |
| 67ade002 | 67ade002-afff-460d-8851-6e17c5b0103c | no | 15,963,510 | 59,113 | 16 | 17 | 132 | 2546 | 0 | 99.1% |
Metric glossary
Definitions for the comparison charts and table metrics.
Action mix
Normalized share of classified events within a run. The top chart includes internal runtime work; the second chart removes overhead so visible coding behavior is easier to compare.
Overhead
Internal harness/runtime work rather than direct repo work. In these traces it is mostly LocalFileStore persistence activity, split into persistence lookups and writes when possible.
Files before first edit
Distinct files explicitly read before the first edit step. Lower is usually better, because it means the run searched less before committing to an edit.
Edits before first pass
Number of edit steps before the first passing test. Lower is usually better; a low search count with many edits suggests the run committed quickly but not cleanly.
Tests before first pass
Tests consumed before the first passing signal. For failed runs, this falls back to total tests run before the trace ended. Lower is usually better.
Tests after first pass
Additional tests run after the first passing signal. Lower is better, because it suggests less verification churn after the patch was already plausible.
Max retry burst
Longest consecutive burst of explicit retry or wait steps. Lower is usually better.
Carry-forward ratio
Share of attributed prompt tokens coming from replayed context plus cached prefix tokens. Higher values mean more prior state is being carried forward into each LLM call.