References and Further Reading
Papers, code, and further reading on harness engineering.
152 sources
Harness design and evaluation 24
How AI is transforming work at Anthropic
https://www.anthropic.com/research/how-ai-is-transforming-work-at-anthropicSame Model, Different Harness: 8090 News
https://www.8090.ai/blog/same-model-different-harnessHarness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows
https://arxiv.org/abs/2605.27922Post-Training RLM Agents for End-to-End M&A Diligence
https://www.harvey.ai/blog/post-training-rlm-agents-for-m-and-a-diligenceBuilding Laminar: Ramp's Managed Agents Platform
https://builders.ramp.com/post/building-laminarHow to Build a Coding Agent (Syntax 976)
https://www.youtube.com/watch?v=IPJ7Mp_ajuQ&t=2936sHarness Lens
https://huggingface.co/spaces/ybelkada/harness-lensDevtools must be open source - exe.dev blog
https://blog.exe.dev/devtools-must-be-open-sourceHarness engineering: leveraging Codex in an agent-first world
https://openai.com/index/harness-engineeringSame Weights, Different Model (Yun Jin)
https://www.linkedin.com/pulse/same-weights-model-yun-jin-ch8bcWhy the Harness Matters More Than the Model (YC Paper Club)
https://www.youtube.com/watch?v=n9xKblqyQ28&t=1277sThe Anatomy of an Agent Harness
https://blog.langchain.com/the-anatomy-of-an-agent-harnessTowards a Science of AI Agent Reliability
https://arxiv.org/abs/2602.16666I keep hearing the same advice about agents
https://gradientflow.substack.com/p/i-keep-hearing-the-same-advice-aboutThe third era of AI software development
https://prod.cursor.com/blog/third-eraDive into Claude Code: The Design Space of Today's and Future AI Agent Systems
https://arxiv.org/abs/2604.14228Provider harnesses are a good starting point
https://openai.com/index/introducing-upgrades-to-codexAn update on Claude Code quality reports
https://www.anthropic.com/engineering/april-23-postmortemSyntax 976: Pi transcript
https://syntax.fm/show/976/pi-the-ai-harness-that-powers-openclaw-w-armin-ronacher-and-mario-zechner/transcriptOpenAI Codex CLI vs Claude Code: A Practical Harness Comparison for Real Repos
https://fieldjournal.ai/blog/codex-cli-vs-claude-codeHow enabling two settings tripled ARC-AGI-3 scores
https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scoresHidden Technical Debt of AI Systems: Agent Runtime
https://leehanchung.github.io/blogs/2026/04/24/hidden-technical-debt-agent-runtimeOpenAI model guide (living documentation)
https://developers.openai.com/api/docs/guides/latest-model?model=gpt-5.5Tools and execution 11
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
https://arxiv.org/abs/2405.15793Introducing advanced tool use for Claude
https://www.anthropic.com/engineering/advanced-tool-useCode Mode: the better way to use MCP | Cloudflare Blog
https://blog.cloudflare.com/code-modeCMU AI Agents, Lecture 2: Tool Use
https://www.cmu-agents.com/slides/lecture-02-tool-use.pdfThe Scaffolding Matters More Than the Interface: A Controlled Comparison of MCP and CLI Tool Use Across Seven Agent Scaffoldings, Five Language Models, and One Software Task
https://arxiv.org/abs/2608.08654OpenJarvis: Personal AI, On Personal Devices
https://arxiv.org/abs/2605.17172Uber: Efficient Software Factory
https://www.uber.com/ca/en/blog/efficient-software-factorySWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (NeurIPS proceedings)
https://proceedings.neurips.cc/paper_files/paper/2024/file/5a7c947568c1b1328ccc5230172e1e7c-Paper-Conference.pdffx built-in tools
https://fx.sh/docs/capabilities/toolsDeepSeek Harness
https://www.deepseek.com/en/harnessDeepSeek Harness architecture
https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/architecture.mdSearch and retrieval 4
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
https://arxiv.org/abs/2005.11401Microsoft GraphRAG
https://www.microsoft.com/en-us/research/project/graphragBRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
https://arxiv.org/abs/2407.12883Training a Specialist Code Search Agent with turbopuffer | Applied Compute
https://www.appliedcompute.com/case-studies/turbopufferContext, memory, and skills 47
What Happens When the Model Eats the Stack? Rethinking the Research Agenda for Data Agents to Withstand the Bitter Lesson
https://arxiv.org/abs/2609.031411M context is now generally available for Opus 4.6 and Sonnet 4.6
https://claude.com/resources/articles/1m-context-gaSession and context management diagram (Thariq Shihipar)
https://x.com/trq212/status/2044548257058328723Rewind file changes with checkpointing
https://code.claude.com/docs/en/agent-sdk/file-checkpointingCompaction on demand
https://platform.claude.com/docs/en/build-with-claude/compaction-on-demandCliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
https://arxiv.org/abs/2609.26779ACON: Optimizing Context Compression for Long-horizon LLM Agents
https://arxiv.org/abs/2510.00615Summer Yue: original OpenClaw incident report
https://x.com/summeryue0/status/2025774069124399363Summer Yue: explanation of the OpenClaw incident
https://x.com/summeryue0/status/2025836517831405980Context Language Models
https://arxiv.org/abs/2609.37725CMU AI Agents, Lecture 4: Memory and Skills
https://www.cmu-agents.com/slides/lecture-04-memory-and-skills.pdfAGENTS.md
https://agents.mdSkills
https://developers.openai.com/api/docs/guides/tools-skillsMemory tool
https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-toolEquipping agents with Agent Skills
https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skillsSkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
https://arxiv.org/abs/2602.12670How to Evaluate Agent Skills (And Why You Should)
https://openhands.dev/blog/evaluating-agent-skillsAgent Skills Evaluation Tutorial
https://github.com/rajshah4/evaluating-skills-tutorialFrontierMemory 1.0 | FrontierMemory
https://frontiermemory.com/blog/v1MemoryTax: The Bitter Lesson of Agent Memory
https://arxiv.org/abs/2610.29823CMU AI Agents, Lecture 3: Long Context
https://www.cmu-agents.com/slides/lecture-03-long-context.pdfShared repository instructions across tools (Thariq Shihipar)
https://x.com/trq212/status/2092302273099796842What Context Does a Coding Agent Actually Need to Act?
https://arxiv.org/abs/2607.09691QM: Multiplayer Agent Harness
https://github.com/yc-software/qmLong-context recall (Sinan Ozdemir)
https://www.linkedin.com/posts/sinan-ozdemir_agenticai-llm-rag-ugcPost-7428125462201102336-B7BrInvestigating how Codex context compaction works - Simon's Dream Universe
https://simzhou.com/en/posts/2026/how-codex-compacts-contextCompaction
https://developers.openai.com/api/docs/guides/compactionRecursive Language Models
https://arxiv.org/abs/2512.24601Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering
https://arxiv.org/abs/2604.08224Hermes Agent
https://github.com/NousResearch/hermes-agentDo Personalized Skills Help Coding Agents? An Empirical Study of Developer Interaction Histories
https://arxiv.org/abs/2608.10319Changes in the system prompt between Claude Opus 4.6 and 4.7
https://simonwillison.net/2026/Apr/18/opus-system-promptAgentic Context Engineering: Evolving Contexts for Self-Improving Language Models
https://arxiv.org/abs/2510.04618Agentic Context Engineering: code and examples
https://github.com/kayba-ai/agentic-context-engineYour harness, your memory
https://www.langchain.com/blog/your-harness-your-memorySarah Wooders, original post linked by LangChain
https://x.com/sarahwooders/status/2040121230473457921Context windows
https://platform.claude.com/docs/en/build-with-claude/context-windowsLLMs Get Lost In Multi-Turn Conversation
https://arxiv.org/abs/2505.06120Lost in Conversation: code and evaluation
https://github.com/microsoft/lost_in_conversationModifying system prompts
https://code.claude.com/docs/en/agent-sdk/modifying-system-promptsThe new rules of context engineering for Claude 5 generation models / claude.dev Blog
https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-modelsDeep Agents v0.7
https://www.langchain.com/blog/deep-agents-v0-7OpenHands Context Condensation for More Efficient AI Agents
https://www.openhands.dev/blog/openhands-context-condensensation-for-more-efficient-ai-agentsDo Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories
https://arxiv.org/abs/2607.27250Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
https://arxiv.org/abs/2602.11988Configuration Smells in AGENTS.md Files: Common Mistakes in Configuring Coding Agents
https://arxiv.org/abs/2606.15828SkillsBench discussion and commentary (Rajistics)
https://www.reddit.com/r/rajistics/comments/1r77v1h/skillsbench_showed_models_arent_good_atModel routing 6
Introducing Cursor Router
https://cursor.com/blog/routerLessons from building Claude Code: Prompt caching is everything / claude.dev Blog
https://claude.com/blog/lessons-from-building-claude-code-prompt-caching-is-everythingGLM 5.2 vs. Sonnet 5: comparing open and closed models on software tasks
https://www.linkedin.com/pulse/glm-52-vs-sonnet-5-comparing-open-closed-software-tasks-zain-hasan-nnlzcAgent-as-a-Router: Agentic Model Routing for Coding Tasks
https://arxiv.org/abs/2606.22902GLM-5.3 vs. Kimi K3: What Coding Agent Benchmarks Don’t Tell You
https://friendli.ai/blog/kimi-k3-glm-5.3-coding-agent-benchmarkInteractive benchmarks: evaluating model routing for AI agents
https://www.notdiamond.ai/blog/interactive-benchmarks-a-new-methodology-for-evaluating-model-routingGoals, verification, and boundaries 28
Astra for Coding: Why Are We Doing This Again? | Armin Ronacher's Thoughts and Writings
https://lucumr.pocoo.org/2026/9/7/astra-whyDeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
https://arxiv.org/abs/2501.12948AI achieves silver-medal standard solving International Mathematical Olympiad problems: Google DeepMind
https://deepmind.google/blog/ai-solves-imo-problems-at-silver-medal-levelBuilding a C compiler with parallel Claudes
https://www.anthropic.com/engineering/building-c-compilerkarpathy/autoresearch: AI agents running research on single-GPU nanochat training automatically
https://github.com/karpathy/autoresearchBeyond permission prompts: making Claude Code more secure and autonomous
https://www.anthropic.com/research/claude-code-sandboxingCMU AI Agents, Lecture 7: Computer Use Agents
https://www.cmu-agents.com/slides/lecture-07-computer-use-agents.pdfWhen May an Agent Stop? Evidence-Carrying Termination for Tool-Using LLMs
https://arxiv.org/abs/2608.23623Using the Ralph Wiggum Loop
https://www.ronin.consulting/artificial-intelligence/using-the-ralph-wiggum-loopThe Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks
https://arxiv.org/abs/2502.08235Long-running Agents - by Addy Osmani - Elevate
https://addyo.substack.com/p/long-running-agentsPractical Loop Engineering - by Addy Osmani - Elevate
https://addyo.substack.com/p/practical-loop-engineeringEvaluation leakage and model behavior (Erik Schluntz)
https://x.com/ErikSchluntz/status/2030042086679220676Changes in refusal behavior (Fireworks AI)
https://x.com/fireworksai_hq/status/2084374420295209209Why we built the benchmarking system before the model
https://hopitai.substack.com/p/why-we-built-the-benchmarking-systemUncovering a universal offline sandbox escape
https://www.primeintellect.ai/blog/universal-offline-sandbox-escapeA Rubric-Supervised Critic from Sparse Real-World Outcomes
https://arxiv.org/abs/2603.03800Reflexion: Language Agents with Verbal Reinforcement Learning
https://arxiv.org/abs/2303.11366Inside Claude Code With Its Creator Boris Cherny
https://www.youtube.com/watch?v=PQU9o_5rHC4Using `deepagents` to Build Deep Research (Python)
https://www.youtube.com/watch?v=geTtqyFnyHAYC-Bench: Benchmarking AI Agents for Long-Term Planning and Consistent Execution
https://arxiv.org/abs/2604.01212CMU AI Agents, Lecture 6: Coding Agents
https://www.cmu-agents.com/slides/lecture-06-coding-agents.pdfCMU AI Agents: Assignment 1
https://github.com/cmu-agents/assignment-1Goals, plans, and approval boundaries (Vaibhav Srivastav)
https://x.com/reach_vb/status/2099397110164488259Unrolling the Codex agent loop
https://openai.com/index/unrolling-the-codex-agent-loopPrime Agent: Long-Running and Background Agents
https://github.com/PrimeIntellect-ai/prime-agent/blob/main/docs/long-running-agents.mdVending-Bench
https://lukaspetersson.com/assets/pdf/vbench_paper.pdfCMU AI Agents, Lecture 11: Advanced RL
https://www.cmu-agents.com/slides/lecture-11-rl-advanced.pdfMultiple agents 21
Building effective AI agents
https://www.anthropic.com/engineering/building-effective-agentsHow we built our multi-agent research system
https://www.anthropic.com/engineering/multi-agent-research-systemSingle-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
https://arxiv.org/abs/2604.02460Towards a Science of Scaling Agent Systems
https://arxiv.org/abs/2512.08296Scaling Inference-Time Compute with Teams of Coding Agents
https://arxiv.org/abs/2610.17471Do Agent Teams Pay Off? A Case Study on Vibe Code Bench
https://www.vals.ai/blogs/multi-agent-vibe-code-benchThe Multi-Agent Architecture That Actually Ships: Luke Alvoeiro, Factory
https://ai.engineer/talks/ow1we5PzK-o-production-multi-agent-architectureNoam Brown - Agent swarms, alignment, & recursive self-improvement
https://www.dwarkesh.com/p/noam-brownBuilding agents: When to fold subagents into your main agent loop
https://claude.com/resources/articles/building-agents-when-to-fold-subagents-into-your-main-agent-loopScaling Discovery through Test-Time Communication
https://arxiv.org/abs/2609.21032Enterprise Multi-Agentic Conversational AI Workflow (Capital One, NVIDIA GTC)
https://www.nvidia.com/en-us/on-demand/session/gtc25-s74439Evaluating multi-agent systems in enterprise tool use
https://snorkel.ai/blog/multi-agents-in-the-context-of-enterprise-tool-useFrom Chaos to Choreography: Multi-Agent Orchestration Patterns That Actually Work: Sandipan Bhaumik
https://www.youtube.com/watch?v=2czYyrTzILgThe Multi-Agent Architecture That Actually Ships (Factory)
https://www.youtube.com/watch?v=ow1we5PzK-o&t=394sSubagents and handoffs (Boris Tane)
https://x.com/boristane/status/2099559571195080975Vals AI: agent teams benchmark discussion
https://x.com/ValsAI/status/2108608719420600709Capable language models can outgrow the benefits of collaboration | Nature Machine Intelligence
https://www.nature.com/articles/s42256-026-01268-yPatterns and problems in multiagent systems
https://www.anthropic.com/research/multiagent-systemsWhy Do Multi-Agent LLM Systems Fail?
https://arxiv.org/abs/2503.13657Refactoring Hermes with 1,393 agents | Nous Research
https://nousresearch.com/refactoring-hermes-with-1393-agentsHermes whole-codebase refactoring pull request
https://github.com/NousResearch/hermes-agent/pull/102117Adaptive and self-improving harnesses 9
Is the Agent Harness Dead? (Jediah Katz)
https://www.youtube.com/watch?v=7Y4sbZgBdncMeta-Harness: End-to-End Optimization of Model Harnesses
https://yoonholee.com/meta-harnessYounes Belkada: task-specific harness experiment setup
https://x.com/yb2698/status/2113669310183035233Younes Belkada: task-specific harness experiment results
https://x.com/yb2698/status/2113669293871358274Harness²: Recursive Agent Harnessing for an Open World
https://arxiv.org/abs/2610.14224Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories
https://arxiv.org/abs/2608.02276Prime Agent: A Self-Improving RLM Harness
https://arxiv.org/abs/2608.23552Harness Continual Learning: Continual Adaptation Beyond Model Parameters
https://arxiv.org/abs/2608.19013Agents: Inner Loop vs Outer Loop
https://www.philschmid.de/inner-loop-vs-outer-loopCourses and lectures 2
CMU 11-768: AI Agents
https://www.cmu-agents.comCMU AI Agents, Lecture 12: RL Systems
https://www.cmu-agents.com/slides/lecture-12-rl-systems.pdfKeep exploring
Watch the video gallery, try the workshop, or explore the projects.