400 independent implementation problems
Every ARC task could require a different algorithm, builder, test process, and optimization strategy.
Rajistics Replicated
Technical evaluation
Multi-agent orchestration for the NeuroGolf workload
NeuroGolf 2026 was a Kaggle competition in which each task needed a correct ONNX program. The rules did not require agents, but the 400 independent problems made parallel agent work useful. We evaluated how OpenHands can execute that work, how a controller coordinates it, and how persistent state allows later agents to reuse validated findings.
System components3controller, execution, and persistent state
Execution structures4compared with live OpenHands agents
Controller approaches4implemented or evaluated
Scaling simulation400task owners with controlled coverage
What the Kaggle competition required
NeuroGolf had 400 separate ARC tasks, each requiring a correct and efficient ONNX program.
Each task described an input-to-output transformation. A submission needed an ONNX graph that reproduced that transformation correctly. Once a graph worked, the competition rewarded reducing its memory use and parameter count.
Correctness first, then program efficiency
A smaller graph was useful only after it passed the task's examples and independent validation.
Many candidate implementations per task
Teams needed to preserve failed approaches, compare candidates, and keep the best validated result for each task.
The three parts of a multi-agent system
Controller, execution, and memory solve different problems.
The controller decides what happens next. The execution layer runs the agents. Memory preserves enough state for later work to improve, recover, and avoid repeating earlier experiments.
Controller
Select, assign, limit, observe, validate, retry, and stop work.
Execution
Run agents with the conversation, sandbox, and isolation boundary the work requires.
Memory
Retain task state, attempts, candidates, failures, artifacts, and validated lessons.
OpenHands supplies several execution structures. Controller logic and persistent workflow state remain application responsibilities, even when the controller runs as an OpenHands automation or beside Agent Canvas.
Why teams used multiple agents
The 400 tasks could be worked on in parallel, but the work still needed central coordination.
Kaggle did not require agents. The workload made them useful: different workers could try different tasks and approaches at the same time. Without a shared task registry and validation process, however, agents could repeat the same work while other tasks were never attempted.
Register
Keep one record for each of the 400 competition tasks.
Assign
Give every attempt a task, owner, budget, and stable ID.
Experiment
Run candidate implementations in bounded agent workspaces.
Validate
Check results outside the agent before accepting them.
Record
Save failures, improvements, artifacts, and reusable lessons.
What this project reproduced
We evaluated several ways to coordinate, execute, and retain state across agent work.
Live OpenHands tests compared execution and controller approaches. A separate deterministic simulation tested scheduler coverage at the 400-task scale without presenting that simulation as model-agent or ONNX performance.
Four controller approaches
We tested bounded external control, scheduled Enterprise automation, a persistent polling implementation, and an in-cluster Agent Canvas controller.
Four execution structures
We compared isolated and grouped Enterprise conversations, Agent Canvas, and parent/subagent delegation using matched workloads.
The same control path can run NeuroGolf workers
A production run would replace the controlled workers with licensed ONNX builders and validators while keeping the scheduler, attempt history, shared memory, and recovery path.
Scope of this test
This test focused on multi-agent orchestration.
The results give us enough evidence to proceed with OpenHands as the execution layer for a full NeuroGolf run. OpenHands would manage agent conversations and sandboxes; the application would continue to own scheduling, experiment history, validation, and promotion.
Conclusion: OpenHands can provide the agent execution layer for this workflow. The remaining work is domain and production integration, not a different orchestration architecture. A 100-agent deployment should still pass multi-controller and API load tests before it is treated as proven at that concurrency.
Where the orchestration lived
OpenHands ran the agents; our application ran the research organization.
The experiments were not an all-in-one OpenHands feature. I used controller code outside the agent runtime to decide what should run, verify the result, and preserve workflow state. OpenHands supplied the execution and lifecycle layer.
Campaign policy and evidence
Task registry, scheduling, coverage, attempt ledger, retrieved lessons, output contracts, deterministic validation, promotion, admission control, retry policy, and cleanup decisions.
Agent execution and runtime lifecycle
Model and tool loop, conversations, native subagents, workspaces, sandbox creation or grouping, event history, usage metadata, and pause controls.
Objectives and final authority
Benchmark design, risk boundaries, stopping rules, review of unusual failures, release approval, and the final Kaggle submission remain human decisions.
Practical interpretation: most of the organizational intelligence was application code outside OpenHands; most of the agent-execution mechanics were inside OpenHands. The controller could itself run as an OpenHands parent or automation, but its durable ledger and independent validators should remain application-owned.
Comparison with public NeuroGolf solutions
Competition teams built persistent research pipelines around general coding agents.
The reviewed writeups describe custom schedulers, task files, checkpoint state, validators, and promotion gates around Codex, ChatGPT, or Claude Code. Their competitive advantage came mainly from domain context and validation—not from a packaged multi-agent runtime.
A persistent research pipeline
The published pipeline treated task state, candidate history, validation, and promotion as durable parts of the competition system. Repeated agent work improved a retained portfolio instead of starting from an empty prompt each time.
Custom files and Codex/ChatGPT handoffs
The team maintained promoted history, rejected attempts, retry state, negative knowledge, task ZIPs, and local checks, then merged the UI and local histories after each attempt.
A custom resumable Codex scheduler
A bounded worker pool used one persistent session per task, backup and restore, strict validation, checkpoint files, and a serialized step for consolidating proven techniques.
External task ownership prevented agent drift
One public postmortem found that agents changed tasks based on local ROI. Restricting each worker to one task workspace restored global coverage control.
Where OpenHands helps: it can replace custom process, sandbox, conversation, event, pause, and operator-visibility glue. It does not replace the NeuroGolf task context, ONNX builders, adversarial validators, scheduling policy, or promotion logic that separated the strongest teams.