Neo: an always-on engineering copilot on our own GPUs

How a LangGraph agent with local models, MCP tools, and an ideas loop became the rig's resident engineer — and why owning the inference changes what a copilot can be.

Cloud copilots clock out when the tab closes. Neo doesn’t. He’s a LangGraph agent that lives on our GPU rig, speaks the same OpenAI-compatible API every tool already knows, and keeps a background loop running on his own ideas when nobody’s talking to him.

Neo's heads-up display: a glowing green orb core flanked by inference-lane utilization bars, a live ideas-loop feed, delegation triggers, and guardrail readouts — Neo's HUD. Inference lanes on the left, the ideas-loop feed on the right, guardrails always in frame. (Operational values sanitized for publication — the layout is the real thing.)

The problem

Per-token pricing quietly shapes engineering behavior: you stop asking exploratory questions, you batch your thoughts, you don’t leave an agent running overnight on “look through this and tell me what’s worth fixing.” The meter changes the work.

And for anything sensitive — client code, business data, half-formed product ideas — shipping every token to a third party is a decision, not a default.

The shape of Neo

Three decisions define him:

Speak the standard. Neo serves an OpenAI-compatible endpoint, so every existing client, script, and IDE plugs in with a base-URL change. No SDK lock-in, no custom protocol tax.
Local first, escalate deliberately. Day-to-day reasoning runs on rig-local models. When a task genuinely needs a frontier model, Neo delegates to coding subagents — an explicit, logged decision instead of a silent default.
A daemon, not just an endpoint. The ideas loop keeps working between conversations — reviewing, drafting, proposing. Everything lands as a draft for human review; nothing ships itself.

How a task actually moves

When work arrives — a chat message, a cron tick, or the ideas loop itself — Neo plans, routes to the right lane, and verifies before anything reaches a human:

Neo's delegation graph: a task enters the LangGraph lead node, fans out to local 32B coding subagents plus an explicitly logged frontier escalation lane, and everything converges on a human-gated draft queue — The delegation graph mid-dispatch. Solid green lanes are local models; the copper dashed lane is a frontier escalation — explicit, logged, and rare. Everything converges on the draft queue, where a human decides.

The copper lane is the part most setups get wrong. Escalating to a frontier model isn’t forbidden — it’s deliberate. Neo has to choose it, the choice is logged, and the day’s count sits on the HUD where it can’t hide.

Why owning the inference matters

When tokens are flat-cost, the copilot’s job description changes. “Read the whole repo and think about it overnight” becomes a reasonable ask. Exploratory conversation stops feeling like a taxi meter. And the privacy boundary is physical: the documents, the code, and the half-baked ideas never leave the building.

Where Neon fits

Our public site runs a different agent — Neon, sandboxed on Cloudflare’s edge with zero access to any of this. That’s the same architecture we build for clients: capable agents inside the boundary, purpose-limited agents outside it, and a bright line between the two.