Local LLM lab and Mission Control

The problem

Experimenting with agents on hosted APIs hides the questions that matter when you run them for real: which model is good enough for which task, what it costs, and how you know what the system is doing right now.

What I built

• Lab: a self-hosted environment for running and evaluating local and hosted models, agents and multi-agent workflows.
• Orchestration: tasks are delegated between specialist agents and different models according to capability, measured performance and resource needs. Benchmarking and routing balance output quality, compute and cost.
• Mission Control: a web layer for managing agents and jobs and exposing system state, with observability treated as a design requirement rather than a later addition.

What I learned

• Observability is the difference between an experiment and a service: if you cannot see what an agent is doing, you cannot trust it with anything.
• Model routing is mostly a measurement problem. Once the benchmarks exist, the routing decisions are easy.