What a 5-agent AI fleet actually costs to runA working analysis of architecture, tokenomics, and the framework behind specialization

Executive summary

This is a working analysis of a personal AI agent fleet — five distinct agents distributed across two hosts, each running a different model, each playing a specialized role. The artifact documents what it actually costs to run, what each model is genuinely good at, and the framework behind specialization. It is intentionally written so a reader without deep AI infrastructure background can follow it.

The thesis is simple: headcount of agents is not the value. Specialization is. Five general-purpose assistants is worse than two specialized ones — every interaction starts with "wait, which one did I tell about X?" and their memories drift apart. The value of a fleet goes up when each agent has a distinct job, a distinct cost profile, and a distinct trust boundary.

What you'll find in this page:

• The architecture, end to end — physical hosts → containers → harness → model → external providers
• A model-by-model benchmark on a single, quantifiable prompt — wall-clock, tokens, cost, correctness
• A host-level comparison — same prompt, run from each agent's container, to isolate hardware/runtime path
• A task-type comparison — same models, four different task shapes (math, code, summarization, creative), to surface what each model is actually good at
• The tokenomics — what every recurring job costs per week, what was redistributed to cheaper models, what was preserved on frontier
• A framework for thinking about future fleet design — secure, cheap, best, focused

The architecture

Two physical hosts. Five Docker containers. Five different models. Three external model providers. One local model service. Inter-agent communication is the next layer up — the A2A protocol is the public standard that makes this kind of fleet legible to other systems.

Functional names used throughout this analysis. Real handles, ports, and network details are intentionally not shown.

The fleet

Each agent has a role, a model tier, and a cost shape. The roles map to the four archetypes worth keeping in mind for any future fleet design: best, workhorse, mobile, restricted/public-facing, and always-on/private.

Methodology

Every comparison in this page is from a serial run — one model call at a time, with a 2-second gap between calls. Parallel runs were avoided so concurrent load couldn't confound the timing numbers.

The standard prompt

For the model benchmark and the host-level comparison, all calls used the same prompt — chosen because it has one correct numerical answer and requires brief reasoning prose:

A store sells apples for $0.85 each. I have $20 and want to buy as
many as possible while leaving enough to buy a $3.50 coffee with the
change. How many apples can I buy? Show your reasoning in 2-3 sentences.

Correct answer: 19 apples ($16.15 spent, $3.85 left → covers the $3.50 coffee).

Why this prompt: it tests arithmetic, constraint reasoning, and writing — all comparable across models. The answer is unambiguous (a model either says 19 or it doesn't), so "correctness" isn't a judgment call.

How cost was computed

Each provider returns token counts in their response (input_tokens, output_tokens, or the equivalent). Cost is computed against the public pricing in effect as of June 2026:

Hardware setup

Two hosts:

• Host A — Apple Silicon Mac Mini. Runs Colima + Docker. Hosts 4 of the 5 agents. Local Ollama service runs here for the Sentinel agent.
• Host B — Windows x64 PC. Runs Docker Desktop. Hosts 1 agent (the Mobile one, paired to WhatsApp).

Both hosts have the same Anthropic and Google API keys available to their containers, so the model availability matrix is identical except for local Ollama (Mac Mini only).

Model benchmark

Same prompt, every reachable model, run serially. This is the headline comparison: speed, cost, and correctness on a representative reasoning task.

Key takeaways

• Speed king: Gemini 2.5 Flash Lite — 0.57s, $0.00004. That's 50× cheaper than Opus and 3.5× faster. For high-volume, low-intelligence tasks, this is the right answer.
• Best price/performance: Haiku 4.5 — $0.0006 per call, 1.5s, correct. Frontier-tier Anthropic reasoning at ~13× less than Opus.
• Frontier on this task: Opus 4.7 and 4.8 are functionally tied; 4.8 was slightly faster and marginally cheaper this run.
• Free and correct: Local Gemma4 got it right, took 12 seconds. Good for non-urgent background jobs where $0 is the requirement.
• Don't use: nemotron-mini (math wrong even after thinking), gemini-3-pro-preview (Google retired it — silent 404).
• Tokenizer note: Anthropic prompts were ~40% larger by input token count for the same prompt (83 vs 58 tokens). Tokenization differs across providers and matters for cost projection.

Host-level comparison

Same prompt, same models, but each run is initiated from inside a different agent's container. This isolates host hardware, network path, and runtime overhead — useful for deciding which host should run which workload.

* Mac Mini's local Ollama service is effectively single-tenant — first parallel attempt dropped a connection; the timing above is from a serial retry.

What this tells us

• Hosts are not the bottleneck. All 5 containers land within ~1s of each other on Gemini Flash, and within 0.13s on Flash Lite. The dominant variable is the provider's time-to-first-token, not your hardware or network path.
• Windows host is only marginally slower than Mac Mini — ~0.4s on Flash, ~0.1s on Flash Lite. Consistent with the extra LAN hop. Not worth re-platforming for.
• Mac Mini's local Ollama is single-tenant. The 4 Mac-side containers all share one Ollama endpoint. Parallel hits drop connections. Any fleet workflow that dispatches multiple local-model calls must serialize them — or queue them.
• Local Gemma4 timing varies 14-25s across containers — same model, same endpoint, just different container's call. Variance is in the Ollama service, not the network. Plan for non-deterministic latency on local.

Task-type comparison

Speed and cost only tell you half the story. Different models are good at different shapes of work — math/logic, code generation, summarization, creative writing. The plan was four models × four task types. Anthropic's credit exhaustion limited what we could measure.

The accidental finding

The credit-exhaustion failure during the benchmark is itself instructive. The fleet kept working — because Gemini was still available and the Sentinel agent's local Gemma4 was free. The provider-diversity design rule didn't survive contact with the bill; it survived because of it. A fleet that depends on a single provider can't tolerate a billing event. A fleet with three independent providers (Anthropic + Google + local) degrades gracefully.

This is the case study for the "design for provider diversity" principle in the Lessons section. We didn't plan for the credit to run out during a public benchmark — but the architecture absorbed it.

Tokenomics — the real cost of running this

The interesting cost isn't a single API call. It's the recurring work — heartbeats, dreaming, inbox classification, polling — running unattended at 3am, every 30 minutes, every 10 minutes. That's where the bill compounds.

Per-job weekly cost (before redistribution)

Assuming each cron firing uses ~500 input + 1500 output tokens (a defensible midpoint for the prompts in production):

Per-job weekly cost (after redistribution)

The same jobs, routed to the model class that actually fits the work:

The guardrail — what was NOT moved off Opus

Cost reduction without quality loss is the whole point. These remain on Opus 4.8 because the output is published, brand-shaping, or genuinely complex:

• Daily Email & Calendar Briefing — synthesizes the morning. Once per day. Quality matters more than the $0.03 cost.
• Inbox Morning Digest — same shape: low-volume, high-leverage synthesis.
• Daily Reports → today.arnao.ai and rai.arnao.ai — published content. Brand voice depends on this. Always on frontier.
• Ad-hoc user requests — when the user pings the Apex agent directly, they're paying the Opus tax on purpose. Don't undermine that.

Lessons learned · a framework for fleet design

After a quarter of running this in production, three principles keep proving themselves out.

1. Specialize by role, not by handle

The most honest organizing principle is the one you'd describe to a stranger in one sentence per agent: "this one is the best; this one is the cheapest; this one is the always-on; this one is restricted." That's the right level of abstraction. Naming agents after pets or letters doesn't survive contact with a real workload — roles do.

2. Build for provider diversity

Quotas die. Models get deprecated. A whole provider can have an outage. When the fleet spans Anthropic + Google + local Ollama, no single provider failure takes the fleet down. The "free + always-on" agent is structurally the most important one because it's the only one that cannot fail for a billing reason.

3. The recurring work is where the bill lives

One off-the-cuff "explain this to me" request is rounding error. A 10-minute cron job firing 144 times a day silently into a frontier model is where the budget evaporates. The audit is worth doing — but the better discipline is to set per-job model at job creation time, never default to "the smartest thing available."

4. Treat the model whitelist as a runtime fact, not a static doc

This fleet was offline for a full day because a model documentation file said "Opus 4.8 works ✅" — but the runtime hadn't been upgraded to recognize it. The lesson: agent skills that change configuration should run a live probe before persisting, and a runtime upgrade should be part of the model-adoption workflow.

5. Functional names beat real names for public artifacts

The functional names used throughout this page (Apex / Operator / Mobile / Concierge / Sentinel) are not just for security — they force you to write about what each agent does, which is the thing that actually changes over time. The handles, ports, and bot tokens are implementation details.

What's next

This page is the first published artifact. It will be updated as the fleet evolves — particularly as host-level and task-type benchmarks complete, and as the inter-agent A2A protocol layer matures from primitive to production.

• Host-level + task-type benchmarks — landing in the next revision.
• Real inter-agent dispatch — today the fleet is five parallel chats with one person. The interesting demo is a real A2A v1.0 round-trip: Concierge takes a request, delegates to Operator, who delegates a complex piece to Apex, who returns a result the Concierge surfaces.
• Commercial fork — the "watch agents collaborate" demo is the public-facing version of something more interesting: a dedicated commercial-data agent (separate trust boundary) chat-interfacing with the podcast transcript corpus this fleet already produces.