Hermes Agent on two DGX Sparks
Since June 28 I've run Hermes Agent from Nous Research around the clock at home. It talks to a model served on two DGX Sparks in the same room; chat never goes to a cloud API. This is what runs, what I changed in the engine so agent traffic works, and what broke in Hermes and how I found it.
What runs
| Machines | 2× NVIDIA DGX Spark (GB10), 121 GiB unified memory each, joined by two ConnectX-7 cables (218 Gb/s) |
|---|---|
| Model | DeepSeek-V4.1-Flash-1017, the MiaAI-Lab 2.9 bpw build, split across both machines on the TensorFold engine |
| Serving | 1,048,576-token context, 4 streams, vision, thinking on |
| Measured | GSM8K 99.2 · HumanEval 96.3 · 72 tok/s on one stream (my harness, Oct 19) |
| Hermes | v0.22.5: a Telegram chat profile (1,499 sessions) and a delegated-work profile (424 sessions) |
| Scheduled | 46 jobs: a morning brief, an overnight research queue, health checks |
One Telegram turn averages 22.7 model steps. Every request opens with about 27,000 tokens of system prompt and tool schemas, and the engine's prefix cache serves 97% of a request from memory. Here, the cost of an agent isn't the prompt. It's the round trips.
What I changed in the engine
Agent traffic isn't chat: long prompts, many short steps, tool calls in between. I sent four patches to the
recipe. Three are on its main branch, credited in its CREDITS.md.
| Patch | Problem | Result | Status |
|---|---|---|---|
| Prefill interleave | One long prompt froze every live stream until its whole prompt pass finished. | Longest gap 90.1 s → 1.3 s (maintainers' run). Write-up | On main |
| Tool-call prose | A non-streamed reply with tool calls came back as content: null; a 3,671-character plan was dropped. | Long tool replies round-trip 3/3 (was 0/3) | On main |
| Plain request text | Text that spelled a control token was read as that token, so a fetched page or file could forge a user turn, </think> or a tool call. | Spelled tokens stay text; clean prompts byte-identical | On main; the maintainers extended it to one more route |
| Step-ahead | Each agent step re-read its own previous reply from scratch. | 8-step loop: first token 1.99 → 1.34 s, output identical 8/8 | Open; needs a rebase after an engine rework |
What running Hermes all day taught me
Four findings, with how I checked each. I re-read every one against Hermes main today, and where
upstream has changed, I say so.
A context window belongs to the running server, not the config. I changed the server to 524,288 tokens over 4 slots (131,072 each), but two profiles still told the summarizer it had 65,536. Hermes lowered its compaction threshold to fit and carried on with no error. Sessions compacted, failed to shrink, compacted again and reset: five ended in under three hours, two of them 31 seconds apart. On Telegram it looked like the bot had stopped listening. The 86,016 in the log was 122,880 × 0.7, which pointed to the exact key. The window is set in three places per profile, and the alias wins.
The compaction threshold has a floor. Under a 512K window, Hermes raises any threshold below 0.75 to 0.75. That's
deliberate and commented in agent/context_compressor.py: lower values make compaction fire again every turn
or two. So my 0.50 had been 0.75 all along. I ran the compressor directly: 0.85 with a 0.35 target keeps 34,119
tokens per compaction instead of 21,504, with about the same room before the next one (63,365 tokens against 64,512).
A database check said ok on a broken index. The session database's full-text index was corrupt for 36 hours,
and three nightly jobs died trying to write to it. PRAGMA integrity_check passed. The FTS5
integrity-check command is the test that caught it. Hermes's in-place rebuild failed because the gateway
was still writing; the same rebuild with the gateway stopped took about a second and lost no rows. My repair script
now does that (back up, stop, rebuild, restart, at most three times a day) and runs every 10 minutes, so a corrupt
index lasts minutes, not a day and a half. The manual repair before it lost 235 messages, so the alert always reports
the message count.
A retry loop multiplied one bad setting. In July a Kanban worker hit a context-overflow 400 and was retried five times per card: 18 crashes in 66 minutes of GPU time. Upstream has since added exit codes that park auth and missing-model failures after one try, and the default failure limit is now 2.
One still open
On main today, a model whose name contains "deepseek" has an empty reasoning field replayed as a single
space (agent/message_sanitization.py). DeepSeek's API needs that: it rejects an empty value on a
tool-call turn. The rule also matches on the model's name, so it applies to my local engine too, and there the space
becomes part of the prompt: the replayed turn no longer matches what the model produced, from its first token on. On
October 21 that was 14% of my thinking-on tool turns, and each one missed the engine's warm state for that reply.
A setting that lets a local provider skip the pad would fix it, if the engine accepts the turn without it. I haven't
tested that yet, so I haven't filed it.
What I don't claim
The benchmark scores come from my own harness on my pair, not a leaderboard. The prefill numbers are the maintainers' run. The Hermes findings come from one heavily used install with a local server whose settings change under it; a cloud-backed setup may never see the first one. I'm not affiliated with Nous Research or MiaAI-Lab.
Sources: recipe CREDITS.md · PRs #5, #44, #43, #41 · Hermes Agent