An AI agent in my homelab

Most of my homelab work used to happen at the desk, after the kid was asleep. Now a lot of it happens on my phone, in Telegram, talking to an agent called Hermes. “Why is Jellyfin slow tonight?” “Bump Paperless and tell me what changed.” It goes, looks, and comes back with an answer — or a pull request.

It writes its own runbooks

Hermes Agent is an open-source agent by Nous Research, running in its own container on my home server. What sold me is that it improves itself. It works from skills — short Markdown runbooks for each part of the house — and I don’t write them, Hermes does. When a fix takes some trial and error, it saves what worked, so the next session starts where the last one ended. It also remembers how I like things done: one recommendation instead of five, verify before claiming.

So when it learns that Paperless can’t be snapshotted and needs a database dump before every update, it writes that down once and doesn’t trip on it twice. Most assistants start every conversation from zero. This one gets a little better each time I use it.

What it does all day

Mostly boring things, which is the point. Every morning a handful of checks land in Telegram: drive health, unhealthy containers, last night’s backups. They used to be n8n workflows that sometimes reported failures as success; now they’re plain scripts on Hermes’s cron, and a failure is loud. The rest is on demand — reading dashboards and logs, digging into whatever looks off, and handling what Renovate can’t.

The fences

Giving a language model access to your infrastructure is a strange thing to do on purpose, especially one that reads untrusted text all day. So I spent more time on what it can’t do:

• Commands need a yes. Anything dangerous waits for my approval in Telegram; scheduled jobs can’t run them at all.
• Least privilege. Every credential gets the narrowest scope that works, most of them read-only.
• It proposes, I merge. Anything that changes what runs is a pull request, and merging stays my tap. Only docs go straight in.
• Nothing listens on the internet. Outbound calls only, and the bot answers me alone.
• The main agent stays in Switzerland. Its model runs at Infomaniak, the same host as this site, so my logs and configs don’t end up on a US cloud.

It’s not perfect. It still gives the odd confident wrong answer, and an approval prompt is easy to tap through when you’re half-watching a match. That one’s on me.

The takeaway

The model was never the hard part. Decide what it can touch, let it write down what it learns, and keep the last tap for yourself — then an agent stops being a toy and becomes the colleague who does the morning checks 🤖.