Agentic Infrastructure: When AI Actually Does the Work

The Weekend That Disappeared

I was going to do an automated AI news gathering system every day, but after three hours, I had to sit back and solve Python path issues, Confluence API rate limits, and wonder why a seemingly "simple" task meant touching five different systems. This is what happens in the home lab: you build a test environment to control and learn, but it turns out to be a maintenance job instead.

The Shift

Open the terminal, check the logs, Google "why is my cron not running" and then continue with the data for up to 45 minutes vanish into rabbit holes WhatsApp to my agent, "setup the news scanner with AI every day, Hit TechCrunch and Hacker News, summarize, and dump structured analysis, and save it in my memory file". 20 minutes later, not only did the job start, subagents deployed, the source was found and output formatted. It's not that it automated something. Scripts do that. The agent decided what to do, executed it, and remembered what it did.

What "Agentic" Actually Means

Not just a chatbot, not just a helper, but not just saying "you might be able to do this" Agentic is using AI to AI with tool access that does rather than suggests in my system consists of: SSH connection to Proxmox nodes, file system management via LXC, timing with Cron, using GitHub API and connecting via WhatsApp or Discord The agent doesn't recommend. My AI does not just have AI but actually makes it work both pct exec, file writing, commits changes and reporting results back

Four Real Examples

1. Daily Intelligence Gathering

Every morning at 8:00 AM, a cron spawns subagents that hit TechCrunch, Hacker News and ArXiv. These subagents run in parallel. Each summarizes its feed, then I will get a summary of content related to the technology I used before drinking my coffee. More importantly, these agents will remember my infra.json, my repos, and my preferences. It doesn't start fresh each session.

# Before:
ssh choz@hydra "cd /mnt/ai && python3 -m venv venv && source venv/bin/activate && python scraper.py"
# Then manually format. Then copy to Confluence.

# Now:
# [Agent runs it, reports: "Done. 12 opportunities, 3 relevant to your stack."]

2. Infrastructure Architecture Decisions

I'm planning kids' internet control. MikroTik hAP or MCP Gateway?

The agent didn't just suggest options. It analyzed:

• RouterOS scripting vs. Python MCP server complexity
• LXC resource overhead (512MB vs. 1GB per container)
• Security isolation tradeoffs (separate containers vs. shared)
• Generated a full decision matrix

Pro tip: Proxmox LXCs running AI workloads often need nesting=1 and keyctl=1 flags. Skip this and permissions will silently fail in unprivileged containers but work in VMs-a nightmare to debug.

3. ComfyUI Workflow Automation

Building a 4-scene cinematic trailer. The agent:

• Generated the JSON workflow from my concept description
• Validated node connections (no floating wires)
• Moved files to ~/apps/comfyui/user/default/workflows/
• Spawned editorial subagents to stress-test the technical approach

All from a single prompt. No manual JSON editing.

4. Multi-Repo Contribution Management

Every morning the agent scans my active repos:

• Issue backlogs and age
• Pending PR reviews
• Contribution opportunities matched against my MEMORY.md priorities
• Dependency updates (Renovate PRs)

It delivers a ranked list: "vervio-viberace foyer UI needs UI work. claude-squad needs CI/CD scaffolding."

I actually focus on work that matters instead of spinning on email.

Why This Works

Context persistence. Unlike shell scripts that start blank every time, the agent maintains state and the original script, topology, repos, schedules, and preferences Tool access. has access to SSH keys, API tokens, and the file system The agent executes, not suggests Conversational debugging. when a problem such as "timed job did not work" occurs The agent will check the source logs, troubleshoot the problem, and redeploys without context switches

How to Build This

1. Give it memory. Persistent files across sessions include infra.json and MEMORY.md The agent reads infra.json, MEMORY.md, and cron schedules before and after every action 2. Give it tools. covers SSH, API and file management, starting with read-only and gradually adding write permissions. 3. Define boundaries. Define what the system can touch and what requires approval My agent can create a file but will not delete the underlying infrastructure without confirmation. 4. Log everything. Log every action, output, and failure. You will need these logs when things go wrong.

For more on this, see running AI locally changed how I think. If you're managing multiple repos or systems across environments, the principles here overlap with what I built in VARGOS.

The Security Tension

"This sounds like giving root to a chatbot."

You're right. The risks are real:

• SSH key exposure
• API token scope creep
• Prompt injection
• "Oops, I deleted everything" scenarios

My mitigation: uses the principle of granting access as much as necessary, the agent will access via SSH to the specified LXC, not the Proxmox host, the use of the API will be limited in scope, and destructive operations require confirmation. However, there is a detailed record of use. Reliability is another issue. AI may write broken cron syntax, misunderstand firewall rules, or generate invalid JSON. To address reliability, a proof layer is used where generated configs are syntax-checked, network changes are staged before deployment, and failures are logged and learned from.

The Tradeoff

For basic infrastructure tasks that need to be repeated, such as cron management, managing system files, and coordinating work across multiple systems, having an agent that just does it, it's hard to say no to "AI-assisted manual". Maybe the ideal format for most teams is to let humans make decisions and let AI take action quickly, but the convenience of commanding "make it so" changed our thinking. I'm watching the logs. I'm just not going back to the terminal.