Monthly Archives: August 2026

I’ve been running Ollama on my home server since early 2024. I also have API access to Claude, GPT-4o, and Gemini. In 2025 I started using both simultaneously – routing some tasks local, some to the cloud, depending on what the job needed. After a year of that, I have opinions.

This isn’t a benchmark post. There are plenty of those. This is about the practical daily-use tradeoffs that don’t show up in leaderboards.

Where Local LLMs Win

Privacy for sensitive content – Anything involving personal information, client work, or content I don’t want stored on a third-party server goes local. Llama 3.1 70B running on my server has no network connection to the outside world after the model is downloaded. Whatever I send it stays on my hardware.

Cost at volume – I run a lot of AI-assisted workflows: content drafting, summarization, code review, classification tasks. At the volume I use, cloud APIs would cost $100-150/month. My local server is already running; Ollama’s marginal cost is electricity, which adds maybe $5-10/month on my power bill.

No rate limits – Local inference has no API rate limits. I can run a batch job that processes 500 documents overnight without worrying about hitting a per-minute token limit or paying premium-tier pricing for throughput.

Latency for short tasks – For quick generation tasks where the model is already loaded in VRAM, local inference is faster than a round-trip to a cloud API. My RTX 3060 generates Llama 3.1 8B output at around 60 tokens/second. That’s snappier than most cloud APIs under load.

Where Cloud AI Wins

Quality ceiling – The honest truth is that GPT-4o and Claude Sonnet are meaningfully better than any model I can run locally right now. For tasks that require sophisticated reasoning, long-context comprehension, or nuanced judgment, the frontier models win. My 70B quantized Llama is impressive but it’s not GPT-4o.

Multimodal capability – Cloud APIs handle vision, audio transcription, and document understanding at a level local models can’t match yet. I use Claude for analyzing screenshots and diagrams because the local vision models I’ve tried are noticeably worse.

No setup or maintenance – Cloud APIs just work. No GPU driver issues, no VRAM constraints, no container restarts when a model update breaks something. If you just want to build something, cloud APIs are faster to start with.

Model variety without storage cost – Running locally, every model I want available takes up disk space and VRAM budget. On cloud APIs, switching models is one parameter change. I don’t have to choose what to keep loaded.

How I Actually Split the Work

My routing logic in practice:

Go local: Anything with personal or client data. Batch processing jobs. Quick classification or extraction tasks where 8B-quality is sufficient. Things that need to run without internet access.

Go cloud: Complex reasoning or writing tasks where I need the best output I can get. Vision/image understanding. Anything where quality matters more than cost. Long-context tasks that exceed what I can run efficiently locally.

About 70% of my AI usage by token count is local. About 70% of my AI usage by task importance is cloud. Those numbers make sense together – local handles the volume, cloud handles the critical stuff.

The Hardware Reality

The quality gap between local and cloud narrows as you put more hardware behind local inference. A consumer RTX 3060 running a 7B model is not in the same league as GPT-4o. Two A100s running Llama 3.1 405B would be a different conversation.

Most home users are working in the 8-70B model range with consumer GPUs. At that level, the honest assessment is: local is good enough for a lot of tasks and not good enough for the hardest tasks. Know which category your task falls into.

What’s Coming

The gap is closing. Models that required 80B parameters two years ago now achieve comparable quality at 8B. Quantization techniques are improving. Hardware efficiency is improving. The trend is clearly toward more local capability per dollar of hardware.

My prediction: in 18 months, a 30B model on a mid-range GPU will be good enough for 90% of daily tasks. The 10% that needs frontier capability will still go to the cloud, but the volume will keep shifting local.

If you want to start running local models, the LincStation N1 is a clean way to get an Unraid box running. Add a GPU if you want inference speed – the Noctua NF-A12x25 fan handles the thermal load from a GPU addition cleanly.

Next: Home Assistant in 2026 – what I actually use it for and what I’ve given up on.

Products mentioned in this post:

• LincStation N1 6-Bay NAS with Unraid License
• Noctua NF-A12x25 PWM 120mm Fan

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.

Unraid’s default Docker interface is fine for getting started. You click through a template, fill in a few fields, and you’ve got a running container. But once you have 20 containers, that approach starts to break down. Things are inconsistently configured, you can’t easily reproduce the setup on a new machine, and there’s no version history for your container configs.

I switched to Compose-based management about 18 months ago. Here’s the structure I landed on.

Why Compose on Unraid

Unraid 6.12 added native Docker Compose support through the compose manager. Before that, you had to use the community plugin. Now it’s built in – you can define stacks in YAML files and manage them from the UI or command line.

The main benefit is reproducibility. My entire container stack is defined in YAML files stored in a git repository. If the server dies and I rebuild from scratch, I clone the repo and bring everything back up with docker compose up -d. That’s not fully automated yet – I still have to restore data volumes – but the configuration layer is covered.

The other benefit is legibility. A Compose file makes the dependencies, environment variables, network configuration, and volume mappings explicit in one place. No more hunting through the Unraid UI trying to remember why a specific port was mapped a certain way.

My Directory Structure

Everything lives under /mnt/user/compose/ on the array. Each stack gets its own subdirectory:

/mnt/user/compose/
  media/
    docker-compose.yml     # Plex, Radarr, Sonarr, Prowlarr, Jellyfin
  nextcloud/
    docker-compose.yml
    .env                   # DB passwords, admin credentials
  ai/
    docker-compose.yml     # Ollama, Open WebUI
  monitoring/
    docker-compose.yml     # Uptime Kuma, Dozzle
  network/
    docker-compose.yml     # AdGuard Home, Nginx Proxy Manager
  security/
    docker-compose.yml     # Vaultwarden, Authentik (if I ever fix it)

The grouping is functional – services that talk to each other or share a network go in the same stack. Services that are independent get their own stack so I can restart them without touching unrelated containers.

Environment Variables and Secrets

Each stack directory has a .env file for environment-specific configuration. Passwords, API keys, and paths that differ between environments go there. The .env files are excluded from the git repo (obviously) and backed up separately.

The Compose file references variables from .env like this:

environment:
  - POSTGRES_PASSWORD=${NEXTCLOUD_DB_PASSWORD}
  - NEXTCLOUD_ADMIN_PASSWORD=${NEXTCLOUD_ADMIN_PASSWORD}

This means the Compose files themselves can be safely stored in version control without exposing credentials. When I need to check “what database password did I use for Nextcloud,” I look in the .env file in the secure backup, not in git history.

Networking: Named Networks Over Default Bridge

The default Docker bridge network works but it’s loose. Any container on the bridge can reach any other container on the bridge by IP. That’s fine for a home server where you trust everything, but it makes it harder to reason about which services actually need to talk to each other.

I define named networks in my Compose files and assign containers only to the networks they need. The media stack has a media_net network. Radarr and Sonarr are on it. Plex is on it. The download client is on it. Nothing outside that stack is on media_net.

Services that need external access – AdGuard Home, Nginx Proxy Manager – have a separate proxy_net network. Vaultwarden is on both proxy_net (for external access) and an isolated vault_net (for the internal database connection).

Volume Management

I use bind mounts rather than named Docker volumes. This means data lives at a predictable path on the host file system rather than in Docker’s internal volume store. For a home server this is usually the right call – you can find your files with a file manager, back them up with normal tools, and restore them without Docker-specific knowledge.

The appdata path for configuration files is /mnt/user/appdata/[service-name]. Media and large data files go on the array at /mnt/user/[category]/. The distinction matters for backup strategy – appdata is small and critical (back it up frequently), while media files are large and recoverable (back up less often or not at all if you can re-download).

The Stack I’m Most Proud Of

The monitoring stack is the one I’ve refined the most. Uptime Kuma polls every service endpoint. Dozzle aggregates container logs so I can see what’s happening without SSH-ing in every time. A Telegram webhook notifies me if any service goes down and stays down for more than 2 minutes.

This setup has caught three real problems in the last six months: a Nextcloud container that silently exited after an update, a DNS resolution failure that took down local hostname routing, and a Plex database lock that was blocking transcoding. In each case I had an alert within 5 minutes of the failure.

What I’d Tell Someone Starting Fresh

Start with Compose from day one. The Unraid template UI is a fine way to understand the options for a new service, but immediately translate it to a Compose file before you forget what you configured. The time you spend doing that pays back the first time you need to rebuild something.

Store your Compose files somewhere safe. A private git repo is ideal. A local backup is the minimum. These files represent hours of configuration work – treat them like code.

The LincStation N1 is a solid appliance if you want to get started with Unraid and Compose without building a server from scratch. It ships with an Unraid license and has enough headroom for a basic Docker stack.

Next post: local LLMs versus cloud AI, from someone who runs both every day.

Products mentioned in this post:

• LincStation N1 6-Bay NAS with Unraid License
• LincStation S1 6-Bay Desktop NAS

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.

The question I get most often when I tell people I run a homelab is: why bother? Google Drive is free. Spotify is ten bucks a month. Dropbox just works. Why spend a Saturday afternoon debugging a Nextcloud sync issue when you could be doing literally anything else?

It’s a fair question. Here’s my honest answer.

The Control Argument

When you use a cloud service, you’re renting access to your own data. The terms can change. The pricing can change. The service can shut down. Most of the time none of those things happen in ways that hurt you, but the possibility is always there.

I’ve had three services I relied on go away or change enough to be useless in the past five years. Google Photos changed its storage policy. Dropbox changed its free tier. Evernote went through multiple owners and quality cliff dives. Each time, I had to migrate data and rebuild a workflow I’d gotten used to.

Self-hosting isn’t immune to disruption – hardware fails, software stops being maintained. But the timeline is in my hands. I don’t wake up to an email saying I have 30 days to migrate.

The Privacy Argument

I’m not especially paranoid. I use Google Maps. I have a Gmail address. I’m not trying to disappear from the internet.

But there’s a category of data where I’d rather not have it indexed by someone else’s machine learning pipeline: family photos, financial records, health information, personal writing. For that category, local storage with encrypted backups is just cleaner. I know where it is and I know who can access it.

Immich for photos, Vaultwarden for passwords, Nextcloud for documents. Those three replacements cover most of the personal data I care about.

The Cost Argument

This one is context-dependent. For a single person with modest data needs, self-hosting is probably more expensive when you factor in hardware, electricity, and your time. Cloud services exist because they’re cheaper at small scale.

At family scale, the math changes. I’ve got six devices syncing photos. Immich is free; iCloud family storage at the size I’d need runs about $100/year. Vaultwarden is free; 1Password family is $60/year. Nextcloud is free; Google One at a comparable storage tier is $30/year. The breakeven on a few hundred dollars of hardware is under two years, and the hardware keeps running after that.

The honest caveat: I’m not counting my time. If you bill at $100/hour and spend 20 hours a year on homelab maintenance, the economics look very different.

The Learning Argument

This is the one I don’t hear talked about enough. Running a homelab teaches you things. I understand networking better because I had to set up VLANs. I understand containerization because I run Docker every day. I understand backup strategies because I’ve had drives fail.

That knowledge is worth something professionally. Every homelab project I’ve done in the last three years has shown up in some form in actual work – a conversation about infrastructure, a debugging session where I already knew the underlying concepts, a tool I could set up because I’d done something similar at home.

What I Don’t Self-Host (And Why)

Email. I’ve thought about it, gotten close, and stepped back every time. Email deliverability from a residential IP is a constant fight against spam filters. The cost of getting it wrong is real – a misdirected invoice or missed calendar invite is a real problem. I use Fastmail and consider it money well spent.

Video conferencing. Running a Jitsi instance sounds fun until you’re on a call with your kid’s teacher and the connection is bad because your upstream bandwidth is maxed. Some services are better as someone else’s problem.

AI APIs for heavy workloads. I run Ollama locally for most things, but when I need GPT-4-level capability at scale, I use the API. The cost of running sufficient hardware to match cloud AI performance is too high to justify for occasional use.

The Right Hardware to Start

You don’t need a full server. A LincStation N1 with an Unraid license included is a reasonable starting point if you want appliance-style simplicity. If you want more control and are comfortable with a bit more setup, build or buy a mini PC with a spare external drive and start with Nextcloud and Vaultwarden. That covers the two highest-value self-hosted services for most people.

Whatever you start with, get a UPS on it. An APC BX1500M is what I use. Unexpected shutdowns are how you lose data.

The important thing is starting small. One service, running well, is better than ten services that need constant attention.

Is It Worth It

For me, yes. I like having control over my infrastructure, I enjoy the learning aspect, and the cost math works out at household scale. I don’t think everyone should self-host everything, and I’m not interested in convincing people it’s morally better. It’s a tool. For some people and some use cases it’s the right tool.

Next post: how I organize my Docker container stack on Unraid with Compose files, including the structure that finally made sense after two years of iteration.

Products mentioned in this post:

• LincStation N1 6-Bay NAS with Unraid License
• APC BX1500M UPS 1500VA
• Noctua NF-A12x25 PWM Fan

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.