Words & Audio About Nothing

I’ve been running Ollama on my home server since early 2024. I also have API access to Claude, GPT-4o, and Gemini. In 2025 I started using both simultaneously – routing some tasks local, some to the cloud, depending on what the job needed. After a year of that, I have opinions.

This isn’t a benchmark post. There are plenty of those. This is about the practical daily-use tradeoffs that don’t show up in leaderboards.

Where Local LLMs Win

Privacy for sensitive content – Anything involving personal information, client work, or content I don’t want stored on a third-party server goes local. Llama 3.1 70B running on my server has no network connection to the outside world after the model is downloaded. Whatever I send it stays on my hardware.

Cost at volume – I run a lot of AI-assisted workflows: content drafting, summarization, code review, classification tasks. At the volume I use, cloud APIs would cost $100-150/month. My local server is already running; Ollama’s marginal cost is electricity, which adds maybe $5-10/month on my power bill.

No rate limits – Local inference has no API rate limits. I can run a batch job that processes 500 documents overnight without worrying about hitting a per-minute token limit or paying premium-tier pricing for throughput.

Latency for short tasks – For quick generation tasks where the model is already loaded in VRAM, local inference is faster than a round-trip to a cloud API. My RTX 3060 generates Llama 3.1 8B output at around 60 tokens/second. That’s snappier than most cloud APIs under load.

Where Cloud AI Wins

Quality ceiling – The honest truth is that GPT-4o and Claude Sonnet are meaningfully better than any model I can run locally right now. For tasks that require sophisticated reasoning, long-context comprehension, or nuanced judgment, the frontier models win. My 70B quantized Llama is impressive but it’s not GPT-4o.

Multimodal capability – Cloud APIs handle vision, audio transcription, and document understanding at a level local models can’t match yet. I use Claude for analyzing screenshots and diagrams because the local vision models I’ve tried are noticeably worse.

No setup or maintenance – Cloud APIs just work. No GPU driver issues, no VRAM constraints, no container restarts when a model update breaks something. If you just want to build something, cloud APIs are faster to start with.

Model variety without storage cost – Running locally, every model I want available takes up disk space and VRAM budget. On cloud APIs, switching models is one parameter change. I don’t have to choose what to keep loaded.

How I Actually Split the Work

My routing logic in practice:

Go local: Anything with personal or client data. Batch processing jobs. Quick classification or extraction tasks where 8B-quality is sufficient. Things that need to run without internet access.

Go cloud: Complex reasoning or writing tasks where I need the best output I can get. Vision/image understanding. Anything where quality matters more than cost. Long-context tasks that exceed what I can run efficiently locally.

About 70% of my AI usage by token count is local. About 70% of my AI usage by task importance is cloud. Those numbers make sense together – local handles the volume, cloud handles the critical stuff.

The Hardware Reality

The quality gap between local and cloud narrows as you put more hardware behind local inference. A consumer RTX 3060 running a 7B model is not in the same league as GPT-4o. Two A100s running Llama 3.1 405B would be a different conversation.

Most home users are working in the 8-70B model range with consumer GPUs. At that level, the honest assessment is: local is good enough for a lot of tasks and not good enough for the hardest tasks. Know which category your task falls into.

What’s Coming

The gap is closing. Models that required 80B parameters two years ago now achieve comparable quality at 8B. Quantization techniques are improving. Hardware efficiency is improving. The trend is clearly toward more local capability per dollar of hardware.

My prediction: in 18 months, a 30B model on a mid-range GPU will be good enough for 90% of daily tasks. The 10% that needs frontier capability will still go to the cloud, but the volume will keep shifting local.

If you want to start running local models, the LincStation N1 is a clean way to get an Unraid box running. Add a GPU if you want inference speed – the Noctua NF-A12x25 fan handles the thermal load from a GPU addition cleanly.

Next: Home Assistant in 2026 – what I actually use it for and what I’ve given up on.

Products mentioned in this post:

• LincStation N1 6-Bay NAS with Unraid License
• Noctua NF-A12x25 PWM 120mm Fan

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.

Unraid’s default Docker interface is fine for getting started. You click through a template, fill in a few fields, and you’ve got a running container. But once you have 20 containers, that approach starts to break down. Things are inconsistently configured, you can’t easily reproduce the setup on a new machine, and there’s no version history for your container configs.

I switched to Compose-based management about 18 months ago. Here’s the structure I landed on.

Why Compose on Unraid

Unraid 6.12 added native Docker Compose support through the compose manager. Before that, you had to use the community plugin. Now it’s built in – you can define stacks in YAML files and manage them from the UI or command line.

The main benefit is reproducibility. My entire container stack is defined in YAML files stored in a git repository. If the server dies and I rebuild from scratch, I clone the repo and bring everything back up with docker compose up -d. That’s not fully automated yet – I still have to restore data volumes – but the configuration layer is covered.

The other benefit is legibility. A Compose file makes the dependencies, environment variables, network configuration, and volume mappings explicit in one place. No more hunting through the Unraid UI trying to remember why a specific port was mapped a certain way.

My Directory Structure

Everything lives under /mnt/user/compose/ on the array. Each stack gets its own subdirectory:

/mnt/user/compose/
  media/
    docker-compose.yml     # Plex, Radarr, Sonarr, Prowlarr, Jellyfin
  nextcloud/
    docker-compose.yml
    .env                   # DB passwords, admin credentials
  ai/
    docker-compose.yml     # Ollama, Open WebUI
  monitoring/
    docker-compose.yml     # Uptime Kuma, Dozzle
  network/
    docker-compose.yml     # AdGuard Home, Nginx Proxy Manager
  security/
    docker-compose.yml     # Vaultwarden, Authentik (if I ever fix it)

The grouping is functional – services that talk to each other or share a network go in the same stack. Services that are independent get their own stack so I can restart them without touching unrelated containers.

Environment Variables and Secrets

Each stack directory has a .env file for environment-specific configuration. Passwords, API keys, and paths that differ between environments go there. The .env files are excluded from the git repo (obviously) and backed up separately.

The Compose file references variables from .env like this:

environment:
  - POSTGRES_PASSWORD=${NEXTCLOUD_DB_PASSWORD}
  - NEXTCLOUD_ADMIN_PASSWORD=${NEXTCLOUD_ADMIN_PASSWORD}

This means the Compose files themselves can be safely stored in version control without exposing credentials. When I need to check “what database password did I use for Nextcloud,” I look in the .env file in the secure backup, not in git history.

Networking: Named Networks Over Default Bridge

The default Docker bridge network works but it’s loose. Any container on the bridge can reach any other container on the bridge by IP. That’s fine for a home server where you trust everything, but it makes it harder to reason about which services actually need to talk to each other.

I define named networks in my Compose files and assign containers only to the networks they need. The media stack has a media_net network. Radarr and Sonarr are on it. Plex is on it. The download client is on it. Nothing outside that stack is on media_net.

Services that need external access – AdGuard Home, Nginx Proxy Manager – have a separate proxy_net network. Vaultwarden is on both proxy_net (for external access) and an isolated vault_net (for the internal database connection).

Volume Management

I use bind mounts rather than named Docker volumes. This means data lives at a predictable path on the host file system rather than in Docker’s internal volume store. For a home server this is usually the right call – you can find your files with a file manager, back them up with normal tools, and restore them without Docker-specific knowledge.

The appdata path for configuration files is /mnt/user/appdata/[service-name]. Media and large data files go on the array at /mnt/user/[category]/. The distinction matters for backup strategy – appdata is small and critical (back it up frequently), while media files are large and recoverable (back up less often or not at all if you can re-download).

The Stack I’m Most Proud Of

The monitoring stack is the one I’ve refined the most. Uptime Kuma polls every service endpoint. Dozzle aggregates container logs so I can see what’s happening without SSH-ing in every time. A Telegram webhook notifies me if any service goes down and stays down for more than 2 minutes.

This setup has caught three real problems in the last six months: a Nextcloud container that silently exited after an update, a DNS resolution failure that took down local hostname routing, and a Plex database lock that was blocking transcoding. In each case I had an alert within 5 minutes of the failure.

What I’d Tell Someone Starting Fresh

Start with Compose from day one. The Unraid template UI is a fine way to understand the options for a new service, but immediately translate it to a Compose file before you forget what you configured. The time you spend doing that pays back the first time you need to rebuild something.

Store your Compose files somewhere safe. A private git repo is ideal. A local backup is the minimum. These files represent hours of configuration work – treat them like code.

The LincStation N1 is a solid appliance if you want to get started with Unraid and Compose without building a server from scratch. It ships with an Unraid license and has enough headroom for a basic Docker stack.

Next post: local LLMs versus cloud AI, from someone who runs both every day.

Products mentioned in this post:

• LincStation N1 6-Bay NAS with Unraid License
• LincStation S1 6-Bay Desktop NAS

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.

The question I get most often when I tell people I run a homelab is: why bother? Google Drive is free. Spotify is ten bucks a month. Dropbox just works. Why spend a Saturday afternoon debugging a Nextcloud sync issue when you could be doing literally anything else?

It’s a fair question. Here’s my honest answer.

The Control Argument

When you use a cloud service, you’re renting access to your own data. The terms can change. The pricing can change. The service can shut down. Most of the time none of those things happen in ways that hurt you, but the possibility is always there.

I’ve had three services I relied on go away or change enough to be useless in the past five years. Google Photos changed its storage policy. Dropbox changed its free tier. Evernote went through multiple owners and quality cliff dives. Each time, I had to migrate data and rebuild a workflow I’d gotten used to.

Self-hosting isn’t immune to disruption – hardware fails, software stops being maintained. But the timeline is in my hands. I don’t wake up to an email saying I have 30 days to migrate.

The Privacy Argument

I’m not especially paranoid. I use Google Maps. I have a Gmail address. I’m not trying to disappear from the internet.

But there’s a category of data where I’d rather not have it indexed by someone else’s machine learning pipeline: family photos, financial records, health information, personal writing. For that category, local storage with encrypted backups is just cleaner. I know where it is and I know who can access it.

Immich for photos, Vaultwarden for passwords, Nextcloud for documents. Those three replacements cover most of the personal data I care about.

The Cost Argument

This one is context-dependent. For a single person with modest data needs, self-hosting is probably more expensive when you factor in hardware, electricity, and your time. Cloud services exist because they’re cheaper at small scale.

At family scale, the math changes. I’ve got six devices syncing photos. Immich is free; iCloud family storage at the size I’d need runs about $100/year. Vaultwarden is free; 1Password family is $60/year. Nextcloud is free; Google One at a comparable storage tier is $30/year. The breakeven on a few hundred dollars of hardware is under two years, and the hardware keeps running after that.

The honest caveat: I’m not counting my time. If you bill at $100/hour and spend 20 hours a year on homelab maintenance, the economics look very different.

The Learning Argument

This is the one I don’t hear talked about enough. Running a homelab teaches you things. I understand networking better because I had to set up VLANs. I understand containerization because I run Docker every day. I understand backup strategies because I’ve had drives fail.

That knowledge is worth something professionally. Every homelab project I’ve done in the last three years has shown up in some form in actual work – a conversation about infrastructure, a debugging session where I already knew the underlying concepts, a tool I could set up because I’d done something similar at home.

What I Don’t Self-Host (And Why)

Email. I’ve thought about it, gotten close, and stepped back every time. Email deliverability from a residential IP is a constant fight against spam filters. The cost of getting it wrong is real – a misdirected invoice or missed calendar invite is a real problem. I use Fastmail and consider it money well spent.

Video conferencing. Running a Jitsi instance sounds fun until you’re on a call with your kid’s teacher and the connection is bad because your upstream bandwidth is maxed. Some services are better as someone else’s problem.

AI APIs for heavy workloads. I run Ollama locally for most things, but when I need GPT-4-level capability at scale, I use the API. The cost of running sufficient hardware to match cloud AI performance is too high to justify for occasional use.

The Right Hardware to Start

You don’t need a full server. A LincStation N1 with an Unraid license included is a reasonable starting point if you want appliance-style simplicity. If you want more control and are comfortable with a bit more setup, build or buy a mini PC with a spare external drive and start with Nextcloud and Vaultwarden. That covers the two highest-value self-hosted services for most people.

Whatever you start with, get a UPS on it. An APC BX1500M is what I use. Unexpected shutdowns are how you lose data.

The important thing is starting small. One service, running well, is better than ten services that need constant attention.

Is It Worth It

For me, yes. I like having control over my infrastructure, I enjoy the learning aspect, and the cost math works out at household scale. I don’t think everyone should self-host everything, and I’m not interested in convincing people it’s morally better. It’s a tool. For some people and some use cases it’s the right tool.

Next post: how I organize my Docker container stack on Unraid with Compose files, including the structure that finally made sense after two years of iteration.

Products mentioned in this post:

• LincStation N1 6-Bay NAS with Unraid License
• APC BX1500M UPS 1500VA
• Noctua NF-A12x25 PWM Fan

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.

I run three WordPress sites. For a long time, managing them meant logging into each one, checking for updates, keeping an eye on uptime, handling comments, and occasionally catching something broken because a plugin update silently wrecked a template. It wasn’t a lot of work per session, but it was constant low-grade overhead.

I’ve mostly automated that now. Here’s how it works.

What the Agent Actually Does

The AI agent running on my Unraid server handles several categories of work across my three sites: elembemedia.com, bacallburns.com, and cobblecreeksoap.com.

Content drafting and scheduling – I describe a topic in a Telegram message. The agent writes a full post draft, generates a cover image, assigns SEO metadata, and pushes it to WordPress as a draft for my review. This post, for example, started as a two-sentence description I sent at 11pm.

Uptime monitoring – Uptime Kuma runs on the server and pings every endpoint every 60 seconds. If something goes down, the agent gets notified and can send me a summary of what’s broken and what it knows about why. I don’t get woken up for a 30-second blip, but I do hear about anything sustained.

Plugin and theme status – Via WP-CLI routed through the infrastructure agent, the AI can check what plugins are outdated across all three sites and give me a consolidated report. I still approve every update, but I don’t have to go hunting for what needs attention.

Basic content audits – The agent can pull a list of posts with broken links, identify drafts that have been sitting untouched for more than 90 days, and flag anything in the comments queue that looks like spam versus a real question I should respond to.

The Technical Stack

This is all built on top of OpenClaw, which is the agent framework I run locally. The core is Claude (Anthropic’s API) with tool access to the WordPress REST API, WP-CLI via SSH to the Docker containers, a monitoring stack, and an image generation pipeline.

The agent framework means I can send a natural-language message and it routes to the right tools. “Check what plugins need updates on elembemedia” becomes a WP-CLI call that returns a structured list. “Write a post about my Unraid setup” becomes a full content generation, image generation, and publish workflow.

The WordPress REST API does the heavy lifting for content operations. Most things you’d do in the admin interface are available via API: creating posts, updating metadata, uploading media, managing categories and tags. WP-CLI handles anything the REST API can’t, like checking plugin update status or running database queries.

What I Still Do Myself

I still review every post before it goes live. The agent drafts, I approve. This isn’t distrust of the output quality – it’s that I’m the one putting my name on these posts and I want to read them before they’re public. The review step also catches the occasional AI hallucination about a product feature that doesn’t exist or a claim I can’t actually back up.

Plugin updates still need my explicit go-ahead. The agent reports what needs updating and can run the update, but I want to know what changed before something updates on a production site.

Anything involving deleting content, changing site settings, or touching the database structure requires me to explicitly direct it. The agent has guardrails against taking irreversible actions without a clear instruction from me in the current conversation.

What Works Better Than Expected

The thing that surprised me most is how much cognitive overhead used to go into the “should I post something this week” decision. When writing and publishing is a 15-minute Telegram conversation instead of a 3-hour session, I post more. The frequency has roughly doubled since I set this up.

The SEO metadata generation is also better than I expected. The agent writes focus keywords, title tags, and meta descriptions that are clearly optimized for search without reading like they were written by a robot. I still tweak them occasionally but most of them are fine as-is.

What’s Harder Than It Looks

Getting the agent to understand the difference between “write a draft” and “publish this now” took more explicit guardrails than I expected. The defaults needed to be conservative – draft everything, require explicit confirmation for anything public-facing.

Image generation is still hit or miss. The local image generation pipeline produces technically decent images but they’re not as polished as stock photography. For a personal technical blog this is fine. For a brand-focused site I’d still probably buy photography.

The monitoring integration took iteration. Getting the right alert threshold – noisy enough to catch real problems, quiet enough to not wake me up for a 30-second AWS hiccup – required tuning over a few weeks.

Is This Overkill

For three small sites with a combined 200 posts and moderate traffic – probably yes. You could manage this with a shared hosting plan and a reminder to log in monthly.

But I run a homelab because I like having control over my infrastructure and learning how these systems work. The agent setup is the most interesting project I’ve done on the server in two years, and it’s changed how I think about what’s practical to do with a local AI.

Next: why I self-host everything I can, and the honest answer to whether it’s actually worth it.

Products mentioned in this post:

• LincStation N1 6-Bay NAS with Unraid License
• Noctua NF-A12x25 PWM 120mm Fan

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.

My home network used to be a flat mess. Everything on the same subnet, the printer talking to the server, IoT devices on the same SSID as my laptop, no local DNS – just a consumer router doing its best. It worked, right up until the day my smart TV started doing things I couldn’t explain and I realized I had no visibility into what was actually happening on my own network.

I rebuilt it properly about two years ago. Here’s how it’s laid out now.

The Hardware

The core is a GL.iNet GL-MT6000 (Flint 2) running OpenWrt. It’s a small, fanless router with 2.5GbE WAN and LAN ports and solid OpenWrt support out of the box. I chose it over the Ubiquiti UniFi path because I wanted full OpenWrt access without paying for a cloud management subscription or maintaining a separate controller device.

For switching I’m using a TP-Link TL-SG108E 8-port smart switch. It’s not a managed switch in the enterprise sense, but it supports 802.1Q VLANs which is all I need. The price-to-capability ratio is hard to beat.

Wireless APs are two TP-Link EAP670s running in standalone mode. No controller, no cloud account. They handle 802.11ax on both bands and support multiple SSIDs mapped to different VLANs.

The VLAN Layout

I run four VLANs:

VLAN 10 – Trusted – My computers, phones, and the Unraid server. Full internal access, DNS through AdGuard Home. This is where 90% of my traffic lives.

VLAN 20 – IoT – Smart TVs, Roku, game consoles, smart home devices. Can reach the internet but can’t initiate connections to VLAN 10. The Unraid server can reach IoT devices (for Home Assistant) but not vice versa. This one change alone fixed the “what is my TV doing” problem.

VLAN 30 – Guest – Isolated internet access, nothing else. I give this SSID to visitors without worrying about what they’re doing. Rate limited to 50Mbps so one guest can’t hog the connection.

VLAN 40 – Servers – DMZ-style isolation for anything exposed to the internet. Right now that’s just Nginx Proxy Manager. Strict firewall rules in both directions.

Local DNS with AdGuard Home

AdGuard Home runs on my Unraid server in Docker. It serves two purposes: ad blocking and local DNS resolution.

The DNS side is what changed how I use my home network. Every service gets a real hostname. Instead of remembering that Jellyfin is at 192.168.10.25:8096, it’s just jellyfin.home. Nextcloud is nextcloud.home. Uptime Kuma is kuma.home. The router points all clients to AdGuard Home for DNS, and AdGuard Home has a rewrite rule for each service pointing to the server’s IP.

The ad blocking is a bonus. I use the Hagezi Pro blocklist plus the Steven Black hosts file. The block rate on my network is usually around 18-22% of all DNS queries.

The Machine That Does Too Much

The Unraid server is nominally a storage and media server. But it also runs AdGuard Home, Nginx Proxy Manager, Vaultwarden, Nextcloud, Home Assistant, and Ollama. At some point it stopped being a NAS and became the infrastructure backbone of my house.

This is fine as long as you’re aware of it. The problem comes when the server needs maintenance. Taking it offline for a drive replacement or Unraid update means losing DNS, which means all my local hostnames stop working, which means half the smart home features stop working. I’ve learned to schedule maintenance carefully.

The right answer is redundancy – a second Pi-hole or AdGuard instance as a backup DNS server. I haven’t gotten there yet. It’s on the list.

Firewall Rules: What Actually Matters

The VLAN topology is only as good as the firewall rules enforcing it. OpenWrt makes this manageable with its zone-based firewall. Each VLAN is a zone, and you define traffic policies between zones.

The rules that matter most in my setup:

• IoT cannot initiate connections to Trusted (but Trusted can reach IoT)
• Guest is fully isolated from everything except WAN
• Servers VLAN can only receive connections from Trusted, never initiate them outbound to Trusted
• All inter-VLAN routing blocked by default; rules are explicit allowlists, not denylist

The last point is important. If you start from “allow all” and try to block things, you’ll miss something. Start from “block all” and explicitly allow what you need.

What I’d Change

I’d add a proper UPS on the networking gear. The server has a UPS, but the router and switches plug straight into the wall. A power blip that doesn’t kill the server still kills the network, which means the server loses connectivity. A small APC BX1500M on the networking stack would fix this.

I’d also set up a backup DNS instance earlier. Every time I reboot the server for maintenance I regret not having a fallback.

Next post: the AI agent that manages my WordPress sites while I sleep. That one gets into the automation side of all this infrastructure.

Products mentioned in this post:

• GL.iNet GL-MT6000 Flint 2 Router
• TP-Link TL-SG108E 8-Port Smart Switch
• APC BX1500M UPS 1500VA

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.

Running a local LLM sounds simple until you actually try it. Install Ollama, pull a model, start chatting – that’s the pitch. The reality involves GPU passthrough that almost worked, a model that ate all my RAM, and a late-night session trying to figure out why the container kept crashing. I’ve done that part so you have a head start.

Here’s how I actually got Ollama running on Unraid, including the mistakes I made along the way.

Why Bother Running Local LLMs at All

The honest answer is privacy and cost. I use AI tools constantly for work – writing, coding, summarizing. Most of that work is fine to send to a cloud API. But some of it isn’t. Client documents, medical information, anything I don’t want stored on someone else’s servers. A local model solves that without relying on a VPN policy I can’t fully verify.

There’s also the cost angle. Once your server is already running 24/7, the marginal cost of adding Ollama is basically zero. At heavy API usage rates, local inference pays for itself in GPU cost within six months.

The Hardware Reality Check

Ollama runs on CPU, but it runs well on GPU. The difference is dramatic – a 7B parameter model might take 30 seconds per response on CPU and under 2 seconds with a decent GPU. If you’re doing anything interactive, you need GPU acceleration.

I’m running an NVIDIA RTX 3060 with 12GB VRAM. That’s enough for most 7B and 13B parameter models at full precision, and 70B models at 4-bit quantization. The 12GB VRAM limit is real – if the model doesn’t fit, Ollama falls back to CPU for the overflow layers and performance tanks.

If you don’t have a discrete GPU, you can still run Ollama on CPU with enough RAM. 32GB gets you to 13B models comfortably. 64GB opens up 34B models. It’s slow but it works for non-interactive tasks like batch summarization.

Installing Ollama on Unraid

There are two good paths: the Community Applications plugin (search for “Ollama”) or a manual Docker template. I went manual because I wanted more control over GPU assignment.

The Docker template basics:

• Image: ollama/ollama:latest
• Port: 11434 (both host and container)
• Volume: /mnt/user/appdata/ollama:/root/.ollama
• Runtime: nvidia (for GPU passthrough)

That last line is where people get stuck.

GPU Passthrough: The Part Nobody Explains Well

For NVIDIA GPU passthrough in Unraid Docker containers, you need the NVIDIA driver plugin installed from Community Applications. It’s listed as “Nvidia-Driver” and it adds the necessary runtime support.

After installing the plugin, go to your Docker container settings and look for the “Extra Parameters” field. Add --runtime=nvidia. Then add two environment variables: NVIDIA_VISIBLE_DEVICES=all and NVIDIA_DRIVER_CAPABILITIES=all.

The thing that caught me: if your GPU is also assigned to a VM (I use mine for a Windows gaming VM), you can’t pass it to Docker and a VM at the same time. You have to pick one. I now stop the Windows VM when I want to use Ollama heavily, and stop the Ollama container when gaming. It’s not elegant, but it works for my usage pattern.

Setting Up Open WebUI

Ollama’s API is great, but you probably want a chat interface. Open WebUI is the best one I’ve found. It connects to Ollama, supports multiple models, has conversation history, and can do image analysis with vision models.

Deploy it as a second Docker container. The key config is setting the Ollama API URL to your server’s IP, not localhost – something like http://192.168.1.x:11434. If you use localhost it’ll try to connect from inside the container and fail.

Which Models Are Actually Worth Running

After testing most of the popular options, here’s what I actually use:

Llama 3.1 8B – My daily driver. Fast, capable, fits in 8GB VRAM comfortably. Good for writing assistance, code review, quick Q&A. Pull it with ollama pull llama3.1:8b.

Llama 3.1 70B Q4 – The heavy one. Takes about 40GB RAM/VRAM to run. I offload most layers to RAM and use the GPU for inference. Slower but noticeably smarter for complex tasks. Worth having for anything where quality matters.

Mistral 7B – Smaller than Llama 3.1 8B but surprisingly capable. I use it when I need something fast on CPU-only.

Nomic Embed Text – Not a chat model – an embedding model. Essential if you want to build a local RAG (retrieval augmented generation) setup. Small and fast.

The Mistakes I Made

I pulled too many models at once. Each model is 4-8GB. My appdata drive ran out of space and the container kept crashing with cryptic errors. Check your free space before pulling models, and set a dedicated path on a drive with room to grow.

I tried to run Ollama and Plex at the same time during a 4K transcode. The GPU can’t serve two masters. The transcode stuttered and the Ollama response was 3x slower. If you’re sharing a GPU between media and AI, set your Plex hardware transcoding to “when available” and accept the tradeoff.

I also didn’t set a context length limit early on. Some models default to huge context windows that eat RAM even when you don’t need them. In your Open WebUI model settings, set a max context of 4096 unless you specifically need more.

Is It Worth It

For me, yes. The ability to run a capable LLM privately, on hardware I own, without a per-token cost is useful. The setup friction is real but it’s a one-time cost. Once it’s running it just works.

If you don’t have a server already running, the calculus is different. A cloud API is cheaper than buying hardware you’ll only use for LLMs. But if you’ve already got an Unraid box, Ollama is one of the best things you can add to it.

Next up: my full home network setup – VLANs, local DNS, and the one machine that does too much.

Products mentioned in this post:

• LincStation N1 6-Bay NAS with Unraid License
• Noctua NF-A12x25 PWM 120mm Fan

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.

Most homelab posts are aspirational. They show you a pristine rack, perfectly labeled cables, and a services dashboard with everything green. What they don’t show you is the three containers that are broken and just sitting there, the drive that spun up three months ago and hasn’t been checked since, and the app you installed for a project you abandoned.

This is the honest version. Here’s exactly what’s running on my Unraid server right now, what I actually use, what’s just squatting RAM, and what I wish I’d done differently when I set it up.

The Hardware

My server is a custom build in a Fractal Design Node 804. It’s got an older Intel Xeon processor, 64GB of ECC RAM, and a mix of drive sizes that would give a proper storage admin a headache. Six data drives ranging from 4TB to 10TB, a 2TB parity drive, and two SSDs in the cache pool.

If I were building from scratch today I’d honestly consider something like the LincStation N1, which ships with an Unraid license already baked in. It’s not a powerhouse, but if your workload is mostly storage and light Docker containers it’s a cleaner starting point than my Frankenserver.

Cooling matters more than people think. I’ve had drives run 10 degrees cooler just from swapping case fans. The Noctua NF-A12x25 is overkill for noise-sensitive setups but it’s the best 120mm fan you can buy and it runs practically silent under normal load.

The Docker Stack

Unraid’s Docker integration is what made me switch from a bare Debian box three years ago. No YAML files to maintain, no compose headaches, just a UI that works. Here’s what’s actually running:

Plex – The anchor tenant. Everything else exists partly to support Plex or to not interfere with Plex. I’ve got about 8TB of media organized by Radarr and Sonarr, and the server transcodes 4K without breaking a sweat.

Radarr + Sonarr + Prowlarr – The automation trio. Once you set these up and forget about them, you realize how much time you used to waste manually managing media. Prowlarr replaced Jackett for me about a year ago and I haven’t looked back.

Jellyfin – Running alongside Plex as a backup and for family members who don’t want to deal with Plex accounts. It’s gotten good. The hardware transcoding support is solid.

Nextcloud – My Google Drive replacement. I’ve got about 200GB synced across three devices. The iOS client has improved dramatically. File sync still occasionally needs a kick to restart, which is annoying, but it’s not something I’d trade back.

Vaultwarden – Self-hosted Bitwarden server. This one is load-bearing for my whole family. It syncs to all our devices, and I export a backup every week automatically. If this goes down, everyone loses their passwords. So it gets the most attention.

Immich – Photo management, Google Photos replacement. This one has leveled up fast. Face recognition, location mapping, albums, sharing – it does everything. My wife was skeptical until she saw the mobile app. Now she uses it daily.

Home Assistant – Running in a dedicated VM, not Docker. More on this in a separate post, but it needs the full VM treatment for Zigbee USB passthrough to work reliably.

Ollama + Open WebUI – Local LLM server. This is the new addition that’s gotten the most tinkering. I run Llama 3.1 8B for quick tasks and Llama 3.1 70B quantized for anything that needs real reasoning. It uses a lot of RAM.

Uptime Kuma – Monitoring dashboard for all my services. Gets a ping every 60 seconds, sends me a Telegram message if anything goes down. Simple and reliable.

Nginx Proxy Manager – Handles all my reverse proxy routing and SSL certificates. Every service gets a clean subdomain. More on this in a dedicated post.

The VMs

Unraid’s VM manager runs on KVM under the hood. I’ve got three VMs currently:

Home Assistant OS – The whole thing runs in a VM with USB passthrough for my Zigbee stick. This is the right way to run HA on Unraid. Don’t put it in Docker unless you enjoy pain.

Windows 11 – For gaming sessions when I want to play something on the living room TV over Moonlight/Sunshine streaming. It sits idle most of the time but fires up fast when I need it.

Ubuntu dev box – A clean Linux environment for projects I don’t want running on my Mac. Docker experiments, random builds, things I might blow up. This VM gets nuked and rebuilt every few months.

What’s Broken and Just Sitting There

Authentik – I tried to set up proper SSO across all my services. Got it halfway working, then hit a wall with Nextcloud integration, and it’s been sitting broken for four months. Every time I open the UI I feel vaguely guilty.

Portainer – Installed it to manage Docker on a second machine. The second machine went away. Portainer is still there doing nothing.

A Minecraft server I set up for my kid in 2024 and haven’t touched since. It’s using 2GB of RAM.

What I’d Do Differently

I’d start with a UPS from day one. I added one later and it caught two power blips in the first month. An unexpected shutdown mid-parity check is a bad day. Get a good unit – I use an APC BX1500M and it’s been rock solid.

I’d also plan my cache drive setup before I needed to expand it. I added the second SSD to the cache pool nine months in, which meant a parity rebuild and a couple hours of anxiety. Plan for two cache drives from the start.

The other thing – document everything. Which port maps to which service, why you chose that configuration, what that weird env variable does. I have a plaintext file for this now. It saved me twice last month when I was troubleshooting something I’d set up a year ago.

What’s Next

The next post goes deep on Ollama: how I set it up on Unraid, what I learned the hard way about GPU passthrough, and which models are actually worth running locally versus just using the API. That one’s coming Thursday.

Products mentioned in this post:

• LincStation N1 6-Bay NAS with Unraid License
• Noctua NF-A12x25 PWM 120mm Fan
• APC BX1500M UPS 1500VA

Affiliate disclosure: Some links in this post are Amazon affiliate links. If you buy through them, I get a small commission at no cost to you. It helps keep the lights on here.