Ollama + Open WebUI Stack

Your Own Private ChatGPT

44% of enterprises cite data privacy as the top barrier to using LLMs. The answer is running the model yourself. Ollama is the Docker for LLMs — it handles quantization, model management, and exposes an OpenAI-compatible API. Open WebUI gives you a polished chat interface. Getting both talking through HTTPS with persistent storage and model pre-loading takes longer than it should. This stack handles it.

What Is Included

• docker-compose.yml — Ollama + Open WebUI + Nginx in one file, GPU and CPU variants included
• GPU mode — NVIDIA CUDA passthrough pre-configured via Docker runtime flags; flip one env var to enable
• CPU mode — works on any server, no GPU required; runs 7B models comfortably on 8 GB RAM
• Nginx reverse proxy — HTTPS termination, websocket support for streaming responses, rate limiting
• model-pull.sh — pulls your chosen models on first start (defaults to llama3.2 and nomic-embed-text)
• .env.example — GPU toggle, model list, domain, auth settings, volume paths
• README.md — hardware sizing guide, model recommendations by use case, API usage examples

How It Works

Set your domain in .env, choose GPU or CPU mode, run docker compose up -d. The model pull script runs automatically and downloads the default models. Open WebUI is available at your domain in a few minutes (plus model download time).

What You Get Immediately

• A working private chat UI at your domain over HTTPS
• OpenAI-compatible API endpoint for connecting other tools (n8n, VS Code, etc.)
• Persistent model storage — models survive container restarts
• Multi-user support via Open WebUI built-in auth
• Hardware sizing guide: which models work on 8 GB, 16 GB, and 24 GB VRAM

Requirements

Linux server with Docker installed. 8 GB RAM minimum for CPU mode. For GPU mode: NVIDIA GPU with CUDA support and the NVIDIA Container Toolkit installed. A domain name for HTTPS (or use HTTP locally). Port 80 and 443 open.