ELK Logging Stack

ELK Logging Stack

A production-tuned Elastic Stack deployment with memory limits that won't crash your host, a pre-configured Index Lifecycle Management (ILM) policy to keep disk usage under control, and a Filebeat config ready to ship logs from any service.

What's Included

• Elasticsearch 8 — Single-node cluster with security disabled for self-hosted use; persistent volume for index storage
• Logstash — Pre-configured pipeline: Beats input → filter with grok patterns → Elasticsearch output
• Kibana 8 — Matched version; bootstraps with the Elasticsearch connection on first start
• Filebeat — Sidecar log shipper configured to tail Docker container logs and forward to Logstash

Production-Ready Configuration

• JVM heap sizes explicitly set for both Elasticsearch and Logstash — prevents runaway memory on shared hosts
• ILM policy pre-loaded: hot phase 7 days, warm phase 30 days, delete phase 60 days
• Index template applied automatically on first start — correct mappings for log timestamps and log levels
• Health checks on Elasticsearch — Logstash and Kibana wait until the cluster is green before connecting
• Filebeat auto-discovers running Docker containers via the Docker socket
• All passwords and API keys managed via .env

What You Get

• docker-compose.yml — fully configured four-service stack with memory limits
• logstash/pipeline/logstash.conf — annotated input/filter/output pipeline
• logstash/config/jvm.options — tuned heap settings
• kibana/kibana.yml — pre-wired to Elasticsearch
• filebeat/filebeat.yml — Docker autodiscovery configuration
• scripts/init-ilm.sh — ILM policy and index template bootstrap script
• .env.example with every variable documented
• README.md — setup guide, memory tuning notes, ILM customization, and Kibana first-run steps

Service URLs

Requirements

Docker Engine with the Compose plugin on Linux. Set vm.max_map_count=262144 on the host (included in README). 4 GB RAM minimum; 8 GB recommended.