VPC Flow Logs and Athena Query Pack

VPC Flow Logs and Athena Query Pack

VPC flow logs are your network audit trail, but they are useless sitting in S3 without a way to query them. This package sets up the full pipeline — flow logs to S3 with Hive partitioning, an Athena table, a Glue crawler for automatic partition discovery, and 10 ready-to-run security queries.

Infrastructure Deployed

• VPC Flow Logs — enabled on all VPCs via Lambda custom resource, publishing to S3 in Parquet format with Hive partitioning (year/month/day/hour) for efficient querying
• S3 Bucket — dedicated flow logs bucket with lifecycle rules (90-day intelligent tiering, 365-day expiry) and server-side encryption
• Glue Database + Crawler — crawler runs hourly to register new partitions, keeping the Athena table current without manual partition maintenance
• Athena Workgroup — dedicated workgroup with per-query data scanned limits to prevent runaway costs

Included Security Queries

1. Top talkers by bytes transferred
2. Rejected traffic by destination port (firewall gap analysis)
3. Port scan detection (source IPs hitting 20+ destination ports)
4. Unusual protocol traffic (non-TCP/UDP/ICMP)
5. Inter-AZ data transfer analysis
6. SSH/RDP brute force attempts
7. Traffic to/from known bad IP ranges
8. Outbound traffic on non-standard ports
9. Short-lived connections (potential C2 beaconing)
10. Daily traffic volume anomalies

What You Get

• vpc-flow-logs.yaml — CloudFormation template
• queries/ — 10 named Athena queries as .sql files
• saved-queries-import.sh — registers queries in the Athena console
• README.md — cost estimates, query usage guide, and adding custom queries