AI Log Triage
Error Detection to First Response in Seconds
Pipeline at a glance
Situation
An engineering team was spending hours investigating error logs that turned out to be expected behavior — external API timeouts, rate limits, DNS blips. Real issues got buried under noise, and the on-call engineer couldn't tell which ERRORs needed attention without digging into each one manually.
Task
Build an automated pipeline that triages ERROR logs in real time, classifies them against known patterns, notifies the team, and for critical errors — automatically creates a GitHub Issue and triggers an AI-powered source code investigation.
Actions
Log Ingestion & Filtering
Railway log drain sends all logs to an n8n webhook. Only ERROR and FATAL level entries pass through.
- Webhook receives Railway log drain payload
- Code node parses timestamp, service name, message, deployment ID
- Non-error logs are dropped before AI analysis
AI-Powered Triage
Gemini AI compares each error against 9 known patterns and classifies severity.
- Known patterns maintained in a markdown file — easy to update
- Three severity levels: Red (needs action), Yellow (monitor), Green (working as designed)
- Structured JSON output with cause, recommended action, and evidence
- If AI response parsing fails, defaults to Red — never silently drops an error
Notification & Escalation
All severities go to Discord. Red alerts automatically escalate to GitHub with AI code investigation.
- Discord: color-coded embed with summary, cause, action, evidence
- Red alerts: GitHub Issue auto-created with structured error context
- Duplicate detection against existing open issues prevents noise
AI Code Investigation
GitHub Actions triggers Claude Code to investigate the source code when a Red issue is created.
- Claude reads the error context from the issue body
- Investigates relevant source files in the repository
- Posts findings as a comment on the issue
- Engineer gets a head start before they even open the codebase
Results
- ERROR logs triaged in seconds instead of hours of manual investigation
- "Investigated for a day, turned out to be expected behavior" incidents eliminated
- Red alerts come with AI-generated code investigation — engineer starts with context, not from scratch
- Adding a new known pattern is a one-line edit to a markdown file