Creating Your First Investigation

Walk through a practical observability workflow from alert to action

This tutorial shows how to investigate a latency spike and apply a safe mitigation.

Scenario

You notice degraded response times for one production deployment in the past hour.

Step 1: Confirm the Spike

1. Open Observability → Metrics.
2. Set time range to Last 1 hour.
3. Pivot by Deployment.
4. Confirm p95/p99 latency increase.

Step 2: Isolate Affected Requests

1. Go to Requests.
2. Filter to the impacted deployment and same time range.
3. Sort by response time descending.
4. Open the slowest requests.

Step 3: Inspect Request Detail

Check these fields in detail view:

• Status and error information.
• Input/output token counts.
• Gateway metadata (IP, method, path, user agent, location).
• Raw request/response payloads.

Step 4: Correlate with Rules

1. Open Rules tab.
2. Filter active rules by relevant type (IP, country, user-agent, rate).
3. Verify whether blocked traffic patterns match expectations.

Step 5: Apply and Verify

1. Edit or add a rule only if data supports mitigation.
2. Re-check Metrics and Requests for the next 15–30 minutes.
3. Document before/after KPIs in your incident notes.

Outcome Checklist

Anomaly confirmed with KPI evidence.

Request-level root-cause clues captured.

Rule change validated against live traffic.