Observability Concepts

Understand tabs, entities, and investigation signals

Workspace Model

Observability is organized around three tabs:

• Metrics: aggregate trends and KPIs.
• Requests: request-level evidence.
• Rules: active governance controls.

Key Entities

Entity	Description	Typical Questions
Inference Request	One end-user or service call	Why did this request fail or slow down?
Deployment	Runtime endpoint for a model	Which deployment is degrading?
Model	Inference model variant/provider	Is this model causing latency increase?
Rule	Traffic control policy	Are blocks intentional and effective?

Signal Types

• Volume: Request rate and traffic shape.
• Performance: Latency, TTFT, processing time.
• Quality: Success/failure rates and status codes.
• Cost/Usage: Input/output token totals and trends.
• Security/Governance: Block events and active rules.

Investigation Loop

1. Detect anomaly in Metrics.
2. Narrow Requests with time range and filters.
3. Inspect request detail for metadata, payload, and IDs.
4. Correlate with Rules and tune policies if needed.

Best Practices

Start broad (24h) and then narrow to the exact incident window.

Use deployment-level pivots before deep diving into single requests.

Track both latency and success rate; one without the other is incomplete.

Keep inactive rules for audit history instead of deleting by default.