Skip to content
Théophile Diot
fren
Navigation menu
fren
All work

2026-presentSelf-hosted observabilityAuthor and operator

AI CLI Observability

The problem

Each AI coding CLI exposes different telemetry, while the useful questions cross tool boundaries: where tokens go, which requests fail, what tools run, and how much equivalent API usage might cost.

Decisions

AI CLI ObservabilityDecision 1 of 4

Decision
Accept authenticated OTLP/HTTP from Codex, Claude Code, and Gemini CLI through one collector.
Consequence
When one CLI changes its telemetry, the shared collector configuration is what has to absorb it.

AI CLI ObservabilityDecision 2 of 4

Decision
Route logs, metrics, and traces to separate Victoria backends, then provision them in one Grafana view.
Consequence
Three storage services to run and upgrade behind a single dashboard.

AI CLI ObservabilityDecision 3 of 4

Decision
Estimate Codex and Gemini cost from versioned model pricing while preserving Claude Code native cost metrics.
Consequence
Two kinds of number share one view, one estimated and one reported, and the difference has to be labelled everywhere it appears.

AI CLI ObservabilityDecision 4 of 4

Decision
Keep the default deployment on loopback and verify authentication, ingestion, dashboards, and datasources with synthetic smoke telemetry.
Consequence
Anyone who wants remote access has to build it themselves, deliberately.

Outcome

The repository turns three built-in telemetry surfaces into one inspectable, self-hosted view without requiring another observability SaaS.

Evidence and scope

This public repository is a sanitized single-host form of a real setup. It contains no production telemetry, credentials, private endpoints, or identity-provider configuration; cost values are estimates, not invoices.