Skip to content
Théophile Diot.
fren
Navigation menu
All work

AI CLI Observability

Self-hosted observability / 2026-present

One local OpenTelemetry stack for comparing usage, latency, failures, tool activity, traces, and estimated cost across three AI coding command-line interfaces (CLIs).

Role
Author and operator
Contribution
Built the OpenTelemetry Protocol (OTLP) path, storage routing, and provisioned dashboard.
Evidence type
Linked public source
3AI CLI integrationsCodex, Claude Code, Gemini CLI
3Telemetry signalsLogs, metrics, traces
1Compose stackProvisioned dashboard and datasources
01

The problem

Each AI coding CLI exposes different telemetry, while the useful questions cross tool boundaries: where tokens go, which requests fail, what tools run, and how much equivalent API usage might cost.

Three clients, one self-hosted telemetry path
The default deployment binds Grafana and OTLP to loopback. Codex and Gemini costs remain estimates.
02

Decisions

  1. Accept authenticated OTLP/HTTP from Codex, Claude Code, and Gemini CLI through one collector.

    CostWhen one CLI changes its telemetry, the shared collector configuration is what has to absorb it.

  2. Route logs, metrics, and traces to separate Victoria backends, then provision them in one Grafana view.

    CostThree storage services to run and upgrade behind a single dashboard.

  3. Estimate Codex and Gemini cost from versioned model pricing while preserving Claude Code native cost metrics.

    CostTwo kinds of number share one view, one estimated and one reported, and the difference has to be labelled everywhere it appears.

  4. Keep the default deployment on loopback and verify authentication, ingestion, dashboards, and datasources with synthetic smoke telemetry.

    CostAnyone who wants remote access has to build it themselves, deliberately.

03

Outcome

The repository turns three built-in telemetry surfaces into one inspectable, self-hosted view without requiring another observability SaaS.

04

Evidence and scope

AI CLI Observability repositoryEnd-to-end smoke test

Looking for a collaborator?

Share the work and its constraints. I reply by email.

Send a message