Monitoring
Health
GET /healthz on the public port answers {"status":"ok"} with 200 once the HTTP server runs. It needs no login and checks nothing beyond that, so use it for liveness and container health checks.
Admin readiness
Admins get an operational snapshot on the readiness page of the admin area in the web app, or from GET /api/v1/admin/readiness:
| Field | Meaning |
|---|---|
appVersion, commit | The build. commit is left out when the build has none. |
databaseDriver | postgres or sqlite. |
migrationVersion, migrationDirty | The applied migration and whether the last one failed halfway. |
backup | The backup marker state: current, stale, not_recorded, invalid or unavailable, with the completion time. See Deployment. |
upstreams | Health of every model upstream. See Models and upstreams. |
The snapshot holds no secrets. Only a failure to read the migration status fails the request; a bad backup marker shows up as a state.
Prometheus metrics
/metrics is served on a separate listener, CHATZ_METRICS_LISTENADDRESS (:9091), which neither example publishes. Scrape it from inside the Docker network, or publish it to a private address.
| Metric | Type | Labels |
|---|---|---|
llm_requests_total | counter | service, stage, model, outcome |
llm_request_duration_seconds | histogram | service, stage, model, outcome |
llm_tokens_total | counter | service, stage, model, kind |
outcomeisokorerror.kindisinput,outputorcache_read.serviceischatzandstageischat.modelfalls back tounknown.- The duration buckets are 0.25, 0.5, 1, 2, 5, 10, 20, 30, 60 and 120 seconds.
- The Go runtime and process collectors are registered too.
Every model call also writes a row to the llm_usage table with its token counts and duration.
pprof
Go’s profiler is served under /debug/pprof/ on CHATZ_PROFILING_LISTENADDRESS (:6060). It exposes command lines and memory, so never publish it. Set the variable to an empty value to turn the listener off, and do the same for metrics if you do not scrape them. A listener that fails to bind logs a warning and leaves the API running.
Logs
chatz logs to standard output in the format and level set by LOG_FORMAT and LOG_LEVEL. Every line carries service, version and, when known, commit. Lines logged while handling a request carry its request_id, the same value as the X-Request-ID response header.
The app also writes chatz.log in its working directory, /app/chatz.log in the image:
- It is cleared on every start, so it holds the current run only.
- It is always JSON at debug level, whatever
LOG_LEVELsays. - Bind-mount a host file there to read it from the host, as the examples do. Without a writable file chatz logs a warning and runs without it.
Both outputs pass through redaction: attributes named like password, token, secret, api_key, authorization or cookie become [REDACTED], and secret-shaped key=value pairs and bearer tokens in text are masked. Ordinary chat text is not redacted, and debug logs include the full prompt of each model round. Keep debug logs local.
Long waits log a heartbeat, so a model stream or tool call that is still running is visible in the log instead of looking like a hang.