stt/infra-observability
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Show parsed log tables instead of raw lines

Browse Source 
Raw lines are close to unreadable when the payload is JSON, and the Kong audit
records are the worst case: a wall of braces where the interesting fields are
status, route and latency.

Each dashboard now leads with a table carrying one column per parsed field,
sortable and filterable, with the unparsed record as the last column and the
cell inspector enabled on every cell. The original logs panel moves into a
collapsed row underneath rather than disappearing: a table cannot render ANSI
colour, expand a record inline, tail live, or turn a trace id into a link to
Tempo.

Parsing is per-format since the five sources agree on nothing — nested JSON for
Kong, JSON for Keycloak, NCSA access lines for the Django API, ANSI-coloured
text for LibreChat, logfmt for the stack itself.

The Kong and stack parsers name their fields explicitly rather than extracting
everything: a bare `| json` on a Kong record yields around seventy columns, and
a bare `| logfmt` explodes Loki's own query-metrics lines into dozens. Columns
are selected by include-pattern rather than by excluding names, because the
stream labels that ride along are not a fixed set.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Stéphane Tailland
2026-08-25 18:11:53 +02:00
parent 0ba38bc8f7
commit 7a8a476ef0
6 changed files with 1295 additions and 88 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+31
View File
@@ -206,6 +206,37 @@ Each application dashboard carries the same four sections: an overview of RED
indicators, HTTP traffic, a runtime section specific to the stack in use, and a
**Logs** row, followed by container resources.
### Parsed log tables
Every dashboard shows its logs twice: a **table** with one column per parsed
field, and, in a collapsed row underneath, the original **logs panel**.
The table is what makes a JSON record readable — status, method, route, latency
as sortable, filterable columns rather than a wall of braces. Its last column
holds the unparsed record, and every cell has the inspector enabled, so the raw
line is one click away.
The logs panel stays because a table cannot do what it does: render ANSI colour,
expand a record inline, tail live, and turn a `trace_id` into a link to Tempo.
It is collapsed so it costs nothing until needed.
Parsing is per-format, because these five sources agree on nothing:
| Source | Format | Parser |
|---|---|---|
| Kong audit | nested JSON | `\| json` with explicit field paths |
| Keycloak | JSON | `\| json` |
| classic-models-api | NCSA access lines | `\| pattern` |
| LibreChat | ANSI-coloured text | `line_format` to strip ANSI, then `\| pattern` |
| Observability stack | logfmt | `\| logfmt` with named fields |
Two choices worth knowing. The Kong and stack parsers name their fields
explicitly rather than extracting everything: a bare `\| json` on a Kong record
yields about seventy columns, and a bare `\| logfmt` explodes Loki's own
query-metrics lines. And `\| pattern` keeps lines it cannot match, with empty
columns, rather than dropping them — so the Django table never silently hides
an application log among the access lines.
The Logs row leans on Loki's `detected_level`, which is inferred at ingestion
time, so a single query works across all three log formats — Keycloak emits JSON,
LibreChat colour-coded text, and the Django API plain gunicorn access lines. No