Logs and data pipelines

Why NDJSON works well for logs and streams

One JSON object per line supports append, chunking, and per-record processing.

The problem it solves

A JSON array often requires the complete document before parsing. NDJSON puts one record on each line, which fits append-only logs, pipelines, and retrying individual failures.

Boundary rules

A non-empty line must be a complete JSON value. Truncated records, multiline strings, or log prefixes make parsing fail. Production pipelines should retain line numbers and failure reasons.

Validate the conversion

Turning a large NDJSON stream into an array loads everything into memory. Check record counts, field consistency, nulls, and missing fields before handing data to a downstream system.

Related tools

Sources

Last updated: 2026-08-23