driftpipev0.9.4docs for Sign in

Command reference

NAME

reference - the whole command surface of driftpipe 0.9.4 on one page

This page folds the manual into a single reference with an anchor on every command, flag, environment variable and file key. The usage and config pages carry the same material with more prose; this one is for looking something up fast.

SYNOPSIS #

driftpipe check    [--schema PATH] [--input PATH] [--sample N] [--max-depth N]
                   [--fail-on MODE] [--ignore PATH]... [--format MODE]
                   [--config PATH] [--timeout DUR] [--print-config]
                   [--quiet] [--no-color]
driftpipe snapshot [--schema PATH] [--input PATH] [--sample N] [--max-depth N]
                   [--force] [--note TEXT]
driftpipe explain  [--schema PATH] [--path PATH] [--format MODE] [--no-color]
driftpipe version  [--key]

COMMANDS #

CommandReadsWritesExit 3
checkstream + lockreport, stdoutyes
snapshotstreamlock fileno
explainlockreport, stdoutno
versionnothingone lineno

No command runs by default. Bare driftpipe prints the synopsis and exits 1, because the two plausible defaults differ by whether the lock file survives.

↑ index

OPTIONS #

--schema PATH #
check / snapshot / explain. Lock file. Default schema.lock. A missing, truncated or future-format lock exits 4.
--input PATH #
check / snapshot. Record stream. Default - (stdin). Gzip is recognised from the first two bytes.
--sample N #
check / snapshot. Stop after N records. Default 5000; 0 reads everything. The first N, not a random N.
--max-depth N #
check / snapshot. Descent limit into nested objects. Default 8. Deeper subtrees fold into one depth_exceeded.
--fail-on MODE #
check. Exit-3 threshold: breaking (default) or any. It replaces the rules map wholesale.
--ignore PATH #
check. Path pattern to drop from the comparison. Repeatable. Matched against the whole path.
--format MODE #
check / explain. text (default), json, or github annotation lines.
--config PATH #
check. Config file. Default .driftpipe.yml when present. --config none loads nothing.
--timeout DUR #
check. Wall-clock limit, default 30s. Go duration strings. On expiry the status is 2 and the partial report is dropped.
--print-config #
check. Resolve every setting, print it with its source, exit 0 without reading the stream. Since 0.9.0.
--quiet #
check. Drop the header and the summary line; differences and the exit status are unchanged.
--no-color #
check / explain. Never emit escape sequences. Implied off a terminal or when NO_COLOR is set.
--force #
snapshot. Overwrite the lock in place. Nothing is backed up; version control is.
--note TEXT #
snapshot. Free-text note stored in the lock and printed by explain.
--path PATH #
explain. Print only the subtree under this path. Without it the whole tree prints.

↑ index

PATH SYNTAX #

One syntax serves --ignore, --path and the config ignore list.

PatternMatches
order.totalthat exact path
order.items[]the merged element shape of an array
order.items[].skua field inside that element shape
meta.*one segment under meta, no deeper
*.trace_idtrace_id under any single top key
debug.**debug and everything beneath it

INPUT FORMATS #

  • NDJSON - one object to a line.
  • A single JSON array of objects, streamed not buffered.
  • Concatenated objects with no separator.
  • Any of the three wrapped in gzip.

Nothing else: no Parquet, Avro, CSV or XML. The reasoning sits under limitations.

ENVIRONMENT #

DRIFTPIPE_CONFIG
Alternate config path. A --config flag beats it.
NO_COLOR
Any value, including empty, disables colour.
TZ
Affects only the lock timestamp, which is stored in UTC regardless.

EXIT STATUS #

CodeMeaning
0clean, or only differences below the threshold
1usage error: unknown flag, bad value, missing argument
2input error: missing or unreadable stream, bad JSON, timeout
3drift at or above the threshold
4lock unreadable, or written by a newer format

2 and 3 are kept apart on purpose: a vanished input must never read as a healthy schema. Try the rules live on the try page.

↑ index

CLASSES #

SymClassConditionDefault
+new_fieldspath observed, absent from the lockwarn
-removed_fieldspath in the lock, not observedfail
~type_wideningint in lock, float observedwarn
!type_narrowingany other replacement or unionfail
!nullabilitynull joins a union that lacked itfail
~depth_exceededa subtree was not expandedwarn

One path yields at most one difference, and nullability outranks widening when both apply. Full reasoning is on internals.

RULES MAP #

Each class resolves to one severity.

SeverityPrintedIn summaryExit 3
allownonono
warnyesyesno
failyesyesyes

--fail-on any sets all six to fail; --fail-on breaking sets removed, narrowing and nullability to fail and the rest to warn, which is also the built-in default.

CONFIG KEYS #

KeyFlag equivalentDefault
version-1 (config format, not tool)
input--input-
schema--schemaschema.lock
sample--sample5000
max_depth--max-depth8
rules.*--fail-onbreaking mapping
ignore--ignoreempty
output.format--formattext
output.color--no-colorauto
output.quiet--quietfalse

An unknown key is a usage error, not a warning. Precedence runs flag, then environment, then file, then default, resolved per setting. The annotated file lives on config.

LOCK FILE #

JSON with sorted keys and a trailing newline, meant to be committed. Current format is 3; a type is one of null bool int float string object array or a sorted |-union.

schema.lock
{
  "driftpipe": 3,
  "created": "2026-09-30T08:14:02Z",
  "note": "orders baseline, 2026-09-29",
  "records": 41160,
  "sample": 0,
  "max_depth": 8,
  "paths": { "order.id": "string", "order.total": "int" }
}
FormatSinceRead by 0.9.4
30.9.0yes, current
20.8.0yes
10.1.0no, exit 4

↑ index