Schema inference
schema infer / infer
packata schema infer <SOURCE> --asset <ID> [OPTIONS]
packata infer <SOURCE> --asset <ID> [OPTIONS]
The infer entry point is an alias for schema inference, with the same options and behavior. Supply an ODCS YAML or Data Package v2 JSON/YAML descriptor. Packata detects its standard from the contents unless you set --input-format, and accepts extensionless descriptors. Descriptors can be local or served over HTTP/HTTPS, directly or inside ZIP, TAR, TAR.GZ or TAR.BR archives. Archives use the shared descriptor-selection rules.
Select one exact asset with exactly one endpoint binding. For CSV inference, that binding must provide a single CSV, TSV or PSV path. For a database, the shared service discovers the selected physical object’s metadata and reports diagnostics when discovery is partial.
Excel (xls/xlsx) and Parquet (parquet/pqt) bindings also use one file path. Parquet extracts declared footer metadata without sampling rows; Excel profiles one rectangular worksheet and reports sampling uncertainty. Both standards support local, HTTP/HTTPS and archive-relative files. Relative paths resolve against the descriptor’s local directory, remote URL directory, or archive member directory. Absolute HTTP/HTTPS data URLs are independent of that base. Archive traversal outside the root is rejected. Explicit format takes precedence over a recognized media type and path extension, including URLs with query parameters or no extension. Unsupported formats remain unsupported.
Excel and Parquet require seekable streams. The CLI stages non-seekable HTTP/ZIP inputs into temporary files, deleted on success, failure or cancellation. Staging is limited to 256 MiB by default, configurable per invocation with --staging-maximum-bytes. Excel additionally enforces its workbook limit (64 MiB by default, or --maximum-bytes); the smaller limit applies while staging, with a distinct workbook-limit diagnostic. Limits are checked while copying, even without a trustworthy Content-Length. HTTP descriptor/archive transfers and non-seekable CSV reads retain a separate fixed 256 MiB limit. TAR materialization retains its separate storage limits. Parquet still reads only footer metadata after staging; staging may transfer the entire file, without materializing rows. Existing footer/field/depth and sampling limits apply. Seekable inputs pass through without staging or a complete-file staging cap. See the shared staging policy for stream ownership, temporary disk requirements and cancellation limitations.
HTTP uses the existing GET transport and .NET redirect behavior. Public and pre-signed HTTPS object URLs work without native cloud-provider configuration. No CLI HTTP header, interactive login or credential configuration is added; do not put credentials in descriptors. HTTP errors fail inference without writing output. Displayed URLs redact queries, fragments and user information. Native s3:// and Azure schemes are unsupported.
| Option | Values / default | Meaning |
|---|---|---|
--input-format <FORMAT> | auto, odcs, datapackage; auto | Source-standard selection, independent of descriptor transport |
--asset <ID> | Required | Exact canonical asset identifier |
--output <PATH>, -o <PATH> | Omitted | Write an enriched copy to a new file; missing parent directories are created |
--write | Off | Atomically replace an ordinary local descriptor; rejected for remote/archive sources; mutually exclusive with --output |
--maximum-records <COUNT> | Positive integer; 1000 | Maximum sampled CSV/Excel data records; ignored for databases, rejected for Parquet |
--maximum-bytes <COUNT> | Positive integer; CSV unlimited, Excel 64 MiB when omitted | CSV sample byte limit or complete Excel workbook byte limit; ignored for databases, rejected for Parquet |
--staging-maximum-bytes <COUNT> | Positive integer; 268435456 (256 MiB) | Per-resource copy limit for non-seekable Excel/Parquet input; independent of footer/sample limits; ignored for CSV/database inference |
--culture <NAME> | Invariant culture when omitted/empty | Culture for inferred CSV values, e.g. fr-FR; ignored for databases |
--header <MODE> | auto, present, absent; auto | CSV/Excel header mode; auto honors a declared header hint; ignored for databases, rejected for Parquet |
--sheet-name <NAME> | Omitted | Excel worksheet name, overriding endpoint selection; mutually exclusive with sheet number |
--sheet-number <NUMBER> | Positive integer; endpoint hint or first worksheet | Excel worksheet number starting at one; mutually exclusive with sheet name |
--no-dialect-inference | Off | Use contract dialect hints without inferring a CSV dialect; ignored for databases |
--connector <SCHEME=ID> | Repeatable; persisted preferences otherwise | Invocation-only connector preference |
--connection-env <ENDPOINT=VARIABLE> | Repeatable; no override by default | Runtime database URL from a named environment variable; not persisted |
CSV sampling and dialect options do not affect database discovery, but their supplied values still have to pass syntax and value validation. Explicit culture/dialect switches are rejected for Excel. Explicit CSV sampling, header, culture and dialect switches are rejected for Parquet. Excel sheet switches are rejected for other file formats. Omitting these options permits normal dispatch.
Inference has no output-format switch because an enriched file keeps the source representation. It also has no --overwrite option. Use --output or its alias, -o, to create a new file. See database override precedence for runtime connection settings.
packata schema infer datapackage.json --asset customers --header present --maximum-records 100
packata infer datapackage.json --asset customers --maximum-bytes 1048576 --culture en-US
packata schema infer datapackage.json --asset customers --no-dialect-inference --output enriched.json
packata schema infer contract.yaml --asset customers --connector postgresql=postgresql-npgsql --connection-env warehouse=WAREHOUSE_URL
packata schema infer workbook-contract.yaml --asset measurements --sheet-name Measurements --header present --write
packata schema infer parquet-contract.yaml --asset measurements --output enriched.yaml
packata schema infer https://example.org/contracts/contract.yaml --asset customers
packata schema infer https://example.org/packages/package.tar.gz --asset customers
Output and mutation
Inference prints a human-readable report of its mapping and inference decisions, the selected schema and dialect, and confidence where available. The report has no machine-readable output contract.
Without an output option, the descriptor is left unchanged. Choose --output to create a new enriched file; an existing file is refused. Choose --write only when you want to replace the source itself. That update has no overwrite prompt.
--write is rejected for remote and archived descriptors before acquisition. --output normalizes all remote-relative resource paths to absolute URLs and reports CLI131 so reloading the standalone descriptor retains their original targets. Archived descriptors can be saved only when all resource references already use absolute locators; archive-relative references cause an explicit refusal before any output is written. Archives are neither extracted nor repackaged by inference. Updates retain atomic local-write protections.
An update enriches the selected resource or schema and its endpoint metadata. It does not convert the document into another standard. Review preservation warnings before accepting it: JSON or YAML is reserialized, comments and formatting may change, YAML anchors are not guaranteed to round-trip, and unsupported source constructs can limit the update.
Parquet maps declared scalar types/nullability, structs and lists; unsupported maps or legacy repeated fields produce error diagnostics. Nested schemas are display-only in the current document updater and persistence fails before writing. Physical Parquet facets remain in the library result and generate a warning when only scalar logical types/nullability are written. No CSV dialect is generated for Parquet.
Excel preserves selected sheet/header/leading-row decisions in spreadsheet dialect settings or ODCS Packata custom properties. Sampling never establishes requiredness; new fields are optional and authored requiredness is preserved. Formula caches may be stale, merged cells are not filled forward, and ambiguous headers/mixed types produce diagnostics. Named tables, arbitrary ranges and multi-row headers are unsupported. See file schema inference for library examples, mappings, stream ownership, resource limits and further limitations.
Runtime connection overrides are not saved in the enriched contract. Relative output paths resolve from the working directory, while data paths resolve from the contract’s resource context. Temporary staging filenames never replace resource identities.
Exit codes
| Code | Meaning |
|---|---|
0 | Inference completed |
1 | Inference, input, write or database operation failed |
2 | Inference caught cancellation; also CLI invocation/settings errors |
130 | Framework-level cancellation when handled before/around command execution |
These are the legacy inference semantics, not the 0/1/2/3/130 contract of validation/inspection/export/preview/diff. CSV inference is synchronous inside its command path and may not stop mid-sample. See read/infer task guide.