colstore.ColStoreWriter

class colstore.ColStoreWriter(path, mode, *, statistics=False)[source]

Bases: object

Append-only writer for a colstore file. See module docstring.

Use colstore.create(), colstore.recreate(), or colstore.update() rather than constructing directly. statistics=True records per-column statistics so later filters can skip data that cannot match; off by default.

Parameters:
property path: Path

Filesystem path the writer is appending to.

property mode: str

The mode this writer was opened in ("create", "recreate", "update").

property n_records: int

Number of records written (including any pre-existing in update mode).

property committed_rows: int

Total row count across all records written so far.

property closed: bool

Whether close() has run.

write(columns)[source]

Append one record. Schema is locked on the first non-empty call.

Parameters:

  • columns (dict[str, numpy.ndarray]) – Column-major data. Names must match (and dtypes match) the schema captured on the first write() (or loaded from the existing file in update mode). All columns must share the same length; that length is the new record’s row count.

  • written (Empty dicts are a no-op (no record)

  • yet (schema not locked)

  • record (in create/recreate mode). To write a zero-row)

  • pass

  • ``{name –

    empty_array_of_dtype:

  • explicitly. (...}``)

Raises:

  • ValueError – On empty schema (first call after empty dicts), ragged columns, or schema mismatch on a non-first call.

  • TypeError – On unsupported column dtypes.

Return type:

None

close()[source]

Commit counters, fsync, release the lock, close the file.

Idempotent: calling close() on an already-closed writer is a no-op.

In create/recreate mode, if nothing was ever written, the (zero- byte) file is removed: there’s no manifest to commit, and a zero-byte .cstore would fail every reader. In update mode, nothing-written means no counter change is needed.

Return type:

None