colstore.Appender

class colstore.Appender(directory, *, name='shard_{index:05d}.cstore', shard_size=None, statistics=False)

Bases: object

Streaming append: roll new shards into a dataset directory from a batch stream.

Construct via colstore.appender(). Each write() accumulates a batch; a new shard is committed whenever the buffer reaches shard_size (rows, a byte string like "512 MiB", or None to roll only on an explicit flush()). close() flushes any remainder and releases the directory lock. Use as a context manager. Like ColStoreWriter, a forgotten close is committed from __del__ with a ResourceWarning.

Parameters:

• directory (StrPath)

• name (str)

• shard_size (int | str | None)

• statistics (bool)

property directory: Path

The dataset directory being appended to.

property n_shards: int

Number of shards committed by this appender so far.

property pending_rows: int

Rows buffered but not yet committed to a shard.

property closed: bool

Whether close() has run.

write(data)

Buffer one batch; commit a shard if shard_size is reached.

Parameters:

data (Any)

Return type:

None

flush()

Commit the buffered batches as one shard now; None if nothing buffered.

Return type:

Path | None

close()

Flush any remainder, release the directory lock; idempotent.

Return type:

None