colstore.store

colstore.store(data, path, *, mode='create', batch_size='auto', show_progress=True, statistics=False, **open_kwargs)

One-shot: write a single-record file and return an opened reader.

Accepted data types: dict[str, numpy.ndarray] (column-major mapping), structured numpy.ndarray (one column per field), or a pandas DataFrame (one column per series). mode is "create" (default; fail if the file exists) or "recreate" (truncate if it exists). For multi-record streaming writes, use create() / recreate() / update() directly.

Parameters:

• batch_size (int, str, or None, default "auto") –

  Write chunking for the progress bar; no effect on the bytes written.

  • "auto" – adaptive: probe with a 1 MiB initial batch, then grow from EMA-smoothed measured bandwidth (2x growth cap per batch); fast NVMe ramps to GiB-class batches, slow disks settle at tens of MiB. Files under 16 MiB single-pass.

  • None – single pass: one tofile call per column.

  • int N – rows x columns per logical batch (batch_size=100_000 with 5 columns: 20 000-row writes).

  • str like "100 MB", "1.5 GiB" – bytes per batch. Units follow IEC 80000-13: decimal kB/MB/GB are powers of 1000 and binary KiB/MiB/GiB are powers of 1024, so "1 MB" is 1,000,000 bytes and "1 MiB" is 1,048,576.

• show_progress (bool, default True) – Whether to display a tqdm progress bar. The bar’s postfix shows the batch count and rows=...Mrows/s; the byte rate is rendered by the byte-counted bar itself.

• statistics (bool, default False) – Record per-column statistics so later filters can skip data that cannot match. Most useful for selective queries on sorted or clustered data.

• data (Any)

• path (str | PathLike[str])

• mode (str)

• open_kwargs (Any)

Return type:

ColStoreReader

:param Returns the opened ColStoreReader for immediate use.: