Skip to content

ondio.parquet

ondio.parquet

write_parquet 

write_parquet(uri: str, table, partition_cols: list[str], *, compression: str = 'snappy', overwrite: bool = False, max_workers: int | None = None, **kwargs: Any) -> None

Write a pyarrow.Table as a hive-partitioned parquet dataset under uri.

The dataset is written to a local temp directory via pyarrow, then uploaded file-by-file through the backend — pyarrow's native S3/GCS filesystems are deliberately not used, so credentials follow the backend path. Requires the parquet extra.

Parameters:

Name Type Description Default
uri str

Destination prefix for the dataset.

 required
table

The pyarrow.Table to write.

 required
partition_cols list[str]

Columns to hive-partition by (empty for none).

 required
compression str

Parquet compression codec.

 'snappy'
overwrite bool

If False, refuse to write when data already exists under the prefix. This check is not atomic with the write. If True, delete existing objects first via list_files + delete.

 False
max_workers int | None

Thread-pool size for the per-file uploads.

 None
**kwargs Any

Forwarded to the backend constructor.

 {}

Raises:

Type Description
OndioError

If data exists under the prefix and overwrite=False.
ImportError

If the parquet extra is missing.