Streaming (batdata.streaming)#
Retrieve data in smaller chunks from a large HDF5 file
- batdata.streaming.iterate_cycles_from_file(hdf5_path: Path | str | HDFStore, make_dataset: bool = False, **kwargs) Iterator[DataFrame | BatteryDataset]#
Stream single-cycle datasets from the HDF5 file
Keyword arguments are passed to
read_hdf(), which includes options to limit which rows and columns are read.- Parameters:
hdf5_path – Path to the data file
make_dataset – Whether to form a
BatteryDatasetfor each cycle, including the metadata from the source file.
- Yields:
All rows belonging to each cycle from the “raw_data” section fo the HDF5 file.
- batdata.streaming.iterate_records_from_file(hdf5_path: Path | str | HDFStore, **kwargs) Iterator[Dict[str, str | float | int]]#
Stream individual records from a file
Keyword arguments are passed to
read_hdf(), which includes options to limit which rows and columns are read.- Parameters:
hdf5_path – Path to the data file
- Yields:
Individual rows from the “raw_data” section of the HDF5 file