Streaming Battery Data

Many battery datasets are too large to fit in memory in a single computer at once. Such data can be read or written incrementally using the streaming module of battery data toolkit, batdata.streaming.

Reading Data as a Stream

The battery-data-toolkit allows streaming the raw time series data from an HDF5 file format.

Stream the data either as individual rows or all rows belonging to each cycle with the iterate_records_from_file() or iterate_cycles_from_file().

Both functions produce a Python generator which retrieves a chunk of data from the HDF5 file incrementally and can be used to produce data individually

row_iter = iterate_records_from_file('example.h5')
row = next(row_iter)
do_something_per_timestep(row)

or as part of a for loop.

for cycle in iterate_cycles_from_file('example.h5'):
    do_something_per_cycle(cycle)

Streaming Data to a File

TBD! Writing incrementally is not yet supported, but a feature on our road map.