MultiThreadedFileReader

The reader returns a NumPy array with shape (frames, rows, cols) and preserves the source pixel dtype. Each iteration reads at most n_threads * chunk_size frames—one chunk per worker. File I/O runs with the Python GIL released.

from aare.experimental import MultiThreadedFileReader

with MultiThreadedFileReader(
    "frames.npy", n_threads=4, chunk_size=128, total_frames=10_000
) as reader:
    for frames in reader:
        process(frames)

Call read() directly for the next batch, or read_all() for all frames remaining from the current position. tell() and seek() expose the iteration position. The context manager closes all worker files on exit. close() is also available for explicit cleanup and may be called repeatedly.

class aare.experimental.MultiThreadedFileReader

Bases: pybind11_object

property bitdepth

property bytes_per_frame

property chunk_size

close(self: aare.experimental.MultiThreadedFileReader) → None

Close all worker files. Safe to call more than once.

property closed

property cols

property dtype

property n_threads

property next_read_bytes

property next_read_frames

read(self: aare.experimental.MultiThreadedFileReader) → numpy.ndarray

Read one chunk per active worker into a NumPy array.

Returns:

An array containing at most n_threads * chunk_size frames. An empty array is returned at the end of the configured frame range. The GIL is released while file data is read.

read_all(self: aare.experimental.MultiThreadedFileReader) → numpy.ndarray

Read all frames remaining from the current position.

property remaining_frames

property rows

seek(self: aare.experimental.MultiThreadedFileReader, frame_index: SupportsInt | SupportsIndex) → None

property source_total_frames

tell(self: aare.experimental.MultiThreadedFileReader) → int

property total_bytes

property total_frames