ClusterFile¶

The ClusterFile() factory is the main interface for reading and writing cluster files. Use mode="r" to read, mode="w" to truncate and write, or mode="a" to append.

The format does not store the cluster dimensions, value type, coordinate type, padding, or byte order. The cluster_size and dtype arguments must match the writer; otherwise the bytes are interpreted incorrectly.

Use frames() when frame boundaries and the frame number matter:

with ClusterFile("clusters.clust") as file:
    for frame in file.frames():
        print(frame.frame_number, frame.size)

Each stored frame is yielded, including empty frames and frames whose clusters are all rejected by filtering. Missing frame numbers are not synthesized. The explicit read_frame() method returns one frame or None at EOF.

Use chunks() to iterate over chunks of chunk_size clusters. Per default the chunk size defined in the constructor is used. Alternatively one can pass an explicit chunk size to the function for that specific traversal:

with ClusterFile("clusters.clust", chunk_size=1000) as file:
    for clusters in file.chunks(10000):
        process(clusters)

Chunk sizes must be positive and count selected clusters after filtering. file.chunks(10000) does not overwrite the constructor argument chunk_size . Iterating directly over file or calling next(file) still reads chunks using that default. Chunks and read_clusters(n_clusters) may split or combine frames, so their frame numbers are not reliable per-cluster metadata. Empty chunks are not yielded. Chunk reads and iteration raise an error if they encounter an incomplete frame header or cluster record, including when ROI or noise filtering is enabled. They return fewer clusters than requested only at a clean end of file.

Both iterators consume the current file position without rewinding. Creating an iterator does not read; each next() reads one result without prefetching the following result. Use one traversal at a time and separate readers for independent cursors. Switching to frames after a chunk stops partway through a frame raises an error until the remaining clusters in that frame have been read. Exhausted iterators continue to raise StopIteration.

Iterators keep their file object alive, but explicitly closing the file or leaving its with block prevents further reads. Every yielded ClusterVector owns its storage and can be retained after iteration advances or the file closes. NumPy views also remain valid across those operations; as usual, resizing or modifying the allocation of their backing vector can invalidate them.

When a gain map is configured, it is applied to every cluster whose complete footprint lies inside the map. Clusters whose footprint extends beyond the gain-map boundaries remain in the returned cluster vector, but all their data values are set to zero.

aare.ClusterFile(fname, cluster_size=(3, 3), dtype=<class 'numpy.int32'>, chunk_size=1000, mode='r')¶

Create a reader or writer for a binary cluster file.

Parameters:

fname (path-like) – Cluster file to open.
cluster_size (tuple[int, int], default=(3, 3)) – Cluster dimensions stored in the file.
dtype (numpy dtype, default=numpy.int32) – Data type of the cluster values stored in the file.
chunk_size (int, default=1000) – Maximum number of selected clusters returned by chunks() and default iteration. Must be positive when iterating over chunks.
mode ({"r", "w", "a"}, default="r") – Open for reading, truncate and write, or append, respectively.

Returns:

The compiled ClusterFile specialization matching cluster_size and dtype.

Return type:

ClusterFile

Notes

The file format contains no cluster shape or data-type metadata. Supplying values that do not match the file causes its bytes to be interpreted incorrectly. Use frames() to iterate over complete frames, including empty or fully filtered frames with their stored frame numbers. Use chunks() or chunks(chunk_size) to iterate over selected clusters in batches. Chunks may combine frames, so their frame number is not reliable per-cluster metadata.

Iterators consume the current file position without rewinding; use one traversal at a time. Each result owns its storage and remains valid after advancing the iterator or closing the file.

Examples

from aare import ClusterFile

with ClusterFile(
    "clusters.clust", cluster_size=(3, 3), dtype=np.int32
) as cf:
    for clusters in cf.chunks():
        # Process clusters in chunks of at most 1000.
        ...

Below is the API of ClusterFile_Cluster3x3i; all compiled variants share the same API.

class aare._aare.ClusterFile_Cluster3x3i¶

Bases: pybind11_object

Read and write legacy binary cluster files. The format contains no cluster type or shape metadata, so this class must match the file.

__init__(self: aare._aare.ClusterFile_Cluster3x3i, fname: os.PathLike | str | bytes, chunk_size: SupportsInt | SupportsIndex = 1000, mode: str = 'r') → None¶: Open a cluster file. Mode must be ‘r’ to read, ‘w’ to truncate and write, or ‘a’ to append.

chunks(*args, **kwargs)¶

Overloaded function.

chunks(self: aare._aare.ClusterFile_Cluster3x3i) -> aare._aare.ClusterFile_Cluster3x3i_ChunkIterator

Iterate from the current position using the constructor’s chunk size, which must be positive. Results own their storage. The iterator keeps the file alive but shares its cursor; use only one traversal at a time.

chunks(self: aare._aare.ClusterFile_Cluster3x3i, chunk_size: typing.SupportsInt | typing.SupportsIndex) -> aare._aare.ClusterFile_Cluster3x3i_ChunkIterator

Iterate over chunks of up to chunk_size selected clusters. The size must be positive and does not change the constructor’s default. Chunks may span frames; their frame numbers are not per-cluster metadata. Only the final chunk can be short.

close(self: aare._aare.ClusterFile_Cluster3x3i) → None¶: Close the file. Calling close more than once is safe.

estimate_n_clusters(self: aare._aare.ClusterFile_Cluster3x3i) → int¶: Estimate the number of clusters from the file size. Frame headers can make this larger than the actual count.

frames(self: aare._aare.ClusterFile_Cluster3x3i) → aare._aare.ClusterFile_Cluster3x3i_FrameIterator¶: Iterate over complete frames from the current position, including empty and fully filtered frames with their stored frame numbers. Results own their storage. The iterator keeps the file alive but shares its cursor; use only one traversal at a time.

read_clusters(self: aare._aare.ClusterFile_Cluster3x3i, n_clusters: SupportsInt | SupportsIndex) → aare::ClusterVector<aare::Cluster<int, (unsigned char)3, (unsigned char)3, unsigned short>, void>¶: Read up to n_clusters without preserving frame boundaries. The result may combine frames, so its frame number is not reliable per-cluster metadata.

read_frame(self: aare._aare.ClusterFile_Cluster3x3i) → object¶: Read and return the next complete frame with its frame number, or None at end of file.

set_gain_map(self: aare._aare.ClusterFile_Cluster3x3i, gain_map: Annotated[numpy.typing.ArrayLike, numpy.float64]) → None¶: Set a two-dimensional, C-contiguous float64 gain map in ADU/energy, indexed as [y, x]. Clusters whose complete footprint extends beyond the map are retained with all data values set to zero.

set_noise_map(self: aare._aare.ClusterFile_Cluster3x3i, noise_map: Annotated[numpy.typing.ArrayLike, numpy.int32]) → None¶: Set a two-dimensional, C-contiguous int32 noise map indexed as [y, x]. The map must cover every cluster center coordinate.

set_roi(self: aare._aare.ClusterFile_Cluster3x3i, roi: aare::ROI) → None¶: Select clusters whose centers lie within the half-open ROI.

tell(self: aare._aare.ClusterFile_Cluster3x3i) → int¶: Return the current byte position in the file.

write_frame(self: aare._aare.ClusterFile_Cluster3x3i, clusters: aare::ClusterVector<aare::Cluster<int, (unsigned char)3, (unsigned char)3, unsigned short>, void>) → None¶: Write one ClusterVector, including its frame number.