-
-
Notifications
You must be signed in to change notification settings - Fork 391
Description
Store.get takes a prototype parameter, which is used to create a buffer in which to store the fetched bytes:
zarr-python/src/zarr/abc/store.py
Lines 196 to 200 in 690c5bb
| async def get( | |
| self, | |
| key: str, | |
| prototype: BufferPrototype, | |
| byte_range: ByteRequest | None = None, |
I think this is not the best way to support a pattern where the caller wants to use a specific memory type. IMO it would be better if the caller was responsible to creating the buffer, and they pass a reference to that buffer to the store.
We should do this with a get_into method with the following signature:
async def get_into(
self,
key: str,
byte_range: ByteRequest | None = None,
buf: Buffer,
) -> GetResult:
GetResult here is a data structure that conveys useful information about the IO operation. It would convey how much of the buffer was filled, whether the result was truncated, etc.
In concrete terms, I don't want to change the store ABC much, but I do think we could incrementally evolve our storage layer by increasingly relying on protocols. See #3758.