Skip to content

Add store methods for writing into an existing buffer #3831

@d-v-b

Description

@d-v-b

Store.get takes a prototype parameter, which is used to create a buffer in which to store the fetched bytes:

async def get(
self,
key: str,
prototype: BufferPrototype,
byte_range: ByteRequest | None = None,

I think this is not the best way to support a pattern where the caller wants to use a specific memory type. IMO it would be better if the caller was responsible to creating the buffer, and they pass a reference to that buffer to the store.

We should do this with a get_into method with the following signature:

    async def get_into(
        self,
        key: str,
        byte_range: ByteRequest | None = None,
        buf: Buffer,
    ) -> GetResult:

GetResult here is a data structure that conveys useful information about the IO operation. It would convey how much of the buffer was filled, whether the result was truncated, etc.

In concrete terms, I don't want to change the store ABC much, but I do think we could incrementally evolve our storage layer by increasingly relying on protocols. See #3758.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions