docs: use in-memory stores for examples that don't demonstrate disk storage

changes/4289.doc.md

@@ -0,0 +1,3 @@
+Documentation examples use fresh in-memory stores unless they demonstrate
+persistent storage. Array, group and quickstart ZIP examples use their own
+temporary directories without deleting existing reader data.

docs/quick-start.md

@@ -29,7 +29,7 @@ import numpy as np
 
 # Create a 2D Zarr array
 z = zarr.create_array(
-    store="data/example-1.zarr",
+    store={},
     shape=(100, 100),
     chunks=(10, 10),
     dtype="f4",
@@ -43,7 +43,7 @@ print(z.info)
 
 Here, we created a 2D array of shape `(100, 100)`, chunked into blocks of
 `(10, 10)`, and filled it with random floating-point data. This array was
-written to a `LocalStore` in the `data/example-1.zarr` directory.
+written to a fresh in-memory store.
 
 ### Compression and Filters
 
@@ -53,7 +53,7 @@ Zarr supports data compression and filters. For example, to use Blosc compressio
 
 # Create a 2D Zarr array with Blosc compression
 z = zarr.create_array(
-    store="data/example-2.zarr",
+    store={},
     shape=(100, 100),
     chunks=(10, 10),
     dtype="f4",
@@ -79,7 +79,7 @@ Zarr allows you to create hierarchical groups, similar to directories:
 ```python exec="true" session="quickstart" source="above" result="ansi"
 
 # Create nested groups and add arrays
-root = zarr.group("data/example-3.zarr", overwrite=True)
+root = zarr.group(store={})
 foo = root.create_group(name="foo")
 bar = root.create_array(
     name="bar", shape=(100, 10), chunks=(10, 10), dtype="f4"
@@ -94,7 +94,7 @@ spam[:] = np.arange(10)
 print(root.tree())
 ```
 
-This creates a group hierarchy with a group (`foo`) and two arrays (`bar` and `spam`).
+This creates an in-memory group hierarchy with a group (`foo`) and two arrays (`bar` and `spam`).
 
 ### Batch Hierarchy Creation
 
@@ -104,7 +104,7 @@ Suppose we want to copy existing groups and arrays into a new storage backend:
 ```python exec="true" session="quickstart" source="above" result="code"
 
 # Create nested groups and add arrays
-root = zarr.group("data/example-4.zarr", attributes={'name': 'root'}, overwrite=True)
+root = zarr.group(store={}, attributes={'name': 'root'})
 foo = root.create_group(name="foo")
 bar = root.create_array(
     name="bar", shape=(100, 10), chunks=(10, 10), dtype="f4"
@@ -126,15 +126,20 @@ be done in a separate step.
 
 ## Persistent Storage
 
-Zarr supports persistent storage to disk or cloud-compatible backends. While examples above
-utilized a [`zarr.storage.LocalStore`][], a number of other storage options are available.
+Zarr supports persistent storage to disk or cloud-compatible backends. While the examples
+above all used in-memory stores, a number of persistent storage options are available.
 
 A single-file store can also be created using the [`zarr.storage.ZipStore`][]:
 
 ```python exec="true" session="quickstart" source="above"
+from pathlib import Path
+from tempfile import TemporaryDirectory
+
+zip_directory = TemporaryDirectory(prefix="zarr-quickstart-")
+zip_path = Path(zip_directory.name) / "example.zip"
 
 # Store the array in a ZIP file
-store = zarr.storage.ZipStore("data/example-5.zip", mode="w")
+store = zarr.storage.ZipStore(zip_path, mode="w")
 
 z = zarr.create_array(
     store=store,
@@ -155,12 +160,14 @@ To open an existing array from a ZIP file:
 ```python exec="true" session="quickstart" source="above" result="ansi"
 
 # Open the ZipStore in read-only mode
-store = zarr.storage.ZipStore("data/example-5.zip", read_only=True)
+store = zarr.storage.ZipStore(zip_path, read_only=True)
 
 z = zarr.open_array(store, mode='r')
 
 # read the data as a NumPy Array
 print(z[:])
+store.close()
+zip_directory.cleanup()
 ```
 
 Zarr also integrates seamlessly with cloud object storage such as Amazon S3 and Google

docs/user-guide/arrays.md

@@ -10,16 +10,14 @@ import numpy as np
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 import zarr
-z = zarr.create_array(store="memory://arrays-demo", shape=(10000, 10000), chunks=(1000, 1000), dtype='int32')
+z = zarr.create_array(store={}, shape=(10000, 10000), chunks=(1000, 1000), dtype='int32')
 print(z)
 ```
 
 The code above creates a 2-dimensional array of 32-bit integers with 10000 rows
 and 10000 columns, divided into chunks where each chunk has 1000 rows and 1000
 columns (and so there will be 100 chunks in total). The data is written to an
-in-memory store: when `fsspec` is installed, a `memory://` URL resolves to a
-[`zarr.storage.FsspecStore`][] backed by fsspec's in-memory filesystem; otherwise a
-[`zarr.storage.ManagedMemoryStore`][] is used. See the [Storage guide](storage.md)
+in-memory [`zarr.storage.MemoryStore`][]. See the [Storage guide](storage.md)
 for more details on stores, and
 [Persistent arrays](#persistent-arrays) for details on storing arrays in other stores.
 See [Data types](data_types.md) for an in-depth look at the data types supported
@@ -82,12 +80,18 @@ persistence of data between sessions. To do this, we can change the store
 argument to point to a filesystem path:
 
 ```python exec="true" session="arrays" source="above"
-z1 = zarr.create_array(store='data/example-1.zarr', shape=(10000, 10000), chunks=(1000, 1000), dtype='int32', overwrite=True)
+from pathlib import Path
+from tempfile import TemporaryDirectory
+
+array_directory = TemporaryDirectory(prefix='zarr-arrays-')
+array_path = Path(array_directory.name) / 'example-1.zarr'
+z1 = zarr.create_array(store=array_path, shape=(10000, 10000), chunks=(1000, 1000), dtype='int32')
 ```
 
 The array above will store its configuration metadata and all compressed chunk
-data in a directory called `'data/example-1.zarr'` relative to the current working
-directory. The [`zarr.create_array`][] function provides a convenient way
+data at `array_path` inside a temporary directory. This directory is automatically
+removed when `array_directory` is cleaned up; choose your own path to retain data
+between Python sessions. The [`zarr.create_array`][] function provides a convenient way
 to create a new persistent array or continue working with an existing
 array. Note, there is no need to close an array: data are automatically
 flushed to disk, and files are automatically closed whenever an array is modified.
@@ -104,7 +108,7 @@ z1[:, 0] = np.arange(10000)
 Check that the data have been written and can be read again:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
-z2 = zarr.open_array('data/example-1.zarr', mode='r')
+z2 = zarr.open_array(array_path, mode='r')
 print(np.all(z1[:] == z2[:]))
 ```
 
@@ -116,8 +120,10 @@ path; `zarr.save_array` accepts `mode="w"` to do so. E.g.:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 a = np.arange(10)
-zarr.save_array('data/example-2.zarr', a, mode="w")
-print(zarr.load('data/example-2.zarr'))
+saved_array_path = Path(array_directory.name) / 'example-2.zarr'
+zarr.save_array(saved_array_path, a, mode='w')
+print(zarr.load(saved_array_path))
+array_directory.cleanup()
 ```
 
 Please note that there are a number of other options for persistent array
@@ -129,7 +135,7 @@ A Zarr array can be resized, which means that any of its dimensions can be
 increased or decreased in length. For example:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
-z = zarr.create_array(store='data/example-3.zarr', shape=(10000, 10000), dtype='int32', chunks=(1000, 1000), overwrite=True)
+z = zarr.create_array(store={}, shape=(10000, 10000), dtype='int32', chunks=(1000, 1000))
 z[:] = 42
 print(f"Original shape: {z.shape}")
 z.resize((20000, 10000))
@@ -145,7 +151,7 @@ used to append data to any axis. E.g.:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 a = np.arange(10000000, dtype='int32').reshape(10000, 1000)
-z = zarr.create_array(store='data/example-4.zarr', shape=a.shape, dtype=a.dtype, chunks=(1000, 100), overwrite=True)
+z = zarr.create_array(store={}, shape=a.shape, dtype=a.dtype, chunks=(1000, 100))
 z[:] = a
 print(f"Original shape: {z.shape}")
 z.append(a)
@@ -206,7 +212,7 @@ argument accepted by all array creation functions. For example:
 ```python exec="true" session="arrays" source="above" result="ansi"
 compressors = zarr.codecs.BloscCodec(cname='zstd', clevel=3, shuffle='bitshuffle')
 data = np.arange(100000000, dtype='int32').reshape(10000, 10000)
-z = zarr.create_array(store='data/example-5.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=compressors, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=compressors)
 z[:] = data
 print(z.compressors)
 ```
@@ -241,7 +247,7 @@ compressor.
 To create an array without any compression, set `compressors=None`:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
-z_no_compress = zarr.create_array(store='data/example-uncompressed.zarr', shape=(10000, 10000), chunks=(1000, 1000), dtype='int32', compressors=None, overwrite=True)
+z_no_compress = zarr.create_array(store={}, shape=(10000, 10000), chunks=(1000, 1000), dtype='int32', compressors=None)
 print(f"Compressors: {z_no_compress.compressors}")
 ```
 
@@ -250,7 +256,7 @@ here is an array using Gzip compression, level 1:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 data = np.arange(100000000, dtype='int32').reshape(10000, 10000)
-z = zarr.create_array(store='data/example-6.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=zarr.codecs.GzipCodec(level=1), overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=zarr.codecs.GzipCodec(level=1))
 z[:] = data
 print(f"Compressors: {z.compressors}")
 ```
@@ -265,7 +271,7 @@ from zarr.codecs.numcodecs import LZMA
 lzma_filters = [dict(id=lzma.FILTER_DELTA, dist=4), dict(id=lzma.FILTER_LZMA2, preset=1)]
 compressors = LZMA(filters=lzma_filters)
 data = np.arange(100000000, dtype='int32').reshape(10000, 10000)
-z = zarr.create_array(store='data/example-7.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=compressors, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), compressors=compressors)
 print(f"Compressors: {z.compressors}")
 ```
 
@@ -290,7 +296,7 @@ from zarr.codecs.numcodecs import Delta
 filters = [Delta(dtype='int32')]
 compressors = zarr.codecs.BloscCodec(cname='zstd', clevel=1, shuffle='shuffle')
 data = np.arange(100000000, dtype='int32').reshape(10000, 10000)
-z = zarr.create_array(store='data/example-9.zarr', shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), filters=filters, compressors=compressors, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype, chunks=(1000, 1000), filters=filters, compressors=compressors)
 print(z.info_complete())
 ```
 
@@ -315,7 +321,7 @@ coordinates. E.g.:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 data = np.arange(10) ** 2
-z = zarr.create_array(store='data/example-10.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype)
 z[:] = data
 print(z[:])
 print(z.get_coordinate_selection([2, 5]))
@@ -333,7 +339,7 @@ e.g.:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 data = np.arange(15).reshape(3, 5)
-z = zarr.create_array(store='data/example-11.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype)
 z[:] = data
 print(z[:])
 ```
@@ -377,7 +383,7 @@ Items can also be extracted by providing a Boolean mask. E.g.:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 data = np.arange(10) ** 2
-z = zarr.create_array(store='data/example-12.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype)
 z[:] = data
 print(z[:])
 ```
@@ -398,7 +404,7 @@ Here's a multidimensional example:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 data = np.arange(15).reshape(3, 5)
-z = zarr.create_array(store='data/example-13.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype)
 z[:] = data
 print(z[:])
 ```
@@ -441,7 +447,7 @@ example, this allows selecting a subset of rows and/or columns from a
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 data = np.arange(15).reshape(3, 5)
-z = zarr.create_array(store='data/example-14.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype)
 z[:] = data
 print(z[:])
 ```
@@ -469,7 +475,7 @@ For convenience, the orthogonal indexing functionality is also available via the
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 data = np.arange(15).reshape(3, 5)
-z = zarr.create_array(store='data/example-15.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype)
 z[:] = data
 print(z.oindex[[0, 2], :])  # select first and third rows
 ```
@@ -495,7 +501,7 @@ orthogonal indexing is also available directly on the array:
 
 ```python exec="true" session="arrays" source="above" result="ansi"
 data = np.arange(15).reshape(3, 5)
-z = zarr.create_array(store='data/example-16.zarr', shape=data.shape, dtype=data.dtype, overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype)
 z[:] = data
 print(np.all(z.oindex[[0, 2], :] == z[[0, 2], :]))
 ```
@@ -508,7 +514,7 @@ a subset of chunk aligned rows and/or columns from a 2-dimensional array. E.g.:
 
 ```python exec="true" session="arrays" source="above"
 data = np.arange(100).reshape(10, 10)
-z = zarr.create_array(store='data/example-17.zarr', shape=data.shape, dtype=data.dtype, chunks=(3, 3), overwrite=True)
+z = zarr.create_array(store={}, shape=data.shape, dtype=data.dtype, chunks=(3, 3))
 z[:] = data
 ```
 
@@ -541,7 +547,7 @@ print(z.blocks[0, 1:3])
 Data can also be modified. Let's start by a simple 2D array:
 
 ```python exec="true" session="arrays" source="above"
-z = zarr.create_array(store='data/example-18.zarr', shape=(6, 6), dtype=int, chunks=(2, 2), overwrite=True)
+z = zarr.create_array(store={}, shape=(6, 6), dtype=int, chunks=(2, 2))
 ```
 
 Set data for a selection of items:
@@ -584,7 +590,7 @@ performance guide.
 Sharded arrays can be created by providing the `shards` parameter to [`zarr.create_array`][].
 
 ```python exec="true" session="arrays" source="above" result="ansi"
-a = zarr.create_array('data/example-20.zarr', shape=(10000, 10000), shards=(1000, 1000), chunks=(100, 100), dtype='uint8', overwrite=True)
+a = zarr.create_array(store={}, shape=(10000, 10000), shards=(1000, 1000), chunks=(100, 100), dtype='uint8')
 a[:] = (np.arange(10000 * 10000) % 256).astype('uint8').reshape(10000, 10000)
 print(a.info_complete())
 ```

docs/user-guide/cli.md

@@ -62,9 +62,10 @@ To open the array/group using the new metadata use:
 import zarr
 
 # create a small array to open (stands in for the migrated store)
-zarr.create_array("data/cli-demo.zarr", shape=(4, 4), chunks=(2, 2), dtype="i4", overwrite=True)
+store = {}
+zarr.create_array(store=store, shape=(4, 4), chunks=(2, 2), dtype="i4")
 
-zarr_with_v3_metadata = zarr.open("data/cli-demo.zarr", zarr_format=3)
+zarr_with_v3_metadata = zarr.open(store=store, zarr_format=3)
 ```
 
 Once you are happy with the conversion, you can run the following to remove the old v2 metadata:

docs/user-guide/groups.md

@@ -8,7 +8,7 @@ To create a group, use the [`zarr.create_group`][] function:
 
 ```python exec="true" session="groups" source="above" result="ansi"
 import zarr
-root = zarr.create_group(store="memory://groups-demo")
+root = zarr.create_group(store={})
 print(root)
 ```
 
@@ -83,10 +83,17 @@ print(root.tree())
 
 The [`zarr.open_group`][] function provides a convenient way to create or
 re-open a group stored in a directory on the file-system, with sub-groups stored in
-sub-directories, e.g.:
+sub-directories. This example uses a temporary directory that is automatically
+removed when `group_directory` is cleaned up; use a path of your choice to retain
+data between Python sessions:
 
 ```python exec="true" session="groups" source="above" result="ansi"
-root = zarr.open_group('data/group.zarr', mode='w')
+from pathlib import Path
+from tempfile import TemporaryDirectory
+
+group_directory = TemporaryDirectory(prefix='zarr-groups-')
+group_path = Path(group_directory.name) / 'group.zarr'
+root = zarr.open_group(group_path, mode='w')
 print(root)
 ```
 
@@ -147,8 +154,10 @@ from zarr.storage import LocalStore
 from pprint import pprint
 import io
 
+# Keep this LocalStore example in the session's own temporary directory.
+hierarchy_path = Path(group_directory.name) / 'hierarchy.zarr'
 node_spec = {'a/b/c': GroupMetadata()}
-nodes_created = dict(create_hierarchy(store=LocalStore(root='data'), nodes=node_spec, overwrite=True))
+nodes_created = dict(create_hierarchy(store=LocalStore(root=hierarchy_path), nodes=node_spec, overwrite=True))
 # Report nodes (pprint is used for cleaner rendering in the docs)
 output = io.StringIO()
 pprint(nodes_created, stream=output, width=60)
@@ -171,7 +180,7 @@ Diagnostic information about arrays and groups is available via the `info`
 property. E.g.:
 
 ```python exec="true" session="groups" source="above" result="ansi"
-root = zarr.group(store="memory://diagnostics-demo")
+root = zarr.group(store={})
 foo = root.create_group('foo')
 bar = foo.create_array(name='bar', shape=1000000, chunks=100000, dtype='int64')
 bar[:] = 42
@@ -196,4 +205,5 @@ Groups also have the [`zarr.Group.tree`][] method, e.g.:
 
 ```python exec="true" session="groups" source="above" result="ansi"
 print(root.tree())
+group_directory.cleanup()
 ```

docs/user-guide/performance.md

@@ -159,7 +159,7 @@ def timed_write(write_empty_chunks):
     data = np.random.randint(0, 255, shape)
     dtype = 'uint8'
     arr = zarr.create_array(
-        f'data/example-{write_empty_chunks}.zarr',
+        store={},
         shape=shape,
         chunks=chunks,
         dtype=dtype,