[DanielDarrenJones]

An asset container keeps a listing of every file on its disk, with each file's size and timestamp. It's loaded on any request that touches an asset. Two things don't make use of it, and both get expensive as a container grows, most of all when its disk is remote.

Checking an asset exists

Asset::exists() asks the container for all its files and checks whether the path is among them. That filters every row of the listing, copies the keys into a new collection, and searches it from the start. metaExists() does the same over the meta files.

exists() is called from meta(), which is called for every value read from an asset, so a page showing 50 assets calls it over a thousand times. On a container of 20,000 images that is about 135 ms per request for the two filters and 0.13 ms per call after that. It's CPU rather than disk, so it costs the same on a local disk.

AssetContainerContents gets hasFile($path), hasMetaFile($path) and file($path), which look the path up directly and apply the same rules as before (a file, not inside .meta, not .DS_Store, .gitkeep or .gitignore). Asset::exists() and metaExists() use them.

Size and last modified

Sorting or filtering assets by a value builds a Stache index for it, which asks every asset in the container for that value. Size and last modified come from the asset's meta, and a meta that isn't cached is read from its .meta file: one read per asset. Both are visible, sortable columns in the asset browser by default, so one click on a column header does it.

On 20,000 images the first sort by size made 19,748 reads. That took about 18 seconds on a local disk. On a disk where a read takes 40 ms it is about 13 minutes.

Asset::size() and lastModified() now use the listing, and fall back to the meta when the listing has no entry for the file (as with the Eloquent driver, whose listing holds only directories). generateMeta() uses the listing's values too instead of asking the disk again. reupload() refreshes the file's listing entry so the size can't be out of date afterwards.

Width, height, duration and blueprint fields still come from the meta, so sorting by those is unchanged.

One existing test changed

it_generates_meta_on_demand_if_a_required_value_is_missing called size() to trigger a meta rebuild. size() no longer needs the meta, so the test calls mimeType() instead.

Benchmark

A generated site with one asset container of 20,000 images in 200 folders, each with its .meta file, and entries that reference them through assets fields. The container's disk is a Flysystem adapter wrapped round the local one that counts every call and adds 40 ms to each (120 ms per 1,000 keys listed), so Statamic takes the paths it takes on S3. No real S3 was used: the call counts are exact, the times are PHP time plus that modelled network time. Requests went through php -S with OPcache, production env, watcher off, against 6.x at 7097691 (Apple M1 Pro, PHP 8.5). Single runs; PHP-only timings moved by up to about 30% between runs, call counts did not.

	Before	After
Page with 50 assets, everything cached (PHP time)	591 ms	274 ms
Asset browser folder of 100, everything cached (PHP time)	599 ms	246 ms
Entry publish form with 50 assets (PHP time)	411 ms	111 ms
Asset browser: sort a folder by size, first time, PHP time alone	18.9 s	0.6 s
The same, with the modelled remote disk	19,748 reads, 809 s	49 reads, 2.5 s
Page showing 10 assets that have no .meta file yet (remote disk)	70 calls	50 calls

Tests

Each fails without the change:

• AssetTest::checking_existence_does_not_filter_the_whole_container_listing, which also covers the paths that must not count as assets.
• AssetTest::it_gets_size_and_last_modified_from_the_container_listing_without_touching_meta.
• AssetTest::generating_meta_uses_the_size_and_timestamp_the_listing_already_has.