feat(gallery): Speed up load times and clean gallery entries#9211
Draft
richiejp wants to merge 2 commits intomudler:masterfrom
Draft
feat(gallery): Speed up load times and clean gallery entries#9211richiejp wants to merge 2 commits intomudler:masterfrom
richiejp wants to merge 2 commits intomudler:masterfrom
Conversation
mudler
reviewed
Apr 3, 2026
pkg/vram/cache.go
Outdated
| ) | ||
|
|
||
| // Model-level estimate result cache — keyed by model ID, avoids re-running | ||
| // the full estimation pipeline (HTTP HEAD, GGUF reads, HF API) on every |
Owner
There was a problem hiding this comment.
mmm readers are already cached in this file, I'm missing what's the issue trying to solve here? the first loading of the page is slow because it runs estimations, but if you go back to page already loaded that's not loaded from scratch- unless I'm missing something?
Collaborator
Author
There was a problem hiding this comment.
I found it to be very slow and this speeds it up mainly because it does the VRAM estimation asynchronously, but yeah I got into a mess here, there is a lot of duplication of VRAM estimation and we are doing quite a bit of unnecessary work. So I've reworked all of it into a single function and replaced the TTL cache with a generational cache and stuff like this.
richiejp
force-pushed
the
chore/gallery-speed-meta
branch
from
81ee94a to
86fda78
Compare
…n_usecases Signed-off-by: Richard Palethorpe <io@richiejp.com>
richiejp
force-pushed
the
chore/gallery-speed-meta
branch
from
86fda78 to
e98d30f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Improve the gallery load times and clean up the gallery entry meta-data, in particular the known_usecases.
clip-04-06_11-49-23.mp4
Notes for Reviewers
Signed commits