[mihow]

Summary

Stacked on #1525 (vectors on the occurrence page and in the filters), which builds on #1462 (feature vector storage). That PR stores a feature vector for each detection but, on purpose, ships almost nothing that reads them. This one is the first thing a person can do with the stored vectors: open an occurrence, ask "which other occurrences look like this one?", and get the occurrence list ordered from most to least similar, according to the model they pick. A reviewer checking an unfamiliar moth can jump from it to its closest look-alikes, and a taxonomist can compare how two different models rank the same neighbours.

Nothing about identification changes. The sort only reorders the existing occurrence list; the project's default filters, permissions and every other filter still apply. Occurrences that have no vector from the chosen model sort last, in both directions. It is also the first reader of the vector table at list scale, so most of the changes below are guard rails that keep a sorted list from being slow or confusing. The storage plan is in #1457 (comment).

List of Changes

#	Change (effect for the user)	How	Notes
1	The occurrence list can be sorted by visual similarity to one occurrence, most similar first, or reversed	GET /occurrences/?ordering=visual_similarity&similar_to=<id>[&similarity_algorithm=<id>]; cosine distance from the occurrence's representative detection's vector, computed in a correlated subquery (OccurrenceQuerySet.with_visual_similarity, representative_embeddings); self_applied_orderings so the ordering filter does not drop it	Occurrences without a vector from that model come last in both directions. Only one model's vectors are compared per query
2	The compared-against occurrence must be named; without similar_to the API answers 400 with a clear message	SingleParamSerializer with required=True	Replaces a guessed seed (the most recently updated occurrence with a vector), which cost extra queries and changed whenever any occurrence was edited. similarity_algorithm still defaults to the model with the most vectors in the project
3	A sorted page loads faster	PostgreSQL JIT is switched off for this list (SET LOCAL jit = off, scoped to the request's transaction); the page count no longer carries the distance subquery (OccurrencePagination.get_count)	JIT off is the large effect (measured below). The count change is small. Pinned by tests: the count SQL contains no <=>; the statement is issued only for this ordering and before the first distance
4	A very long list is refused with a request to narrow the filters, instead of timing out	SIMILARITY_SORT_MAX_OCCURRENCES (environment variable, default 50,000), checked on the page count before any distance is computed	The number and the behaviour are proposals, see Open questions
5	The occurrence you are comparing against always appears in its own list, even if the project's default score or taxa filters would hide it	apply_default_filters(..., keep_pk=seed) ORs the seed into the default filters only	Filters the user chose still apply to it. Without this the list could open without the occurrence you asked about
6	The occurrence page has a "More actions" menu with one "Show similar by " entry per model that has a vector for that occurrence	Reuses the Popover from the UI kit and the existing link button styles; each entry passes its own similarity_algorithm; replaces a single link that used the first model	Prototype awaiting design. There is no Figma design for this menu
7	While a comparison is active the sortable column is headed "Similarity", and the first click sorts most similar first	TableColumn.defaultSortOrder is now honoured by the table header's first click (it only affected the sort control before); a read-only "Similar to occurrence" chip clears when another sort is chosen	Other columns keep their existing first-click order (descending)
8	The sort control and the column header show the same arrow direction for the same order (ascending points up in both)	SortControl rotated its arrow the opposite way to TableHeader	Affects every page that uses the sort control, since they all disagreed with their table headers
9	The model chosen in the menu is the model used for the list	similarity_algorithm was not a registered filter, so the list request never sent it and the default model was always used; it is now registered, shown read-only as "Compared by model", and cleared with the seed. Clearing the "Similar to occurrence" chip also ends the similarity sort	A demo where both models had equal vector counts had hidden this
10	Each row of a similarity-sorted list shows how similar it is, as a percentage under its thumbnail	New read-only visual_similarity field (cosine similarity, 1 is identical, null without a vector) from the existing distance annotation, which is renamed visual_distance; absent from other lists, the detail view and exports; no extra queries	The percentage display is a prototype awaiting design. Cosine similarity can be negative, and the demo shows that as a negative percentage

Open questions

1. The cap. Is 50,000 eligible occurrences the right limit, and is "400 with a request to narrow the filters" the right behaviour (alternatives: sort only the first N by another key, or fall back to an approximate index)? The number is an estimate from the earlier measurements below, not a decision. It is one setting, SIMILARITY_SORT_MAX_OCCURRENCES.
2. The "More actions" menu. It is a prototype so the feature can be tried. Placement, label, icon and whether other actions belong in the same menu need a design.
3. Default model. When similarity_algorithm is omitted, the model with the most vectors in the project is used. If the seed has no vector from that model the request is refused. Would it be better to require the model as well?
4. Column header. The sortable column is still the snapshots column, renamed while a comparison is active. A dedicated column showing the distance would be clearer but needs the distance in the API response.

Detailed Description

Where the time goes (measured)

Profiled in-process against two PostgreSQL 16 databases with pgvector, the occurrence list endpoint at 20 rows per page, cachalot off, no debug toolbar, one local machine:

• A local demo database (read-only session): its largest project has 8,288 occurrences and 12,168 vectors from the model compared.
• A synthetic database of mine (throwaway): 50,012 occurrences, each with one 2,048-value random vector. Shared buffers are 128 MB, so about 36,700 buffer reads per run come from the operating system cache.

What	8,288 occurrences	50,012 occurrences
List query, JIT off (this PR)	147 ms (EXPLAIN ANALYZE, warm)	790-850 ms
List query, JIT on (before this PR)	200-430 ms (JIT compile 36-68 ms)	1,460-1,500 ms (JIT compile about 710-730 ms)
Page count, with the distance subquery in the SQL (before)	not measured	98-105 ms
Page count, without it (this PR)	18 ms	54-64 ms
Whole request, 20 rows	350-440 ms	950-1,060 ms
Same request with the Django debug toolbar enabled (dev settings)	about 900-1,100 ms	not measured
First request vs later requests in a fresh process	about the same	about the same
After restarting PostgreSQL (empty shared buffers, warm OS cache)	not measured	844 ms vs 800 ms

Observations and what they imply:

1. Cost is linear in eligible occurrences, about 16 microseconds each with JIT off: roughly 11 microseconds for the per-occurrence vector probe and distance, and the rest for the joins and grouping that the plain list also does. That matches the 16-19 ms per 1,000 quoted from Store feature vectors for Detections, introduce BaseEmbedding model and general patterns for vectors #1462, and the page query, not the count, is where the time goes.
2. JIT was the large, fixable cost. It added about 0.7 s at 50,000 occurrences (measured), and it is now off for this list.
3. A correction to the earlier version of this description. It said a sorted page costs two distance passes because the count also computes the distance once per eligible occurrence. The count SQL did carry the subquery, but PostgreSQL drops an unused output column of a subquery, so the count was cheap (about 100 ms at 50,000), not a second pass. Removing it still roughly halves the count, so the change stays, but it is a small gain.
4. The plain list has an N+1 of its own: 144 queries for a 20-row page and 48 for a 5-row page (about 6.4 per row). The similarity sort adds a fixed handful on top (136 and 46 in the same comparison, so the sorted list is not worse per row). This is the existing per-row permission and prefetch cost of the occurrence list, accounting for roughly 0.2 s of a 20-row request. It deserves its own ticket.
5. A slow first click in a development stack is mostly the stack. The Django debug toolbar in the local settings records every query and roughly tripled the request time here. A 5 s first request was reported on a local development stack. I could not reproduce anything close to it on warm data (about 1 s with the toolbar, under 0.45 s without), and I cannot clear the operating system cache here, so the remaining difference is an inference and not a measurement: the first read of each vector from disk. Vectors are stored out of line (about 4 KB each at 2,048 values), so a cold first page has to read one per eligible occurrence. If that is right, it is a property of cold data and of how many occurrences are sorted, which the cap and the filters bound.

What we still need to verify

• Time a first and a second page on a large production-sized copy with the operating system cache dropped (it needs a machine where that is allowed), with and without a filter, to confirm point 5 and to choose the cap from data. The 50,000 default is still an extrapolation from points 1 and 2.
• Whether a two-step query (rank vectors with a lean query, then fetch the page's rows by id) is worth its complexity. By the plan above it could save roughly the 190 ms of joins and grouping at 50,000 occurrences, but not the distance work, so it is not proposed here.
• Whether production PostgreSQL's shared_buffers can hold the vectors of a typical project (a 50,000-vector model is about 280 MB).

Notes for reviewers

• The seed is validated like the other parameters (SingleParamSerializer): a non-integer, zero, an occurrence outside the project, and an occurrence without a vector from the chosen model each return 400 naming the parameter.
• One model per query is kept: the representative vector is looked up for one algorithm only, and an occurrence whose only vector is from another model sorts as if it had none.
• Project visibility is checked first; for a user who cannot see the project the sorted list is empty, exactly like the plain list, and says nothing about the seed.
• The row field visual_similarity is read from the existing annotation, so it costs no query; a test pins that the plain list, the detail view and the export serializer do not have it.
• The query-count test asserts the sort adds a fixed handful of queries (project visibility, default model, seed visibility, seed vector, and the JIT statement) however many rows a page has.

How to Test

1. Store vectors for two models on a project's detections (the example processing service, or the test fixtures in ami/main/test_visual_similarity.py).
2. GET /api/v2/occurrences/?project_id=<p>&ordering=visual_similarity&similar_to=<occurrence id> returns the seed first and its nearest neighbours next; adding similarity_algorithm=<id> switches the model; ordering=-visual_similarity reverses it with vector-less occurrences still last.
3. Leave out similar_to: the response is 400 naming similar_to.
4. Set SIMILARITY_SORT_MAX_OCCURRENCES=3 and repeat on a list of more than 3: 400 asking to narrow the filters; add a filter that brings the list under 3 and it sorts.
5. Raise the project's default score threshold above the seed's score: the seed still leads the list and the others it would hide are gone.
6. In the interface, open an occurrence that has vectors, open More actions, choose "Show similar by ", then click the Similarity header once (most similar first) and again (reverse). Choosing another sort clears the "Similar to occurrence" chip.
7. Open the occurrence page of an occurrence with vectors from two models and choose the model that is not the project's default: the list and the "Compared by model" chip follow the choice. The sort control and the Similarity header both show an upward arrow for most similar first.
8. Tests: python manage.py test ami.main.test_visual_similarity ami.ml.test_detection_embeddings; cd ui && yarn lint && npx tsc --noEmit && yarn test.

Screenshots

The More actions menu on an occurrence that has vectors from two models (prototype, awaiting design):

The list after choosing "Show similar by Color-sensitive backbone": the compared occurrence first, then its neighbours, each with its similarity percentage; the sortable column is headed Similarity, and the filters show the compared occurrence and the model used:

The same occurrence compared by the other model, which has vectors for only eight of the twelve occurrences, so the other four come last with a dash:

These were taken on a throwaway stack with a generated demo project and synthetic four-value vectors, so the neighbours shown are by construction (one model ranks by colour, the other by size).

Checklist

• Full backend suite at 11ae8c6 (with main, Store feature vectors for Detections, introduce BaseEmbedding model and general patterns for vectors #1462 and Show which models have feature vectors on an occurrence, and filter occurrences by them #1525 at their current heads), run serially: 936 tests pass, 2 skipped; makemigrations --check passes
• tsc --noEmit and ESLint pass on the UI after merging Show which models have feature vectors on an occurrence, and filter occurrences by them #1525

🤖 Generated with Claude Code

https://claude.ai/code/session_0121zMVjnPsqeDFSBXRCPvMy

[netlify]

✅ Deploy Preview for antenna-preview canceled.

Name	Link
🔨 Latest commit	2d4e03f
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/6adbb2416e1f21a393511e99

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}