[mihow]

Summary

#1462 stores a feature vector for every detection, from any number of models, but on its own nothing in the interface mentions them. This PR is the small visible part on top: the occurrence page says which models have vectors for an occurrence, the occurrence list can be filtered by a model that only produces vectors (such as BioCLIP), and a job's "View occurrences" list includes occurrences that the job only added vectors to.

It is split out of #1462 so the storage could be reviewed and merged on its own. It needs both #1462 (the vector table) and #1518 (the shared query that the occurrence algorithm and job filters use), and merges after both.

List of Changes

#	Change (what it does for users)	How (implementation)
1	The occurrence Fields tab shows which models have feature vectors for the occurrence	embedding_algorithms on the occurrence detail response, one query; a "Feature vectors" row in the UI
2	Occurrence lists and exports stay as fast as before	the field is only on the detail response, and exports leave it out (EXPORT_EXCLUDED_FIELDS), since it would cost a query per occurrence
3	The occurrence algorithm filter finds occurrences by a model that only produces vectors	build_processed_by_algorithm_q() (#1518) gains a vector EXISTS, added only when the selected algorithms have vectors
4	Those models are offered in the algorithm filter's choices	Algorithm.objects.used_in_project() adds a vector id list, read through the vector table's own project column
5	A job's "View occurrences" list includes occurrences the job only added vectors to	build_created_or_updated_by_job_q() (#1518) gains a vector EXISTS, served by the partial (job, detection) index from #1462

Related Issues

• Builds on Store feature vectors for Detections, introduce BaseEmbedding model and general patterns for vectors #1462 (vector storage) and Show post-processing results in the occurrence algorithm filter, and keep the occurrence result queries side by side #1518 (shared occurrence results query). Merge order: Store feature vectors for Detections, introduce BaseEmbedding model and general patterns for vectors #1462, Show post-processing results in the occurrence algorithm filter, and keep the occurrence result queries side by side #1518, then this PR.
• This PR's base is Store feature vectors for Detections, introduce BaseEmbedding model and general patterns for vectors #1462's branch, so until Show post-processing results in the occurrence algorithm filter, and keep the occurrence result queries side by side #1518 merges its diff also shows Show post-processing results in the occurrence algorithm filter, and keep the occurrence result queries side by side #1518's changes (about 100 lines).

Detailed Description

Why the vector term is conditional

Postgres cannot plan EXISTS ... OR EXISTS ... as a semi-join, so ORing a term per table into the algorithm filter slows the common case of filtering by one classifier. The builder adds the vector EXISTS only when the selected algorithms have vectors, the same rule #1518 uses for algorithm results, so filtering by one classifier stays a single EXISTS.

Measured on #1462 with EXPLAIN (ANALYZE, BUFFERS), medians of 5 runs, read-only. They used a developer machine's copy of production data (projects of 179k and 45k occurrences, main tables only) and a small integration database that has the vector and result tables (21k detections, 30k vectors). Selecting one classifier on the 45k-occurrence project: page 1 takes about 0.8 ms on main, about 240 ms with one EXISTS per output table joined by OR, and about 0.8 ms with the conditional terms. The OR figure is approximated, because the production copy has no vector table. The counts are 223 ms, 461 ms and 138 ms. On the 179k-occurrence project, the OR shape took about 700 ms to count and the conditional shape takes 187 ms. A selection that mixes a classifier and an embedding model still needs two OR terms, and page 1 takes 19.5 ms on the small database. Timings for the vector table at production scale are not measured, because the table does not exist in the production copy yet.

Tests

• TestEmbeddingsInOccurrenceApi (ami/ml/test_detection_embeddings.py): the detail field lists each algorithm with a vector, costs one query whatever the number of algorithms, and is not on the list view; the algorithm filter finds and excludes occurrences by their vectors, including a vector stored after a cached answer.
• TestOccurrenceJobFilter.test_vector_only_link_to_a_job_matches and TestOccurrenceAlgorithmChoices (vector-only model offered; vectors in another project do not leak).
• DataExportTest.test_json_export_leaves_out_the_excluded_fields: exported rows never carry the field and its query never runs.

How to Test

1. docker compose -f docker-compose.ci.yml run --rm django python manage.py test ami.ml.test_detection_embeddings ami.main.tests.TestOccurrenceJobFilter ami.main.tests.TestOccurrenceAlgorithmFilterQuerySet ami.ml.tests.TestOccurrenceAlgorithmChoices ami.exports.tests.
2. Run a pipeline whose processing service returns embeddings on each detection (ami-data-companion Bump react-admin from 4.8.4 to 4.11.4 in /frontend #175 with features_for_all_detections on), then open one of its occurrences: the Fields tab lists the models under "Feature vectors".
3. In the occurrence list, filter by an embedding-only model (for example BioCLIP): only occurrences with its vectors remain, and the model appears in the filter's choices.

Screenshots

Checklist

• No model change; makemigrations --check reports no changes.
• Full backend suite at 142b086 (with main, Store feature vectors for Detections, introduce BaseEmbedding model and general patterns for vectors #1462 at 0585484 and Show post-processing results in the occurrence algorithm filter, and keep the occurrence result queries side by side #1518 at 5421043), run serially: 911 tests pass, 2 skipped

🤖 Generated with Claude Code

https://claude.ai/code/session_0121zMVjnPsqeDFSBXRCPvMy

[netlify]

✅ Deploy Preview for antenna-preview canceled.

Name	Link
🔨 Latest commit	9c4e8b3
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/6adbaf9b1509997769fcb748

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}