[mihow]

Summary

#1543 adds occurrence sets: named groups of occurrences that one project keeps and uses again. This PR lets people and the UI work with them over the REST API. A user can create a set, add occurrences to it or remove them, freeze it so that results that refer to it stay true, and copy a frozen set to make a new version. The occurrence list can show only the members of a set, or leave them out.

The API follows the model in #1543 and the design on the saved-sets epic (#1548, design notes). Each set belongs to one project, and that project's roles decide who can change it. Any project member can read a project's sets. ML data managers and project managers create, edit, freeze and copy them, as they manage capture sets.

List of Changes

Change (effect)	How	Notes
People can list, read, create, rename, describe and delete sets	OccurrenceSetViewSet in ami/main/api/occurrence_set.py, at /api/v2/occurrences/sets/	The list needs project_id
Pickers can list a project's sets in one request	GET .../sets/choices/?project_id= returns names, methods and dates, up to 100, most recently updated first	Same pattern as capture set choices
People can add or remove occurrences in bulk	POST .../sets/{id}/add/ and .../remove/ with {"occurrence_ids": [...]}, 1 to 1,000 ids. The ids are checked with one query; any id outside the set's project refuses the whole request	400 on a frozen set
People can freeze a set	POST .../sets/{id}/freeze/ records frozen_at and frozen_by. A frozen set refuses add, remove and a second freeze. It can still be renamed	No unfreeze
People can copy a set to make a new version	POST .../sets/{id}/copy/ with an optional name. It returns the new set, not frozen, with the same members and copied_from set	Needs the create permission
The UI can show how old a set is and which actions apply	Each set returns method, kwargs, occurrences_count, populated_at, frozen_at, frozen_by (name and image only), copied_from, and user_permissions (update, delete, freeze, copy)
The occurrence list can include or exclude a set	occurrence_set_id and not_occurrence_set_id, as OccurrenceSetFilter in OCCURRENCE_FILTER_BACKENDS, documented in the OpenAPI schema	The stats endpoint and exports can run the same backend
Permissions come from the set's own project	ObjectPermission with the model's checks. add, remove and freeze need the update permission; copy needs create	Replaces the earlier OccurrenceSetPermission and its shared-set rules
The OpenAPI schema stays readable	ENUM_NAME_OVERRIDES names the capture set and occurrence set method enums

Endpoints

Method and path	What it does
GET /occurrences/sets/?project_id=	List a project's sets, with member counts
GET /occurrences/sets/choices/?project_id=	Names and dates for pickers
POST /occurrences/sets/	Create a set: {"name", "description", "project"}. method is manual and kwargs must be empty
GET, PATCH, DELETE /occurrences/sets/{id}/	Read, rename or describe, delete. The project cannot change
POST /occurrences/sets/{id}/add/, .../remove/	Change the members: {"occurrence_ids": [...]}
POST /occurrences/sets/{id}/freeze/	Freeze the set
POST /occurrences/sets/{id}/copy/	Copy the set: optional {"name"}
GET /occurrences/?project_id=&occurrence_set_id=	Only the members of a set
GET /occurrences/?project_id=&not_occurrence_set_id=	Everything except the members of a set

Detailed Description

Errors.

• Invalid input gives a 400: bad or foreign ids, an empty list, more than 1,000 ids, kwargs on a manual set, a change of project, an occurrence_set_id that is not a positive number.
• Refused lifecycle actions also give a 400 with the reason, for example "A frozen set cannot be edited. Copy it to make a set that can change."
• The model checks the frozen state again under a row lock, so a freeze cannot land in the middle of an edit.

Query counts are measured in tests with strict counts:

• List: 10 queries for 3 sets and for 6 sets. Permissions are read once per project, not once per set.
• choices: 10 queries.
• add with 7 ids: 12 queries.

Tests. ami/main/models_future/tests/test_occurrence_set_api.py covers:

• the endpoints and the errors;
• the include and exclude filters;
• a permission matrix (superuser, ML data manager, project manager, basic member, non-member, anonymous) for every action;
• draft-project visibility.

The model tests from #1543 stay in test_occurrence_set.py. The earlier test file for the many-to-many API is removed.

What comes next

This is the order of work from #1548:

• The UI (Filter occurrences by a saved set, and save selected occurrences as a set #1545) and export (Export the occurrences in a chosen occurrence set #1546) will be updated to this API.
• Sets from filters come in a later PR:
  • a filters method, with the occurrence-list filters saved in kwargs;
  • populate in a job, which sets populated_at;
  • compare (what populating again would change, without saving);
  • convert to manual;
  • "Save as set" on the occurrence list.
• One FilterSet for the occurrence list (Use one shared path from occurrence filters to results, for the occurrence list and for exports #1537) comes first, so saved filters mean the same thing everywhere.
• Capture sets and taxa lists adopt the same lifecycle after that.

Note for the stacked branches (@mohamedelabbas1996)

• Filter names: occurrence_set_id and not_occurrence_set_id on the occurrence list. They are also available to exports, through OccurrenceSetFilter in ami.main.api.views.
• Creating a set takes project, not project_id, and no occurrences. Create the set, then POST .../add/ with the ids. Scoring and training should use a frozen set, or freeze a copy, and record the set's id and frozen_at.
• One project per set: for_project() and the shared-set rules are gone. Scope with project=....
• Endpoints: add, remove, freeze and copy are as listed above.

Open questions

1. Should roles beyond the curators (for example identifiers, for flagging) edit hand-made sets?
2. Should freezing have its own permission?
3. Curators can delete a frozen set. A score that refers to it would then name a deleted set.
4. Scoring and training jobs could refer to a set by a foreign key on Job or by a model reference in the job settings.
5. When the occurrence list is filtered by a set, the project's default filters still apply. A hand-picked occurrence below the score threshold is hidden unless the caller passes apply_defaults=false, so the UI may need to show "N of M shown".
6. Should create also accept a first list of occurrence_ids, so the UI can save a selection in one request?

How to Test the Changes

docker compose run --rm django python manage.py test ami.main.models_future.tests.test_occurrence_set ami.main.models_future.tests.test_occurrence_set_api ami.users ami.exports
docker compose run --rm django python manage.py makemigrations --check --dry-run

Deployment Notes

There are no new migrations in this PR; it uses the ones in #1543.

Checklist

• I have tested these changes appropriately.
• I have added and/or modified relevant tests.
• I updated relevant documentation or comments.
• I have verified that this PR follows the project's coding standards.
• Any dependent changes have already been merged to main. (Stacks on Introduce Occurrence Sets (manually curated only) #1543.)

🤖 Generated with Claude Code

https://claude.ai/code/session_01MeUoKCfkiZaZjhja7e6AGh

[netlify]

✅ Deploy Preview for antenna-preview canceled.

Name	Link
🔨 Latest commit	fff8db3
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/6ad9669a63ab3f63dcdc8d79

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[mihow]

Claude says: A note on permissions, for when this PR is next picked up. There are now three ways to avoid the per-row permission cost:

1. NoObjectPermissionsMixin (Filter captures and sessions by device type, site and date #1561, ami/base/serializers.py) returns an empty user_permissions list. It is for read-only nested rows and choice lists that never need permissions.
2. OccurrenceSetPermissionsMixin in this PR caches permissions on the serializer for each request.
3. Stop every list endpoint from looking up permissions once per row #1430 resolves each project's permissions once per request for all list endpoints.

#1430 and the cache in this PR solve the same problem. It may be simpler to adopt the #1430 approach here once it lands, instead of keeping a separate cache. The occurrence set choices actions in #1492 and #1501 can use the mixin for their choice serializers, as the job, device and site choices do.

[netlify]

✅ Deploy Preview for antenna-ssec ready!

Name	Link
🔨 Latest commit	fff8db3
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-ssec/deploys/6ae44f5fed87758d42c81af0
😎 Deploy Preview	https://deploy-preview-1544--antenna-ssec.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.