CMR-11240: We no longer exclude concepts that are not in progress when a concept identifier is used in the search query#2405
Merged
daniel-zamora merged 4 commits intomasterfrom Mar 31, 2026
Merged
Conversation
…de non-operational collections
📝 WalkthroughWalkthroughUpdated non-operational collection filtering to bypass the non-operational default filter when searches include collection identifier parameters; added explicit precedence for an explicit Changes
Sequence Diagram(s)(omitted — changes are limited to parameter parsing, docs, and tests and do not introduce a multi-component runtime flow requiring visualization) Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
daniel-zamora
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
system-int-test/test/cmr/system_int_test/search/collection/collection_progress_search_test.clj (2)
79-100: Consider wrapping flag toggle in try-finally for test isolation.If any assertion fails between enabling the flag (line 79) and disabling it (line 100), the flag remains enabled, potentially affecting subsequent tests in the same test run. This applies to both
search-collection-progress-active-filterandsearch-collection-progress-identifier-bypass.♻️ Suggested refactor using try-finally
(testing "flag OFF - param ignored, all collections returned" (util/are3 [items search] (d/refs-match? items (search/find-refs :collection search)) "no params" all-colls {} "include-non-operational=false" all-colls {:include-non-operational "false"} "include-non-operational=true" all-colls {:include-non-operational "true"})) - (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! true)) - - (testing "flag ON - filter applies unless overridden" - (util/are3 [items search] - (d/refs-match? items (search/find-refs :collection search)) - - "no params - only operational collections (excludes PLANNED, DEPRECATED, PREPRINT, INREVIEW)" - active-colls {} - - "include-non-operational=false - only operational collections" - active-colls {:include-non-operational "false"} - - "include-non-operational=true - all collections" - all-colls {:include-non-operational "true"} - - "explicit collection-progress=PLANNED - PLANNED returned (filter not applied)" - [coll2] {:collection-progress "PLANNED"} - - "collection-progress=PLANNED + include-non-operational=false - empty (explicit filter wins)" - [] {:collection-progress "PLANNED" :include-non-operational "false"})) - - (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! false)))) + (try + (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! true)) + + (testing "flag ON - filter applies unless overridden" + (util/are3 [items search] + (d/refs-match? items (search/find-refs :collection search)) + + "no params - only operational collections (excludes PLANNED, DEPRECATED, PREPRINT, INREVIEW)" + active-colls {} + + "include-non-operational=false - only operational collections" + active-colls {:include-non-operational "false"} + + "include-non-operational=true - all collections" + all-colls {:include-non-operational "true"} + + "explicit collection-progress=PLANNED - PLANNED returned (filter not applied)" + [coll2] {:collection-progress "PLANNED"} + + "collection-progress=PLANNED + include-non-operational=false - empty (explicit filter wins)" + [] {:collection-progress "PLANNED" :include-non-operational "false"})) + (finally + (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! false))))))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@system-int-test/test/cmr/system_int_test/search/collection/collection_progress_search_test.clj` around lines 79 - 100, Wrap the flag toggle calls to cmr.search.config/set-enable-non-operational-collection-filter! in a try/finally so the flag is always reset to false even if assertions fail; specifically, around the test block that sets the flag to true (the code invoking (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! true))) put the tests (the util/are3 block for search-collection-progress-active-filter and similarly in search-collection-progress-identifier-bypass) inside the try and call (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! false))) in the finally to guarantee test isolation.
156-157: Native-id correctly defaults to EntryTitle—optional clarity improvement.The test is correct:
coll3is ingested withEntryTitle "ET3", and since no explicit:native-idis set,umm-c-collection->conceptdefaults it to the trimmed EntryTitle via theitem->native-idfunction. The search{:native-id "ET3"}therefore correctly matchescoll3.This assumption is safe and consistent with the test pattern used for entry-title on line 150–151. However, to improve clarity, consider either explicitly setting
:native-id "ET3"in the collection data or retrieving it from the ingested reference (e.g.,(:native-id coll3)), which would make the implicit derivation explicit.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@system-int-test/test/cmr/system_int_test/search/collection/collection_progress_search_test.clj` around lines 156 - 157, Test relies on implicit defaulting of native-id from EntryTitle for coll3 which is correct but unclear; make it explicit by either adding :native-id "ET3" to the collection data used to ingest coll3 or by using the actual native-id from the ingested reference (e.g., refer to (:native-id coll3)) in the search assertion. Update the test where coll3 is defined or where the search vector is built to use the explicit native-id so readers don't need to infer umm-c-collection->concept / item->native-id behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@search-app/docs/api.md`:
- Line 1944: Update the NOTE sentence to use the API's underscore-style query
parameter names instead of hyphens: replace "concept-id", "entry-id",
"entry-title", "short-name" and "native-id" with "concept_id", "entry_id",
"entry_title", "short_name" (with "version" where noted) and "native_id", and
keep the reference to the include-non-operational=true flag and the rule about
searching by collection identifiers unchanged.
---
Nitpick comments:
In
`@system-int-test/test/cmr/system_int_test/search/collection/collection_progress_search_test.clj`:
- Around line 79-100: Wrap the flag toggle calls to
cmr.search.config/set-enable-non-operational-collection-filter! in a try/finally
so the flag is always reset to false even if assertions fail; specifically,
around the test block that sets the flag to true (the code invoking
(side/eval-form
`(cmr.search.config/set-enable-non-operational-collection-filter! true))) put
the tests (the util/are3 block for search-collection-progress-active-filter and
similarly in search-collection-progress-identifier-bypass) inside the try and
call (side/eval-form
`(cmr.search.config/set-enable-non-operational-collection-filter! false))) in
the finally to guarantee test isolation.
- Around line 156-157: Test relies on implicit defaulting of native-id from
EntryTitle for coll3 which is correct but unclear; make it explicit by either
adding :native-id "ET3" to the collection data used to ingest coll3 or by using
the actual native-id from the ingested reference (e.g., refer to (:native-id
coll3)) in the search assertion. Update the test where coll3 is defined or where
the search vector is built to use the explicit native-id so readers don't need
to infer umm-c-collection->concept / item->native-id behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: a57a5d0d-f659-46ee-8cfc-64940665e1e6
📒 Files selected for processing (4)
search-app/docs/api.mdsearch-app/src/cmr/search/services/parameters/conversion.cljsystem-int-test/src/cmr/system_int_test/utils/url_helper.cljsystem-int-test/test/cmr/system_int_test/search/collection/collection_progress_search_test.clj
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@search-app/docs/api.md`:
- Line 1944: Update the NOTE to clarify that the identifier-based bypass only
prevents the service from auto-applying the default non-operational collection
filter — it does not override an explicit query parameter; state that when
caller sets include-non-operational=false (or true) explicitly, that explicit
setting still applies even if identifier params (concept_id, entry_id,
entry_title, short_name+version, native_id) are present.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2405 +/- ##
==========================================
- Coverage 57.90% 57.89% -0.01%
==========================================
Files 1067 1067
Lines 73454 73462 +8
Branches 2127 2126 -1
==========================================
+ Hits 42532 42534 +2
- Misses 28942 28944 +2
- Partials 1980 1984 +4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
eudoroolivares2016
approved these changes
Mar 30, 2026
| Valid values: ACTIVE, PLANNED, COMPLETE, DEPRECATED, NOT PROVIDED, PREPRINT, INREVIEW, SUPERSEDED | ||
|
|
||
| **NOTE:** When the non-operational collection filter is enabled (feature flag), collections with PLANNED, DEPRECATED, PREPRINT, and INREVIEW status are excluded from default search results. Use `include-non-operational=true` to include them. This is an experimental feature and may not be enabled in all environments. | ||
| **NOTE:** When the non-operational collection filter is enabled (feature flag), collections with PLANNED, DEPRECATED, PREPRINT, and INREVIEW status are excluded from default search results. Use `include-non-operational=true` to include them. The default filter is automatically bypassed when searching by collection identifiers (`concept_id`, `entry_id`, `entry_title`, `short_name` with `version`, or `native_id`), allowing you to find specific collections regardless of their progress status. However, if you explicitly set `include-non-operational=false` (or `true`), that explicit setting takes precedence even when using identifier parameters. |
Contributor
There was a problem hiding this comment.
Can we add documentation for the include_non_operational url parameter in the docs while we are in here
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
system-int-test/test/cmr/system_int_test/search/collection/collection_progress_search_test.clj (1)
79-100: Always restore the feature flag withtry/finallyto prevent cross-test leakage.The flag reset currently depends on reaching the last line of each test. If an exception occurs earlier, later tests can run with the wrong global config.
🔧 Suggested hardening pattern
- (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! true)) - - (testing "flag ON - filter applies unless overridden" - ...) - - (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! false)))) + (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! true)) + (try + (testing "flag ON - filter applies unless overridden" + ...) + (finally + (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! false))))))Apply the same pattern to the second test block (Line 132 onward).
Also applies to: 132-171
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@system-int-test/test/cmr/system_int_test/search/collection/collection_progress_search_test.clj` around lines 79 - 100, The test toggles the global flag via (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! true)) and then resets it to false, but if an exception occurs the reset is skipped; change both test blocks that call set-enable-non-operational-collection-filter! to wrap the body in a try/finally so the final (side/eval-form `(cmr.search.config/set-enable-non-operational-collection-filter! false)) runs in the finally clause; locate the occurrences surrounding the testing "flag ON - filter applies unless overridden" block and the second test block (the one starting around line 132) and move their test logic into try with the reset call placed in finally.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@search-app/docs/api.md`:
- Line 1944: The note is using a different query parameter spelling
(`include-non-operational`) than the rest of the doc which uses
`include_non_operational`; change the hyphenated form to the underscored form so
the note uses `include_non_operational` (ensure occurrences in the sentence
about bypass behavior and the example flag text match
`include_non_operational=true` / `include_non_operational=false` and keep the
identifier list as-is) to maintain a single consistent parameter name across the
document.
---
Nitpick comments:
In
`@system-int-test/test/cmr/system_int_test/search/collection/collection_progress_search_test.clj`:
- Around line 79-100: The test toggles the global flag via (side/eval-form
`(cmr.search.config/set-enable-non-operational-collection-filter! true)) and
then resets it to false, but if an exception occurs the reset is skipped; change
both test blocks that call set-enable-non-operational-collection-filter! to wrap
the body in a try/finally so the final (side/eval-form
`(cmr.search.config/set-enable-non-operational-collection-filter! false)) runs
in the finally clause; locate the occurrences surrounding the testing "flag ON -
filter applies unless overridden" block and the second test block (the one
starting around line 132) and move their test logic into try with the reset call
placed in finally.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 10536bbf-467c-46d4-8779-4c8457990cd8
📒 Files selected for processing (2)
search-app/docs/api.mdsystem-int-test/test/cmr/system_int_test/search/collection/collection_progress_search_test.clj
| Valid values: ACTIVE, PLANNED, COMPLETE, DEPRECATED, NOT PROVIDED, PREPRINT, INREVIEW, SUPERSEDED | ||
|
|
||
| **NOTE:** When the non-operational collection filter is enabled (feature flag), collections with PLANNED, DEPRECATED, PREPRINT, and INREVIEW status are excluded from default search results. Use `include-non-operational=true` to include them. This is an experimental feature and may not be enabled in all environments. | ||
| **NOTE:** When the non-operational collection filter is enabled (feature flag), collections with PLANNED, DEPRECATED, PREPRINT, and INREVIEW status are excluded from default search results. Use `include-non-operational=true` to include them. The default filter is automatically bypassed when searching by collection identifiers (`concept_id`, `entry_id`, `entry_title`, `short_name` with `version`, or `native_id`), allowing you to find specific collections regardless of their progress status. However, if you explicitly set `include-non-operational=false` (or `true`), that explicit setting takes precedence even when using identifier parameters. |
There was a problem hiding this comment.
Use one query parameter spelling for include_non_operational in this note.
Line 1944 uses include-non-operational, but the heading and examples below use include_non_operational. Keeping one style here avoids client confusion.
✏️ Proposed doc fix
-**NOTE:** When the non-operational collection filter is enabled (feature flag), collections with PLANNED, DEPRECATED, PREPRINT, and INREVIEW status are excluded from default search results. Use `include-non-operational=true` to include them. The default filter is automatically bypassed when searching by collection identifiers (`concept_id`, `entry_id`, `entry_title`, `short_name` with `version`, or `native_id`), allowing you to find specific collections regardless of their progress status. However, if you explicitly set `include-non-operational=false` (or `true`), that explicit setting takes precedence even when using identifier parameters.
+**NOTE:** When the non-operational collection filter is enabled (feature flag), collections with PLANNED, DEPRECATED, PREPRINT, and INREVIEW status are excluded from default search results. Use `include_non_operational=true` to include them. The default filter is automatically bypassed when searching by collection identifiers (`concept_id`, `entry_id`, `entry_title`, `short_name` with `version`, or `native_id`), allowing you to find specific collections regardless of their progress status. However, if you explicitly set `include_non_operational=false` (or `true`), that explicit setting takes precedence even when using identifier parameters.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@search-app/docs/api.md` at line 1944, The note is using a different query
parameter spelling (`include-non-operational`) than the rest of the doc which
uses `include_non_operational`; change the hyphenated form to the underscored
form so the note uses `include_non_operational` (ensure occurrences in the
sentence about bypass behavior and the example flag text match
`include_non_operational=true` / `include_non_operational=false` and keep the
identifier list as-is) to maintain a single consistent parameter name across the
document.
jceaser
approved these changes
Mar 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
What is the objective?
Bypass the non-operational collection filter when searching by collection identifiers (concept-id, entry-id, entry-title, short-name+version, native-id) to fix broken client workflows.
What are the changes?
What areas of the application does this impact?
Required Checklist
Additional Checklist
Summary by CodeRabbit
Documentation
Improvements