fix(spp_studio): remove or rewrite pack items with non-evaluable CEL expressions

[gonzalesedwin1123]

Fixes #431.

Problem

24 of the 106 spp.studio.pack.item records shipped in spp_studio/data/packs/ carry filter
expressions that cannot work on any install: they reference registrant fields or studio variables
that exist in no module, so the logic they install can never translate or evaluate. They also fail
spp_cel_load_testing's strict studio-validation test
(test_all_pack_cel_expressions_translate), which gates the merge of #432 and would fail the
weekly ci-full run once that migration lands.

Approach

Per-item rewrite where a real near-equivalent survives, remove the rest — no placeholder
variables were invented, and no test was weakened. Every candidate rewrite was validated
empirically through the exact resolve+translate path the test uses (odoo shell against an
installed DB), because translation stops at the first bad identifier — terms after it in the
original expressions had never actually been validated.

Fixed properly (1)

Exclusion Criteria · Institutional Residence Exclusion keeps its shipped expression
(!in_institutional_care): the item only lacked a variable definition — core spp_registry
already defines the in_institutional_care field on individuals. This PR adds the missing
in_institutional_care standard variable over that field. standard_variables.xml is not
noupdate, so existing databases get the variable through the regular upgrade data load.

Rewritten (4)

Item	Old expression	New expression
OVC · Child Age Eligibility	age < child_age_limit || (age < 25 && is_in_education)	age < child_age_limit
OVC · Vulnerable Household Check	is_elderly_headed || is_child_headed || is_skipped_generation || has_chronically_ill_head	is_elderly_headed || is_child_headed
GMI · Residency Requirement	is_legal_resident && residency_months >= 12	residency_months >= 12
Public Works · Poverty Status Check	per_capita_income <= poverty_threshold || is_food_insecure	per_capita_income <= poverty_threshold

Descriptions updated where the semantics narrowed.

Removed (19)

No meaningful sub-expression survives — every load-bearing identifier is backed by nothing:

• Disability Assistance · Disability Status Verification
• OVC Support · Orphan Status Check (the !has_mother && !has_father fallback expands to
  members.*, which doesn't exist in individual context — no rescue)
• OVC Support · School Enrollment Requirement
• Social Pension · No Formal Pension Check
• Guaranteed Minimum Income · Employment Barriers Check (3 of its 4 criteria are broken —
  is_single_parent also fails validation; the single surviving term isn't a near-equivalent)
• Public Works · Physical Work Capability
• Public Works · Seasonal Availability
• Conditional Cash Transfer · Health Conditionality Compliance
• Geographic Targeting · Limited Service Access
• Exclusion Criteria · Government Employee, Formal Sector, Vehicle Ownership, Business Ownership,
  Housing Quality, Pension Receipt, Duplicate Program, Income Tax Payer, Bank Balance,
  Livestock Ownership (10 items; the pack keeps Land Ownership and Institutional Residence)

Migration

Pack data is noupdate="1", so existing databases keep the broken records after the XML change.
migrations/19.0.2.0.2/post-migration.py applies the same 19 removals and 4 rewrites directly
(the restored Institutional Residence item needs no migration handling — only its variable, which
arrives via the regular data load).
Version 19.0.2.0.1 → 19.0.2.0.2 + HISTORY fragment.

Verification

All local (Docker), with spp_cel_load_testing taken from PR #432 @ 419752d6 since the strict test lives there:

1. Baseline (red) — fresh install with original data: test_all_pack_cel_expressions_translate fails with exactly the 24 items from spp_studio: shipped logic packs contain filter expressions referencing nonexistent fields/variables (24 items) #431 (14 translation errors + 10 unresolved-variable items).
2. Fresh install with this fix (green) — 0 failed, 0 error(s) of 6 tests; Translated 30/25 CEL expressions (0 errors, 0 with unresolved variables).
3. Upgrade path (green) — DB installed with the ORIGINAL pack data at 19.0.2.0.1 (pristine origin/19.0), then upgraded with this branch: migration logs removed 19 and rewrote 4 shipped pack items, item count 106 → 87, the in_institutional_care variable is created by the data load, rewritten/restored expressions verified in the DB, and the strict suite passes on the upgraded DB (0 failed of 6; Translated 32/26, 0 errors, 0 unresolved).
4. spp_studio own suite — Passed: 225 | Failed: 0 | Errors: 0.
5. Lint — ruff / ruff-format / prettier all pass on the changed files. README.rst deliberately not regenerated locally (CI's pinned generator is the authority; its diff will be applied verbatim).

Out of scope / follow-ups

• Several standard variables remain defined against nonexistent fields (is_orphan,
  owns_car, land_hectares, residency_years, is_single_parent, …). They are the same
  defect one level down but are not exercised by the strict test unless a pack item references
  them; needs its own issue.
• Re-home candidates: two removed concepts have real data in optional modules —
  Disability Status Verification maps cleanly onto spp_disability_registry
  (has_disability, disability_severity_id, assessments), and livestock/land exclusions onto
  spp_farmer_registry/spp_land_record. Since pack items are plain data records, those modules
  can ship their own items into the shipped packs, defined against their own fields — the right
  long-term home for domain templates. Needs its own issue.
• The CEL translator validates boolean field references against the model but lets numeric
  comparisons through unvalidated (r.land_hectares <= 2 translates even though the field exists
  nowhere) — latent quirk worth a look.
• required_modules support for "template" packs (issue spp_studio: shipped logic packs contain filter expressions referencing nonexistent fields/variables (24 items) #431 Option 3) if aspirational packs are
  wanted as a product feature.

Reviewer caveat

Neither this PR's CI nor #432's exercises the strict test (per-module jobs don't co-install
spp_studio and spp_cel_load_testing). Only a manual ci-full run after both merge does.
The local runs documented above are the substitute evidence.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.81%. Comparing base (3939520) to head (6751987).
⚠️ Report is 4 commits behind head on 19.0.

Additional details and impacted files

@@            Coverage Diff             @@
##             19.0     #444      +/-   ##
==========================================
+ Coverage   74.76%   74.81%   +0.05%     
==========================================
  Files         509      565      +56     
  Lines       35008    39497    +4489     
==========================================
+ Hits        26172    29551    +3379     
- Misses       8836     9946    +1110     

Flag	Coverage Δ
spp_base_common	91.07% <ø> (ø)
spp_cel_event	85.34% <ø> (?)
spp_dci_demo	94.28% <ø> (ø)
spp_dci_indicators	96.23% <ø> (ø)
spp_farmer_registry	88.58% <ø> (?)
spp_farmer_registry_cr	61.24% <ø> (ø)
spp_farmer_registry_demo	63.43% <ø> (ø)
spp_mis_demo_v2	70.38% <ø> (ø)
spp_programs	65.53% <ø> (+0.25%)	⬆️
spp_registry	87.22% <ø> (+0.07%)	⬆️
spp_security	69.56% <ø> (ø)
spp_starter_sp_mis	88.88% <ø> (ø)
spp_studio	62.92% <ø> (?)
spp_studio_api_v2	95.35% <ø> (?)
spp_studio_change_requests	84.85% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 61 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[emjay0921]

One change requested. The mechanics of this PR are clean and I verified them item by item — notes in a separate comment, along with a root-cause finding that is bigger than this PR.

The GMI rewrite is the only fail-open change in the set, and it reads like the others

Three of the four rewrites drop OR-branches, which makes the predicate stricter — fewer registrants qualify. That is the safe direction for a shipped eligibility template. The GMI one drops an AND-conjunct:

is_legal_resident && residency_months >= 12   →   residency_months >= 12

That makes the rule more permissive: it now passes registrants the policy author wrote it to exclude. The new description — "Minimum residency duration requirement (12 months)." — drops any mention of legal residency, so nothing downstream records that the requirement was there and was removed. The PR body lists it in the same table as the three narrowing rewrites, and the note that "descriptions updated where the semantics narrowed" does not cover this case.

I checked whether the conjunct could be rescued the way Institutional Residence was: it cannot. There is no is_legal_resident field on any model (ir_model_fields on a dev DB, and a repo-wide field index), so the two consistent options are the same ones applied to the other 19 items:

1. Remove the item, like the 19 whose load-bearing terms have nothing behind them. This is the option I would take — a template that silently grants eligibility is worse than no template.
2. Keep item and expression as shipped, accepting that it does not evaluate until the field exists, and track it under the variable-catalogue issue (#446).

If there is a reason to prefer the narrowed-but-permissive form, that is a policy call rather than a data-hygiene one, and it should be stated explicitly in the PR body and the HISTORY entry so it is not read as an equivalent rewrite.

Everything else I found is non-blocking.

[emjay0921]

Non-blocking findings, plus the verification I did on the mechanics.

The root cause survives this fix — filed as #446

The 19 removed pack items were consumers of the defect, not the defect. spp.cel.variable records for is_legal_resident, is_food_insecure, is_in_education, is_skipped_generation, has_chronically_ill_head and is_single_parent all exist in standard_variables.xml; what is missing is the fields they point at. Across the catalogue, 25 of the 27 source_type="field" variables have a source_field that exists on no model — verified against a repo-wide index of 2,837 field definitions and then against ir_model_fields on a dev database. Details and the full list are in #446.

The practical consequence is that Studio's variable picker still offers these to no-code users, so the first filter anyone builds on one reproduces #431 from the UI. Worth keeping in mind when #431 is closed: the pack items are fixed, the class is not.

It also means the framing in the PR body is inverted for these six. It says the items reference "studio variables that exist in no module" and that "every load-bearing identifier is backed by nothing" — the variables exist, their fields do not. That is worth being precise about because it is the exact mirror of the in_institutional_care case this PR handles correctly (field existed, variable was missing, so adding the variable fixed it). The inverse case has a different set of fixes, which is what #446 is for.

Removed items leave their installed artifacts behind

spp.studio.pack.item.installed_logic_id is a Many2one to spp.cel.expression, so unlink() on a pack item does not touch logic already installed from it. On a database where one of the 19 packs was installed, the broken spp.cel.expression record survives this migration — and #432's TestStudioLogicValidation validates exactly those records, not just pack items. Worth deciding whether the migration should follow installed_logic_id for the removed set, or whether installed artifacts are considered the deployment's own data at that point.

The migration overwrites local edits without checking

REWRITTEN_ITEMS is applied with json.dumps({"cel_expression": expression}), which replaces the whole logic_data blob. Pack data is noupdate="1" precisely so local changes survive upgrades, so an admin who already repaired one of these four locally — plausible, since a deployment could have defined the missing variables itself — loses that silently, and any other keys in logic_data go with it. Guarding the write on "current expression still matches the known-broken shipped value" would make it safe to run anywhere.

Minor

The new var_in_institutional_care uses applies_to="both". Among the demographic-category variables, 17 use individual and 1 uses both, and the field is defined on individuals in spp_registry — so individual looks like the right value, and both offers it in group context where it cannot mean anything.

Verified

• The XML removals and the migration's REMOVED_ITEMS match exactly — 19 and 19, no drift in either direction. (My first extraction counted 18 and I nearly reported a mismatch; that was my regex missing prettier's multi-line <record> formatting, not a defect here.)
• All four rewritten expressions are identical between the pack XML and the migration, so fresh installs and upgraded databases converge on the same state.
• The migration guards if not version: return so it is skipped on fresh installs, uses env.ref(..., raise_if_not_found=False) throughout, and is idempotent on re-run.
• in_institutional_care genuinely exists at spp_registry/models/individual.py:40 and resolves on res.partner in an installed schema, and standard_variables.xml is not noupdate, so the claim that existing databases pick up the new variable through the regular data load holds.
• Version 19.0.2.0.1 → 19.0.2.0.2 matches the migration directory; the HISTORY entry names every removed item rather than summarising, which is the right call for a data-removal change.
• 1 + 4 + 19 = 24, reconciling with spp_studio: shipped logic packs contain filter expressions referencing nonexistent fields/variables (24 items) #431's inventory.

One carry-over

Neither this PR's CI nor #432's runs the assert that would actually validate this fix — test_all_pack_cel_expressions_translate only executes when spp_studio and spp_cel_load_testing are installed together, which no per-module job does. A manual ci-full run once both land is still the verification step.