Skip to content

Update Python dependency versions identified by dependabot#1481

Merged
isabeleliassen merged 8 commits intocsg-org:mainfrom
InspiringApps:chore/update-dependencies
Apr 21, 2026
Merged

Update Python dependency versions identified by dependabot#1481
isabeleliassen merged 8 commits intocsg-org:mainfrom
InspiringApps:chore/update-dependencies

Conversation

@landonshumway-ia
Copy link
Copy Markdown
Collaborator

@landonshumway-ia landonshumway-ia commented Apr 17, 2026

In an effort to close out the list of Dependabot generated PRs, this introduces python dependency updates that dependabot has identified.

This includes a list of minor updates, as well as the following major updates:
Marshmallow 3.x -> 4.x see changelog for description of breaking changes

The biggest change for this was replacing the DateTime marshmallow field type with the AwareDateTime type. The DateTime marshmallow type, as of 4.x, is much more permissive and will allow date strings (YYYY-MM-DD) to silently pass validation and will set it as a naive datetime string, which we want to avoid as our datetimes need to be timezone aware to support converting to various local timezones. Scanning through our identified writes to the current datetime fields showed that we already enforce timezone awareness when storing datetime fields, and we migrated off of naive timestamps before we even launched in prod, so it is expected that this is a backwards compatible change with our production environment. Even so, extensive smoke testing, UI testing, and zap scanning should be performed against this change before releasing to prod (unfortunately, given that this is a core schema related update, the feature flag framework will not be feasible to use at this fundamental scope without impacting system performance).

We were not able to update the attrs dependency to the latest 26.x version, as this is a transitive dependency of jsii (used by CDK). Until jsii ships a new version that accepts attrs 26, the CDK app cannot move to attrs 26, so the shared venv can't either. For now it will remain as is and we will have Dependabot ignore that update. The only other location we use it is as a dev dependency for moto in the common tests, so it is not bundled in the runtime code.

Closes #1319

Summary by CodeRabbit

Release Notes

  • Bug Fixes

    • Enhanced datetime field validation to require timezone-aware timestamps across multiple data schemas, improving consistency and preventing timezone-related data validation errors.
  • Chores

    • Updated core dependencies including AWS SDK tools, Python libraries, and development tools to their latest stable versions for improved security and performance.

Introduces major updates:
Marshmallow 3.x -> 4.x
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 17, 2026

Warning

Rate limit exceeded

@landonshumway-ia has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 54 minutes and 48 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 54 minutes and 48 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4cd44053-2ec0-46b3-810d-0f3415501404

📥 Commits

Reviewing files that changed from the base of the PR and between 1c9a7ab and 69d6aa0.

📒 Files selected for processing (13)
  • backend/compact-connect/lambdas/python/cognito-backup/requirements-dev.txt
  • backend/compact-connect/lambdas/python/common/requirements-dev.txt
  • backend/compact-connect/lambdas/python/common/requirements.txt
  • backend/compact-connect/lambdas/python/compact-configuration/requirements-dev.txt
  • backend/compact-connect/lambdas/python/custom-resources/requirements-dev.txt
  • backend/compact-connect/lambdas/python/data-events/requirements-dev.txt
  • backend/compact-connect/lambdas/python/disaster-recovery/requirements-dev.txt
  • backend/compact-connect/lambdas/python/provider-data-v1/requirements-dev.txt
  • backend/compact-connect/lambdas/python/search/requirements-dev.txt
  • backend/compact-connect/lambdas/python/staff-user-pre-token/requirements-dev.txt
  • backend/compact-connect/lambdas/python/staff-users/requirements-dev.txt
  • backend/compact-connect/requirements-dev.txt
  • backend/compact-connect/requirements.txt
📝 Walkthrough

Walkthrough

Updated multiple Marshmallow schema datetime fields from DateTime to AwareDateTime (enforcing timezone-aware datetimes) across compact-connect and cosmetology-app; bumped marshmallow constraints to allow 4.x; and applied widespread dependency version bumps (boto3/botocore, aws-lambda-powertools, packaging, CDK pins, dev tools). Tests updated to use +00:00 timestamps.

Changes

Cohort / File(s) Summary
Schema: compact-connect datetime conversions
backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/adverse_action/record.py, .../base_record.py, .../data_event/api.py, .../investigation/record.py, .../license/ingest.py, .../license/record.py, .../military_affiliation/record.py, .../privilege/record.py, .../provider/api.py, .../provider/record.py
Replaced marshmallow.fields.DateTime with marshmallow.fields.AwareDateTime for multiple timestamp fields (required/optional flags unchanged).
Schema: cosmetology-app datetime conversions
backend/cosmetology-app/lambdas/python/common/cc_common/data_model/schema/.../record.py, .../data_event/api.py, .../license/ingest.py, .../license/record.py, .../investigation/record.py, .../provider/record.py
Same DateTime → AwareDateTime conversions mirrored in cosmetology-app schema files; imports adjusted accordingly.
Requirements (marshmallow constraint changes)
backend/compact-connect/lambdas/python/common/requirements.in, backend/cosmetology-app/lambdas/python/common/requirements.in, backend/compact-connect/lambdas/python/purchases/requirements-dev.in
Bumped marshmallow constraint from <4.0.0 to <5.0.0 and raised minimum to 4.3.0 in affected requirement files.
Pinned dependency bumps (boto3/botocore, powertools, packaging, marshmallow, dev tools)
**/.../requirements*.txt (many: backend/compact-connect/..., backend/cosmetology-app/..., backend/multi-account/..., backend/.../requirements-dev.txt, etc.)
Numerous requirement files updated: boto3/botocore bumped (1.42.89→1.42.90/1.42.91 where noted), aws-lambda-powertools 3.27.0→3.28.0, packaging 26.0→26.1, marshmallow pins updated to 4.3.0, dev tool pins (filelock, ruff) updated.
AWS CDK pins
backend/compact-connect/requirements.txt, backend/cosmetology-app/requirements.txt, backend/multi-account/.../requirements.txt
Bumped CDK-related packages (e.g., aws-cdk-lib 2.249.0→2.250.0, aws-cdk-cloud-assembly-schema versions updated).
Test fixtures
backend/compact-connect/lambdas/python/search/tests/function/test_expiration_reminders.py
Test OpenSearch fixture timestamps updated to ISO8601 with explicit UTC offset (+00:00) for dateOfUpdate.
Other small constraint edits
backend/cosmetology-app/lambdas/python/common/requirements-dev.in
Removed duplicate attrs>=25, <26, retaining attrs>=25.4,<26.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • jlkravitz

Poem

🐇 I hopped through schemas, neat and spry,
Changing datetimes to know the sky,
Marshmallow grew a major four,
Timestamps now whisper "+00:00" more,
A tiny rabbit cheers — time zones, hi! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change—updating Python dependencies identified by dependabot—which is the core focus of the PR.
Description check ✅ Passed The description comprehensively covers the changes, rationale for Marshmallow migration, impact assessment, and known limitations. While it deviates from the template format, it provides substantial detail necessary for understanding the changes.
Linked Issues check ✅ Passed The PR closes issue #1319 (Q2 2026 - Update backend project dependencies) which aims to update backend dependencies and verify continued correct operation. The PR delivers multiple dependency updates and extensive validation of schema changes.
Out of Scope Changes check ✅ Passed All changes are within scope: dependency version updates in requirements files and schema field type changes (DateTime→AwareDateTime) directly necessitated by Marshmallow 4.x upgrade.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Now that we are moving from Marshmallow 3.x -> 4.x, the datetime fields are
more permissive, and will convert a bare date string YYYY-MM-DD into a naive
datetime format, which we do not want. We need to enforce that these fields are
timezone aware to avoid any future bugs/issues inserting naive timestamps.
@landonshumway-ia landonshumway-ia marked this pull request as ready for review April 17, 2026 18:58
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/privilege/record.py (1)

135-142: ⚠️ Potential issue | 🟠 Major

_enforce_datetimes no longer provides the backwards-compat guarantee it claims under Marshmallow 4.

ensure_value_is_datetime (in common.py) only upgrades length-10 date strings (YYYY-MM-DD) to UTC-aware datetime strings — if the input is already a datetime-shaped string, it returns it unchanged. That was fine when these fields were DateTime (tolerant of naive ISO strings), but now with AwareDateTime any historical record whose dateOfIssuance/dateOfRenewal was written as a naive datetime string (e.g. 2024-01-02T03:04:05) will bypass this hook's "fix up" and then fail AwareDateTime validation on load.

Given this is the schema that most explicitly advertises legacy-data tolerance, please either:

  1. Extend ensure_value_is_datetime to also coerce naive datetime strings to UTC-aware (parse, set tzinfo=UTC if missing, re-serialize), or
  2. Confirm via a migration/backfill that no naive datetime values remain for these fields in production.

Option 1 is minimally invasive and preserves the safety net already described in the docstring. It also transparently fixes the same risk in other schemas that call into ensure_value_is_datetime.

♻️ Sketch of the extended helper
 def ensure_value_is_datetime(value: str):
     ...
     dt = datetime.fromisoformat(value)
     if len(value) == 10:
         value_dt = datetime.combine(dt, datetime.max.time(), tzinfo=UTC).replace(microsecond=0)
         return value_dt.isoformat()
-    # Not a date string, return the original
-    return value
+    # Promote naive datetime strings to UTC-aware for Marshmallow 4 AwareDateTime compatibility
+    if dt.tzinfo is None:
+        return dt.replace(tzinfo=UTC).isoformat()
+    return value

Also note the updated comment on line 138 now reads "for dateOfRenewal and dateOfIssuance to datetime values" but the real intent after this PR is "to timezone-aware datetime values" — worth tightening.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/privilege/record.py`
around lines 135 - 142, The _enforce_datetimes backward-compat guarantee breaks
under Marshmallow 4 because ensure_value_is_datetime currently only upgrades
YYYY-MM-DD strings and leaves naive ISO datetime strings unchanged, causing
AwareDateTime validation failures; update ensure_value_is_datetime (used by
_enforce_datetimes) to parse ISO datetime strings and, if they lack timezone
info, attach UTC (e.g., parse -> set tzinfo=UTC -> re-serialize/return
timezone-aware datetime) so dateOfIssuance and dateOfRenewal are always
timezone-aware before AwareDateTime validation, and update the doc/comment in
_enforce_datetimes to say "timezone-aware datetime values"; ensure function
names referenced: ensure_value_is_datetime, _enforce_datetimes, and fields
dateOfIssuance/dateOfRenewal/AwareDateTime.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In
`@backend/cosmetology-app/lambdas/python/common/cc_common/data_model/schema/license/ingest.py`:
- Line 3: The AwareDateTime field used for eventTime in
SanitizedLicenseIngestDataEventSchema may reject timezone-naive timestamps;
update the schema's eventTime AwareDateTime declaration to set
default_timezone=timezone.utc so Marshmallow will treat naive datetimes as UTC
when loading (import timezone from datetime as needed) and ensure existing
producers' offset-bearing strings continue to validate unchanged.

In `@backend/cosmetology-app/lambdas/python/common/requirements.in`:
- Line 5: Replace the existing dependency line "marshmallow>=3.21.3, <5.0.0"
with a constraint that excludes the vulnerable 4.0.0–4.1.1 releases; either use
the two-platform-specific lines "marshmallow>=3.21.3, <4.0.0; python_version <
'3.x'" and "marshmallow>=4.1.2, <5.0.0; python_version >= '3.x'" or a single
unified exclusion "marshmallow>=3.21.3, <5.0.0, !=4.0.0, !=4.0.1, !=4.1.0,
!=4.1.1" in place of the original "marshmallow>=3.21.3, <5.0.0" entry so the
vulnerable CVE-2025-68480 versions are not installable.

---

Outside diff comments:
In
`@backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/privilege/record.py`:
- Around line 135-142: The _enforce_datetimes backward-compat guarantee breaks
under Marshmallow 4 because ensure_value_is_datetime currently only upgrades
YYYY-MM-DD strings and leaves naive ISO datetime strings unchanged, causing
AwareDateTime validation failures; update ensure_value_is_datetime (used by
_enforce_datetimes) to parse ISO datetime strings and, if they lack timezone
info, attach UTC (e.g., parse -> set tzinfo=UTC -> re-serialize/return
timezone-aware datetime) so dateOfIssuance and dateOfRenewal are always
timezone-aware before AwareDateTime validation, and update the doc/comment in
_enforce_datetimes to say "timezone-aware datetime values"; ensure function
names referenced: ensure_value_is_datetime, _enforce_datetimes, and fields
dateOfIssuance/dateOfRenewal/AwareDateTime.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4f61e516-79f0-49f5-8094-691d74958777

📥 Commits

Reviewing files that changed from the base of the PR and between 0c2b88c and 2ccfec5.

📒 Files selected for processing (48)
  • backend/compact-connect/lambdas/python/cognito-backup/requirements-dev.txt
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/adverse_action/record.py
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/base_record.py
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/data_event/api.py
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/investigation/record.py
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/license/ingest.py
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/license/record.py
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/military_affiliation/record.py
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/privilege/record.py
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/provider/api.py
  • backend/compact-connect/lambdas/python/common/cc_common/data_model/schema/provider/record.py
  • backend/compact-connect/lambdas/python/common/requirements-dev.txt
  • backend/compact-connect/lambdas/python/common/requirements.in
  • backend/compact-connect/lambdas/python/common/requirements.txt
  • backend/compact-connect/lambdas/python/compact-configuration/requirements-dev.txt
  • backend/compact-connect/lambdas/python/custom-resources/requirements-dev.txt
  • backend/compact-connect/lambdas/python/data-events/requirements-dev.txt
  • backend/compact-connect/lambdas/python/disaster-recovery/requirements-dev.txt
  • backend/compact-connect/lambdas/python/provider-data-v1/requirements-dev.txt
  • backend/compact-connect/lambdas/python/search/requirements-dev.txt
  • backend/compact-connect/lambdas/python/search/tests/function/test_expiration_reminders.py
  • backend/compact-connect/lambdas/python/staff-user-pre-token/requirements-dev.txt
  • backend/compact-connect/lambdas/python/staff-users/requirements-dev.txt
  • backend/compact-connect/requirements-dev.txt
  • backend/compact-connect/requirements.txt
  • backend/cosmetology-app/lambdas/python/cognito-backup/requirements-dev.txt
  • backend/cosmetology-app/lambdas/python/cognito-backup/requirements.txt
  • backend/cosmetology-app/lambdas/python/common/cc_common/data_model/schema/adverse_action/record.py
  • backend/cosmetology-app/lambdas/python/common/cc_common/data_model/schema/base_record.py
  • backend/cosmetology-app/lambdas/python/common/cc_common/data_model/schema/data_event/api.py
  • backend/cosmetology-app/lambdas/python/common/cc_common/data_model/schema/investigation/record.py
  • backend/cosmetology-app/lambdas/python/common/cc_common/data_model/schema/license/ingest.py
  • backend/cosmetology-app/lambdas/python/common/cc_common/data_model/schema/license/record.py
  • backend/cosmetology-app/lambdas/python/common/cc_common/data_model/schema/provider/record.py
  • backend/cosmetology-app/lambdas/python/common/requirements-dev.in
  • backend/cosmetology-app/lambdas/python/common/requirements-dev.txt
  • backend/cosmetology-app/lambdas/python/common/requirements.in
  • backend/cosmetology-app/lambdas/python/common/requirements.txt
  • backend/cosmetology-app/lambdas/python/compact-configuration/requirements-dev.txt
  • backend/cosmetology-app/lambdas/python/custom-resources/requirements-dev.txt
  • backend/cosmetology-app/lambdas/python/data-events/requirements-dev.txt
  • backend/cosmetology-app/lambdas/python/disaster-recovery/requirements-dev.txt
  • backend/cosmetology-app/lambdas/python/provider-data-v1/requirements-dev.txt
  • backend/cosmetology-app/lambdas/python/search/requirements-dev.txt
  • backend/cosmetology-app/lambdas/python/staff-user-pre-token/requirements-dev.txt
  • backend/cosmetology-app/lambdas/python/staff-users/requirements-dev.txt
  • backend/cosmetology-app/requirements-dev.txt
  • backend/cosmetology-app/requirements.txt
💤 Files with no reviewable changes (1)
  • backend/cosmetology-app/lambdas/python/common/requirements-dev.in

Comment thread backend/cosmetology-app/lambdas/python/common/requirements.in Outdated
@landonshumway-ia
Copy link
Copy Markdown
Collaborator Author

@jlkravitz This is ready for your review. I have deployed this to IA's test environment and ran smoke testing against it to verify the privilege purchase flow and staff user features still work with the marshmallow schema updates. Thanks

Copy link
Copy Markdown
Collaborator

@jlkravitz jlkravitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@isabeleliassen This is good to merge. I'm not merging myself since this is a major version update for a dependency of the project (not a patch or minor update), per our discussion.

@isabeleliassen isabeleliassen merged commit 423e6f4 into csg-org:main Apr 21, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Q2 2026 - Update backend project dependencies

3 participants