Allow pushing user-allocation membership to Keycloak by QuanMPhm · Pull Request #249 · nerc-project/coldfront-plugin-cloud

QuanMPhm · 2025-10-03T20:29:45Z

Closes nerc-project/operations#948. More details in the commit message
There are still some questions I have below, so this is still a draft for now.

QuanMPhm · 2025-10-06T17:42:21Z

@knikolla Two more questions:

Do we also want validate_allocations to add PIs to Keycloak groups for pre-existing allocations?
When a PI adds a user to an Coldfront project or allocation, do those users also get added to a the project's Keycloak group?

QuanMPhm · 2025-10-30T19:07:52Z

@knikolla I've addressed your comments except one. Also, do you have responses to these questions?

knikolla · 2026-03-18T13:22:44Z

@QuanMPhm please resolve conflicts. Are there any questions that I missed answering?

QuanMPhm · 2026-03-20T12:18:01Z

@knikolla Just this one. I will resolve the conflicts while waiting for your answer

knikolla · 2026-03-20T14:49:39Z

@knikolla Just this one. I will resolve the conflicts while waiting for your answer

Responded.

knikolla

Did a quick first pass and provided some comments.

Also this needs to be possible configurable via a setting.

knikolla · 2026-03-23T19:48:36Z

@QuanMPhm Actually, another thought, do you think it would make sense to implement this in the Keycloak plugin? https://github.com/nerc-project/coldfront-plugin-keycloak

It could listen to signals in the same way that the cloud plugin listens to signals. It already has a keycloak client implemented.

And there is nothing in pushing users to a Keycloak group that is specific to either OpenShift or OpenStack.

QuanMPhm · 2026-03-24T16:10:47Z

@knikolla I see that it does make sense to seperate the Keycloak functionality from the rest of the plugin. It makes sense to me. I forgot that repo existed. There would need to be some overhaul to add integration and unit tests to coldfront-plugin-keycloak. Is that fine?

knikolla · 2026-03-25T13:43:53Z

@knikolla I see that it does make sense to seperate the Keycloak functionality from the rest of the plugin. It makes sense to me. I forgot that repo existed. There would need to be some overhaul to add integration and unit tests to coldfront-plugin-keycloak. Is that fine?

For now let's keep it here (as not to frontload the work) and we can easily split it out later if needed. Perhaps try implementing it here via signals so as to keep it loosely coupled so that if we need to split it later it doesn't require a lot of uncoupling.

QuanMPhm · 2026-04-06T22:23:28Z

@knikolla @larsks I have a question about the Coldfront Keycloak PR. I know we previously decided to represent user-membership to an allocation by adding them to a Keycloak group with the same name as the ALLOCATION_PROJECT_ID . Would it make more sense for the group name to be the allocation's ID on Coldfront instead? This has a stronger guarantee of uniqueness, and we won't have to disambiguate for projects with the same name in different clusters. Developers or tools can correlate the allocation id to the on-cluster project id by using the Coldfront API.

knikolla

Looking much better. Almost there.

A follow-up to this (after we merge the validate_allocations refactor) should investigate how to cleanly implement this in validate_allocations too.

knikolla · 2026-04-07T21:04:06Z

Looking much better. Almost there.

A follow-up to this (after we merge the validate_allocations refactor) should investigate how to cleanly implement this in validate_allocations too.

Actually... I would make it a whole separate CLI command. validate_keycloak_groups.

QuanMPhm · 2026-04-08T02:37:21Z

@knikolla I've allowed the group name template string to accept any allocation attribute. Further documentation is in the docstring for the function _get_keycloak_group_name

knikolla · 2026-04-08T17:55:33Z

@naved001 would appreciate a pass from you.

knikolla

Had some more time to think about how to expose enabling/disable the feature.

Copilot

Pull request overview

Adds Keycloak integration to push ColdFront allocation membership into Keycloak group membership, driven by a resource-level group-name template and exercised via new functional tests/CI workflow.

Changes:

Add Keycloak client + tasks to add/remove allocation users to/from Keycloak groups based on a configurable template.
Wire Keycloak add/remove behavior into allocation-user signals when Keycloak is enabled via environment.
Introduce Keycloak functional tests and CI scripts/workflow to run them.

Reviewed changes

Copilot reviewed 9 out of 11 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
src/coldfront_plugin_cloud/tests/functional/keycloak/test_keycloak.py	Adds functional tests validating Keycloak user/group membership behavior for allocation changes.
src/coldfront_plugin_cloud/tests/functional/keycloak/init.py	Introduces package for Keycloak functional tests.
src/coldfront_plugin_cloud/tasks.py	Adds Keycloak client caching and add/remove group membership tasks; adds group-name templating helper.
src/coldfront_plugin_cloud/signals.py	Hooks Keycloak add/remove tasks into allocation user add/remove signals behind an env flag.
src/coldfront_plugin_cloud/management/commands/validate_allocations.py	Tweaks logging/comment wording related to user sync validation.
src/coldfront_plugin_cloud/kc_client.py	Adds a Keycloak admin API client for groups/users operations.
src/coldfront_plugin_cloud/attributes.py	Adds resource attribute for Keycloak group name template.
requirements.txt	Adds `requests` dependency for the Keycloak client.
ci/setup-keycloak.sh	Adds Keycloak container bootstrap script for CI/local functional tests.
ci/run_functional_tests_keycloak.sh	Adds a runner script to execute Keycloak functional tests with required env vars.
.github/workflows/test-functional-keycloak.yaml	Adds a GitHub Actions workflow to run Keycloak functional tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

QuanMPhm · 2026-04-13T12:55:27Z

@knikolla @naved001 I have responded to all comments

A Keycloak admin client has been added When `activate_allocation` is called, the user is added to a Keycloak group named using a format string defined in the allocation's resource attribute "Format String for Keystone Group Names" If the user does not already exist in Keycloak, the case is ignored for now Keycloak integration is optional, toggled by setting the env var "KEYCLOAK_BASE_URL" Authentication to Keycloak is done via client credentials grant When `deactivate_allocation` is called, the user is removed from the Keycloak group New functional test added for Keycloak integration A comment in `validate_allocations` has been updated to reflect the more restrictive validation behavior, where users on cluster projects will be removed if they are not part of the Coldfront allocation (rather than if they are not registered on Coldfront at all).

QuanMPhm · 2026-04-22T18:57:58Z

Addressed all comments, except one waiting for @knikolla response

QuanMPhm requested review from knikolla, larsks and naved001 October 3, 2025 20:29

QuanMPhm commented Oct 3, 2025

View reviewed changes

Comment thread src/coldfront_plugin_cloud/kc_client.py Outdated

QuanMPhm commented Oct 3, 2025

View reviewed changes

Comment thread src/coldfront_plugin_cloud/tests/functional/openshift/test_allocation.py Outdated

QuanMPhm force-pushed the ops_948/auth_kc branch from 9a53156 to d7da5c4 Compare October 5, 2025 01:32

QuanMPhm marked this pull request as draft October 6, 2025 17:42

knikolla mentioned this pull request Oct 6, 2025

Kristi does Code Reviews (Week of 2025-10-06 and 13) CCI-MOC/ops-issues#1624

Closed

4 tasks

knikolla requested changes Oct 21, 2025

View reviewed changes

Comment thread src/coldfront_plugin_cloud/kc_client.py Outdated

Comment thread src/coldfront_plugin_cloud/tasks.py Outdated

Comment thread src/coldfront_plugin_cloud/tests/functional/openshift/test_allocation.py Outdated

QuanMPhm force-pushed the ops_948/auth_kc branch from d7da5c4 to 3b80589 Compare October 30, 2025 19:03

QuanMPhm requested a review from knikolla October 30, 2025 19:03

QuanMPhm force-pushed the ops_948/auth_kc branch from 3b80589 to cb1d628 Compare November 4, 2025 18:06

QuanMPhm commented Nov 4, 2025

View reviewed changes

Comment thread src/coldfront_plugin_cloud/base.py Outdated

joachimweyl mentioned this pull request Nov 6, 2025

Kristi reviews automatic approval of allocation change requests CCI-MOC/ops-issues#1637

Open

3 tasks

QuanMPhm force-pushed the ops_948/auth_kc branch 2 times, most recently from a217f31 to 0358cb7 Compare March 20, 2026 20:28

QuanMPhm marked this pull request as ready for review March 20, 2026 20:41

QuanMPhm force-pushed the ops_948/auth_kc branch from 0358cb7 to 8fc4ea6 Compare March 20, 2026 20:51

knikolla reviewed Mar 23, 2026

View reviewed changes

Comment thread src/coldfront_plugin_cloud/openshift.py Outdated

knikolla reviewed Mar 23, 2026

View reviewed changes

Comment thread src/coldfront_plugin_cloud/base.py Outdated

knikolla requested changes Mar 23, 2026

View reviewed changes

QuanMPhm force-pushed the ops_948/auth_kc branch from 97d7220 to 151aea2 Compare April 7, 2026 20:53

knikolla requested changes Apr 7, 2026

View reviewed changes

Comment thread src/coldfront_plugin_cloud/attributes.py Outdated

QuanMPhm force-pushed the ops_948/auth_kc branch from 151aea2 to d3087ea Compare April 8, 2026 02:35

QuanMPhm requested a review from knikolla April 8, 2026 14:38

knikolla requested changes Apr 9, 2026

View reviewed changes

Comment thread src/coldfront_plugin_cloud/tasks.py Outdated

Comment thread src/coldfront_plugin_cloud/tasks.py

naved001 reviewed Apr 9, 2026

View reviewed changes

QuanMPhm force-pushed the ops_948/auth_kc branch from d3087ea to 4489473 Compare April 10, 2026 18:23

knikolla requested a review from Copilot April 10, 2026 18:24

Copilot started reviewing on behalf of knikolla April 10, 2026 18:24 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

knikolla requested changes Apr 10, 2026

View reviewed changes

Comment thread src/coldfront_plugin_cloud/kc_client.py

Comment thread src/coldfront_plugin_cloud/kc_client.py

QuanMPhm force-pushed the ops_948/auth_kc branch 2 times, most recently from 51df802 to 6cb51d7 Compare April 10, 2026 20:01

QuanMPhm requested review from knikolla and naved001 April 13, 2026 12:54

larsks reviewed Apr 14, 2026

View reviewed changes

QuanMPhm force-pushed the ops_948/auth_kc branch from 6cb51d7 to 5a6f8d6 Compare April 22, 2026 18:48

QuanMPhm mentioned this pull request Apr 22, 2026

Update validate_allocations command to add users to Keycloak #306

Open

QuanMPhm requested a review from larsks April 22, 2026 18:57

naved001 approved these changes Apr 24, 2026

View reviewed changes

This was referenced May 4, 2026

Allow using validate_allocations to validate Keycloak group memberships #307

Open

Tentative ColdFront changes for 2025-06 maintenance #308

Open

knikolla approved these changes Jun 2, 2026

View reviewed changes

QuanMPhm merged commit 57beb08 into nerc-project:main Jun 2, 2026
5 checks passed

Conversation

QuanMPhm commented Oct 3, 2025

Uh oh!

Uh oh!

Uh oh!

QuanMPhm commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

QuanMPhm commented Oct 30, 2025

Uh oh!

Uh oh!

knikolla commented Mar 18, 2026

Uh oh!

QuanMPhm commented Mar 20, 2026

Uh oh!

knikolla commented Mar 20, 2026

Uh oh!

Uh oh!

Uh oh!

knikolla left a comment

Choose a reason for hiding this comment

Uh oh!

knikolla commented Mar 23, 2026

Uh oh!

QuanMPhm commented Mar 24, 2026

Uh oh!

knikolla commented Mar 25, 2026

Uh oh!

QuanMPhm commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

knikolla left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

knikolla commented Apr 7, 2026

Uh oh!

QuanMPhm commented Apr 8, 2026

Uh oh!

knikolla commented Apr 8, 2026

Uh oh!

knikolla left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

QuanMPhm commented Apr 13, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

QuanMPhm commented Oct 6, 2025 •

edited

Loading

QuanMPhm commented Apr 6, 2026 •

edited

Loading

QuanMPhm commented Apr 22, 2026 •

edited

Loading