Skip to content

[enhance](nereids) improve masking of user's password for ALTER USER and CREATE USER commands in audit logs#62141

Open
iaorekhov-1980 wants to merge 17 commits intoapache:masterfrom
iaorekhov-1980:feat/sql_password_masking
Open

[enhance](nereids) improve masking of user's password for ALTER USER and CREATE USER commands in audit logs#62141
iaorekhov-1980 wants to merge 17 commits intoapache:masterfrom
iaorekhov-1980:feat/sql_password_masking

Conversation

@iaorekhov-1980
Copy link
Copy Markdown
Contributor

What problem does this PR solve?

This PR adds masking for users passwords for CREATE USER and ALTER USER commands.
The masked values will be stored in audit table and audit files instead of actual values.
The same functionality already exists for SET USER PASSWORD and SET LDAP_ADMIN_PASSWORD commands, so we other commands related to passwords should be masked as well.

Could you please include this PR into 4.x branches, please!

Issue Number: close #62140

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  1. EncryptSQLTest.java is updated to include two checks for two mentioned commands - ALTER USER and CREATE USER
  • Does this need documentation?
    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Copy Markdown
Contributor

Thearas commented Apr 6, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@iaorekhov-1980 iaorekhov-1980 changed the title feat: tests only, so should fail [enhance] (nereids) improve masking of users' password in the audit logs Apr 7, 2026
@iaorekhov-1980 iaorekhov-1980 changed the title [enhance] (nereids) improve masking of users' password in the audit logs [enhance] (nereids) improve masking of users' password for ALTER USER and CREATE USER commands in audit logs Apr 7, 2026
@iaorekhov-1980 iaorekhov-1980 force-pushed the feat/sql_password_masking branch 2 times, most recently from 9851a15 to 7ddf5e0 Compare April 7, 2026 08:44
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@iaorekhov-1980 iaorekhov-1980 changed the title [enhance] (nereids) improve masking of users' password for ALTER USER and CREATE USER commands in audit logs [enhance](nereids) improve masking of users' password for ALTER USER and CREATE USER commands in audit logs Apr 7, 2026
@iaorekhov-1980 iaorekhov-1980 changed the title [enhance](nereids) improve masking of users' password for ALTER USER and CREATE USER commands in audit logs [enhance](nereids) improve masking of user's password for ALTER USER and CREATE USER commands in audit logs Apr 7, 2026
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

1 similar comment
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 37.50% (3/8) 🎉
Increment coverage report
Complete coverage report

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

4 similar comments
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@iaorekhov-1980 iaorekhov-1980 force-pushed the feat/sql_password_masking branch from 53a743b to be55c79 Compare April 8, 2026 08:35
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

3 similar comments
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 75.00% (3/4) 🎉
Increment coverage report
Complete coverage report

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

/review

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall. I did not find any blocking correctness issues in the changed Nereids audit encryption path.

Critical checkpoint conclusions:

  • Goal and correctness: The PR goal is to mask passwords in audit output for CREATE USER and ALTER USER. The final code achieves that in the Nereids path by marking AlterUserCommand as NeedAuditEncryption and masking the password token in visitGrantUserIdentify, which is the shared parser path used by both CREATE USER and ALTER USER.
  • Scope and minimality: The change is small and focused. The parser change only labels the existing password literal in grantUserIdentify, the masking logic is centralized in one existing encryption visitor hook, and the command side change for ALTER USER is minimal.
  • Concurrency: No new concurrency or locking behavior is introduced.
  • Lifecycle and initialization: No special lifecycle or static initialization risks introduced.
  • Configuration: No new config items added.
  • Compatibility: No incompatible storage, protocol, or symbol changes. Grammar label changes are internal to the parser visitor and do not affect external compatibility.
  • Parallel paths: I checked the shared grantUserIdentify path and the audit hook in AuditLogHelper; the implementation now covers both Nereids CREATE USER and ALTER USER consistently.
  • Conditional checks: The new null check on ctx.pwd is straightforward and necessary because IDENTIFIED BY ... is optional in the grammar.
  • Test coverage: The PR adds focused unit coverage for both CREATE USER and ALTER USER masking. I could not fully validate by running EncryptSQLTest in this runner because FE Maven resolution failed on missing org.apache.doris:fe-foundation:1.2-SNAPSHOT, so local execution here was blocked by environment dependency resolution rather than a code failure.
  • Observability: Existing audit logging path remains the same; no extra observability appears necessary for this change.
  • Transaction and persistence: Not applicable.
  • Data write and atomicity: Not applicable.
  • FE and BE variable passing: Not applicable.
  • Performance: The change reuses the existing audit encryption reparse path and adds no meaningful extra overhead beyond paths already using NeedAuditEncryption.
  • Other issues: None identified in the reviewed scope.

Residual risk:

  • The added tests cover the plain IDENTIFIED BY form. The shared implementation should also mask the IDENTIFIED BY PASSWORD form, but that variant is not explicitly covered by this PR test coverage.

@iaorekhov-1980 iaorekhov-1980 force-pushed the feat/sql_password_masking branch from f2bbb44 to cd0cedd Compare April 15, 2026 13:44
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run buildall

1 similar comment
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (4/4) 🎉
Increment coverage report
Complete coverage report

@iaorekhov-1980 iaorekhov-1980 force-pushed the feat/sql_password_masking branch from 9dbb47f to 117bf77 Compare May 4, 2026 07:53
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run buildall

2 similar comments
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run buildall

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run buildall

@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

Hello, @morrySnow / @starocean999
Could you please help me to complete the review from copilot and execute the required checks?
As it looks like team city check can't be launched by comments as usual :(

@morningman
Copy link
Copy Markdown
Contributor

run buildall

@morningman morningman self-assigned this May 4, 2026
@morningman
Copy link
Copy Markdown
Contributor

[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20210_20210&logView=flowAware)  [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:testCompile (default-testCompile) on project fe-core: Compilation failure: Compilation failure:
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20211_20211&logView=flowAware)  [ERROR] /root/doris/fe/fe-core/src/test/java/org/apache/doris/nereids/parser/EncryptSQLTest.java:[470,13] cannot find symbol
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20212_20212&logView=flowAware)  [ERROR]   symbol:   variable Config
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20213_20213&logView=flowAware)  [ERROR]   location: class org.apache.doris.nereids.parser.EncryptSQLTest
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20214_20214&logView=flowAware)  [ERROR] /root/doris/fe/fe-core/src/test/java/org/apache/doris/nereids/parser/EncryptSQLTest.java:[502,13] cannot find symbol
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20215_20215&logView=flowAware)  [ERROR]   symbol:   variable Config
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20216_20216&logView=flowAware)  [ERROR]   location: class org.apache.doris.nereids.parser.EncryptSQLTest
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20217_20217&logView=flowAware)  [ERROR] -> [Help 1]
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20218_20218&logView=flowAware)  [ERROR]
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20219_20219&logView=flowAware)  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20220_20220&logView=flowAware)  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20221_20221&logView=flowAware)  [ERROR]
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20222_20222&logView=flowAware)  [ERROR] For more information about the errors and possible solutions, please read the following articles:
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20223_20223&logView=flowAware)  [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20224_20224&logView=flowAware)  [ERROR]
[03:10:57 ](http://43.132.222.7:8111/buildConfiguration/Doris_DorisCompile_Compile/938433?expandBuildDeploymentsSection=false&hideTestsFromDependencies=false&hideProblemsFromDependencies=false&expandPull+Request+Details=true&expandBuildProblemsSection=true&showLog=938433_20225_20225&logView=flowAware)  [ERROR] After correcting the problems, you can resume the build with the command

FE Compile error

@iaorekhov-1980 iaorekhov-1980 dismissed stale reviews from starocean999 and morrySnow via cf4e875 May 5, 2026 15:32
@github-actions github-actions Bot removed the approved Indicates a PR has been approved by one committer. label May 5, 2026
@iaorekhov-1980
Copy link
Copy Markdown
Contributor Author

run feut

@morningman
Copy link
Copy Markdown
Contributor

run buildall

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

PR approved by at least one committer and no changes requested.

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.x dev/4.1.x reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] (nereids) improve masking of users' password for ALTER USER and CREATE USER in audit logs

6 participants