NIFI-16290 - Node offload should give processors a bounded grace period to stop cleanly before forced termination

[pvillard31]

Summary

NIFI-16290 - Node offload should give processors a bounded grace period to stop cleanly before forced termination

StandardFlowService.offload() currently calls rootGroup.stopProcessing() and discards the returned future, then immediately force-terminates any processor whose logical scheduled state is STOPPED. Since logical STOPPED also covers the physical STOPPING state, this can terminate a processor before its asynchronous stop lifecycle has actually finished (overtaking @OnUnscheduled, aborting the wait for active threads to drain, and skipping @OnStopped) so processors never get a real chance to release external resources cleanly during node offload. This PR adds a bounded grace period: offload now waits on the aggregate stopProcessing() future for up to nifi.flowcontroller.graceful.shutdown.period before falling through to the existing unconditional termination sweep. A hung or wedged processor, including one holding an uncommitted ProcessSession, still cannot block offload indefinitely (the fallback always runs regardless of outcome: completed, timed out, interrupted, or exceptional). The graceful-wait outcome is logged for operational visibility.

The change is covered by a new unit test suite in TestStandardFlowService that deterministically exercises multiple paths: normal completion, timeout, interruption during the processor/RPG/queue waits with correct restore semantics, exceptional/cancelled futures, and that termination still runs even when the graceful wait completes normally, which is required to roll back FlowFiles held in an uncommitted session. The clustered OffloadIT system test retains its original testOffload scenario, now also running against a short graceful.shutdown.period override so it exercises the new code path on every iteration.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
• Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using ./mvnw clean install -P contrib-check
  • JDK 21
  • JDK 25

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files