Skip to content

Configurable webhook responses#4666

Open
midigofrank wants to merge 14 commits intomainfrom
configurable-webhook-responses
Open

Configurable webhook responses#4666
midigofrank wants to merge 14 commits intomainfrom
configurable-webhook-responses

Conversation

@midigofrank
Copy link
Copy Markdown
Collaborator

@midigofrank midigofrank commented Apr 27, 2026

Description

Adds support for configurable webhook responses in sync (after completion) mode.

Previously, when a webhook trigger ran in sync mode, the response was always the final run state with a fixed 201 status code. This PR allows operators to:

  • Configure custom HTTP status codes per trigger — separate codes for success and error runs — via the trigger inspector UI (Options → Response Status)
  • Override the response status and/or body at runtime by setting _webhookResponse in any job's final state, with partial overrides now supported (only status, only body, or both)
  • The _webhookResponse value is read from the state at the end of the run, so any job in the workflow can set or overwrite it

Response metadata headers (x-meta-work-order-id, x-meta-run-id) are now documented and included on all webhook responses (both async and sync).

Closes #4479
Closes #4480
Closes #4481
Closes #4482
Closes #4483
Closes #4484
Closes #4598

This PR has a docs sibling: OpenFn/docs#765

Validation steps

  1. Create a webhook-triggered workflow and set Response Mode → Sync (After Completion)
  2. Under Options, click Configure Response Status and add a Success Status Code (e.g. 200) — verify it is returned on a successful run
  3. Add an Error Status Code (e.g. 422) — trigger a failing run and verify the status code is returned
  4. Remove a status code via the trash icon — verify it falls back to 201
  5. In a job, set _webhookResponse: { status: 202 } (no body) — verify the custom status is used and the body is the final run state
  6. Set _webhookResponse: { body: { ack: true } } (no status) — verify the custom body is used and the status comes from the config (or 201 if none set)
  7. Set both status and body in _webhookResponse — verify both are used
  8. Set _webhookResponse to a non-map value (e.g. a string) — verify it is silently ignored and defaults apply
  9. Set _webhookResponse with an invalid status type (e.g. a string) — verify the malformed response message is returned
  10. Switch back to Async (Before Start) — verify the response configuration is cleared and the trigger responds immediately
  11. Check that x-meta-work-order-id and x-meta-run-id headers are present on both async and sync responses

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

  • I have used Claude Code
  • I have used another model
  • I have not used AI

You can read more details in our
Responsible AI Policy

Pre-submission checklist

  • I have performed an AI review of my code (we recommend using /review
    with Claude Code)
  • I have implemented and tested all related authorization policies.
    (e.g., :owner, :admin, :editor, :viewer)
  • I have updated the changelog.
  • I have ticked a box in "AI usage" in this PR

- Fix snapshot serialization to recursively handle embedded schemas
  using Ecto.embedded_dump + __schema__(:embeds), excluding
  :sync_webhook_response_config from the flat cast field list
- Drop incorrect primary_key: false from sync_webhook_response_config embed
- Fix default_response_body/3 nil-config clause so success runs
  without a configured body correctly return the final state
- Rename webhook response headers to hyphenated form:
  x-meta-work-order-id, x-meta-run-id
- Add run_channel tests covering all webhook response dispatch paths:
  _webhookResponse override, configured status codes/body, malformed
  responses, before_start triggers
- Rewrite webhooks_controller delayed-response tests to use
  WorkOrders.Events for synchronisation instead of Process.sleep,
  and assert on x-meta-* response headers
- Fix integration tests that incorrectly asserted data/meta wrapping
  on the response body; add integration tests for configured
  success_code, error_code, custom body, and _webhookResponse override
- Serialize sync_webhook_response_config as webhook_response in YAML
  export; body is JSON-encoded as a literal block scalar matching how
  job.body is handled
- Expose webhook_response in the provisioning API GET response
- Accept webhook_response on provisioning import and remap it to
  sync_webhook_response_config; decode body if it arrives as a JSON
  string (from YAML literal block)
- Add Jason.Encoder derives so sync_webhook_response_config is included
  when the trigger is serialized for the session context
- Add tests covering export, API output, and import including the
  JSON-string body decoding path
- Export sync_webhook_response_config as webhook_response in YAML code
  view, omitting null fields
- Import webhook_response back to sync_webhook_response_config when
  applying a YAML spec; JSON-parse body if it arrives as a string from
  the literal block
- Include sync_webhook_response_config in transformTrigger so status
  code edits trigger the unsaved changes indicator
- Add WebhookResponseConfig type and webhook_response field to
  SpecWebhookTrigger; add webhook_response schema to workflow-spec.json
@midigofrank midigofrank self-assigned this Apr 27, 2026
@github-project-automation github-project-automation Bot moved this to New Issues in Core Apr 27, 2026
@midigofrank midigofrank requested a review from doc-han April 28, 2026 09:04
@midigofrank midigofrank marked this pull request as ready for review April 28, 2026 14:18
@github-actions
Copy link
Copy Markdown

Security Review ✅

  • S0 (project scoping): New trigger field sync_webhook_response_config is written through the existing provisioner/collaborative-editor paths that already enforce project access; the run-channel broadcast uses work_order:#{run.work_order_id}:webhook_response, and the webhook controller in lib/lightning_web/controllers/webhooks_controller.ex:138-160 subscribes only to the topic of the work order it just created, so no cross-project read or write surface is introduced.
  • S1 (authorization): No new web-layer actions or endpoints — the trigger schema additions in lib/lightning/workflows/trigger.ex:64-66,90-101 and the provisioning JSON field in lib/lightning_web/controllers/api/provisioning_json.ex:105-117 flow through existing :provision/trigger-edit gates, and the run-channel response logic in lib/lightning_web/channels/run_channel.ex:312-419 is reachable only via an authenticated worker holding a run-scoped JWT.
  • S2 (audit trail): Trigger updates via the provisioner remain wrapped in the existing audit_workflows/2 multi (lib/lightning/projects/provisioner.ex:88,123-149) which fires updated_by_provisioner, so the added embedded sync_webhook_response_config is captured under the same workflow audit envelope as kafka_configuration and webhook_reply; no new config-resource write paths are introduced that bypass auditing.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 28, 2026

Codecov Report

❌ Patch coverage is 81.72043% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.73%. Comparing base (4df6cbb) to head (1294869).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
lib/lightning/collaboration/workflow_serializer.ex 38.46% 8 Missing ⚠️
lib/lightning_web/channels/run_channel.ex 85.71% 5 Missing ⚠️
lib/lightning/workflows/snapshot.ex 76.92% 3 Missing ⚠️
lib/lightning/export_utils.ex 91.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4666      +/-   ##
==========================================
- Coverage   89.78%   89.73%   -0.05%     
==========================================
  Files         444      445       +1     
  Lines       21660    21719      +59     
==========================================
+ Hits        19447    19490      +43     
- Misses       2213     2229      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New Issues

1 participant