feat: Add Task Processor metrics #27
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds Prometheus metrics to track task processing in both API and Task Processor modes. It includes new metrics definitions, instrumentation in task execution and task enqueuing, test updates for metric assertions, and README documentation updates.
- Added new counters and histogram in src/task_processor/metrics.py.
- Instrumented task processing in src/task_processor/processor.py and task enqueuing in src/task_processor/decorators.py.
- Updated unit tests and documentation to validate the new metrics.
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/task_processor/test_unit_task_processor_processor.py | Added a test to verify finished task metrics and task duration metrics. |
| tests/unit/task_processor/test_unit_task_processor_decorators.py | Added a test to verify enqueued task metrics on delay. |
| src/task_processor/processor.py | Updated the task runner to record task duration and finished task metrics. |
| src/task_processor/metrics.py | Added metric definitions for enqueued tasks, finished tasks, and task duration. |
| src/task_processor/decorators.py | Updated logging and triggered enqueued task metrics. |
| settings/dev.py | Enabled Task Processor mode. |
| README.md | Documented the newly added Task Processor metrics. |
tests/unit/task_processor/test_unit_task_processor_processor.py
Outdated
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds Task Processor metrics to monitor enqueued tasks, finished tasks, and task durations, along with test coverage and documentation updates.
- Added new tests to assert that the relevant metrics are recorded correctly.
- Integrated metric collection points in the task processor and decorators.
- Updated settings and documentation to reflect the new metrics.
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/task_processor/test_unit_task_processor_processor.py | Added tests for task run metrics. |
| tests/unit/task_processor/test_unit_task_processor_decorators.py | Added tests for metrics on task creation via decorators. |
| src/task_processor/processor.py | Instrumented task execution with timing and finish metrics. |
| src/task_processor/metrics.py | Introduced new Prometheus counters and a histogram for task metrics. |
| src/task_processor/decorators.py | Updated to increment enqueued task metrics in the delay method. |
| src/common/test_tools/plugin.py | Added registry reset logic for metric collectors. |
| settings/dev.py | Enabled TASK_PROCESSOR_MODE for development. |
| README.md | Documented the new Task Processor metrics. |
Comments suppressed due to low confidence (1)
src/task_processor/processor.py:111
- [nitpick] Consider simplifying the timer usage by directly invoking the timer as a context manager with labels, for example using a with-statement on metrics.task_processor_task_duration_seconds.labels(...).time(). This may simplify context management and reduce potential errors with timer closure.
ctx = ExitStack()
Contributor
matthewelwell
left a comment
matthewelwell
left a comment
There was a problem hiding this comment.
These metrics look great. I'm interested to know how we seed the data.
If we start the task processor in an environment that already has a backlog of tasks, how does it handle that?
khvn26
force-pushed
the
feat/task-processor-metrics
branch
from
6584113 to
338d7f8
Compare
Member
Author
I don't think we should:
Let me know if you have a stronger opinion, @matthewelwell. CC @rolodato in case you want to chime in here. |
matthewelwell
approved these changes
Mar 27, 2025
khvn26
force-pushed
the
feat/task-processor-metrics
branch
from
338d7f8 to
6648586
Compare
khvn26
force-pushed
the
feat/task-processor-metrics
branch
4 times, most recently
from
b7713cf to
2c76e44
Compare
rolodato
approved these changes
Apr 1, 2025
Contributor
rolodato
left a comment
rolodato
left a comment
There was a problem hiding this comment.
LGTM, left some suggestions but no blockers 👍
- Port `flagsmith/flagsmith-task-processor` - Port `waitfordb` management command - Introduce PostgreSQL for tests - Bump Poetry from 2.0.1 to 2.1.1
…ished_tasks_total`
- consistent usage of `settings.TASK_PROCESSOR_MODE` - test for failure running tasks when `TASK_PROCESSOR_MODE` is `False` - add `task_processor_mode` pytest marker - consistent usage of `django_db` marker
for more information, see https://pre-commit.ci
khvn26
force-pushed
the
feat/task-processor-metrics
branch
from
2c76e44 to
80943c0
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds the following Task Processor metrics:
task_processor_enqueued_tasks_total: Counter labeled withtask_identifier. Represents all tasks created via.delay(). Exposed both in API and Task Processor mode.task_processor_finished_tasks_total: Counter labeled withtask_identifierandresult("success","failure"). Exposed only in Task Processor mode.task_processor_task_duration_seconds: Histogram labeled withtask_identifierandresult("success","failure"). Exposed only in Task Processor mode.