feat(hardware): add daily hardware aggregates recomputed from raw - #2037
Open
alanpeixinho wants to merge 4 commits into
Open
alanpeixinho wants to merge 4 commits into
alanpeixinho wants to merge 4 commits into
Conversation
alanpeixinho
force-pushed
the
feat/hardware-daily-aggregates
branch
3 times, most recently
from
August 17, 2026 21:59
39d0d5b to
27eb2e6
Compare
alanpeixinho
changed the title
alanpeixinho
marked this pull request as ready for review
August 17, 2026 22:10
alanpeixinho
marked this pull request as draft
August 17, 2026 22:13
alanpeixinho
marked this pull request as ready for review
August 25, 2026 12:59
alanpeixinho
force-pushed
the
feat/hardware-daily-aggregates
branch
from
August 25, 2026 14:36
b5c7772 to
89f9401
Compare
Member
There was a problem hiding this comment.
--origins does not apply to hardware_daily_* ?
Those tables are in the default --tables set and are deleted with only checkout_day < cutoff. running prune_db --older-than "30 days" --origins maestro (the docs example) can wipe every origin’s daily aggregates while leaving other tenants’ raw rows.
Contributor
Author
There was a problem hiding this comment.
Nice catch. Daily prune was ignoring --origins and deleting by checkout_day only. Fixed in c9392fb — hardware_daily_* now filter on checkout_origin the same way raw rows use origin.
alanpeixinho
force-pushed
the
feat/hardware-daily-aggregates
branch
2 times, most recently
from
September 2, 2026 17:29
ea02b2d to
c9392fb
Compare
alanpeixinho
changed the title
felipebergamin
approved these changes
Sep 10, 2026
- Add daily aggregate tables and recompute_hardware_daily (idempotent, locked, keeps rows when raw is pruned) - Materialize day_tests once per day so both aggregate inserts share one raw scan - Cron: hourly for today, every 6h for the last 7 days (drop the 60-day run) - Wire prune_db, seed_test_data and update_db for the new tables - Leave hardware_status written and read as before Closes #2034 Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>
Store checkout origin so listings can filter without a join, rebuild fifteen days for late tests, and drop the unused healthcheck. ANALYZE the day temp table and raise work_mem in the cron txn so the planner hashes instead of spilling. Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>
* filter hardware_daily_* by checkout_origin when --origins is set * align dry-run count table columns * add integration test for origin-scoped daily prune Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>
Resolves conflicting 0019 leaf nodes after rebasing onto main so CI can run migrate. Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>
alanpeixinho
force-pushed
the
feat/hardware-daily-aggregates
branch
from
September 23, 2026 12:57
ac58f3a to
43c8bc7
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it is
Adds daily hardware aggregate tables (hardware_daily_builds, hardware_daily_tests) built from raw checkouts / builds / tests.
exist, keeps the stale rows.
Closes #2034.
How to test
migrate
Run local database and apply migration
manual recompute
python manage.py recompute_hardware_daily --day=2025-07-18python manage.py recompute_hardware_daily --days=7verify rows
Check that the aggregate counts for builds, boots and tests match the sum of the raw tables (builds and tests).
prune dry-run (aggregates only)
python manage.py prune_db --older-than "30 days" \ --tables hardware_daily_builds,hardware_daily_tests --dry-run