Skip to content

fix(mq): 图片与动作拆开独立 Stream,避免同队互堵放大上游 429 - #814

Merged
johnnyzhang-eng merged 2 commits into
1024XEngineer:mainfrom
xiaocheny214:fix/split-generation-image-action-streams
Aug 27, 2026
Merged

fix(mq): 图片与动作拆开独立 Stream,避免同队互堵放大上游 429#814
johnnyzhang-eng merged 2 commits into
1024XEngineer:mainfrom
xiaocheny214:fix/split-generation-image-action-streams

Conversation

@xiaocheny214

@xiaocheny214 xiaocheny214 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • 图片与动作不再共用 windup:stream:generation:新任务分别进 generation-image / generation-action,按类型独立出队。
  • 消费者按线程池空槽领取,满槽不再 XREADGROUP,避免 PEL 里堆已领未跑的消息。
  • 代码默认并发对齐现网 4/2/2;旧 generation Stream 保留 drain 消费者吃残留消息。

Closes #813

Test plan

  • uv run pytest tests/test_mq_catalog.py tests/test_mq_worker.py tests/test_i2v_poll.py tests/test_generation_quota.py tests/test_generation_api.py tests/test_mq.py tests/test_mq_delayed.py tests/test_env_example_keys_are_live.py -q
  • 新图片任务只出现在 generation-image;新动作/poll 只出现在 generation-action
  • 图堆积时动作仍能按 2 路出队
  • 无环境变量时并发为 4/2/2
  • generation Stream 上残留消息仍能被 drain 消费

@vercel

vercel Bot commented Aug 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
windup Ignored Ignored Preview Aug 27, 2026 6:21am

@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.81319% with 12 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...end/packages/app/src/windup_app/worker/consumer.py 86.66% 8 Missing ⚠️
...nd/packages/app/src/windup_app/bootstrap/worker.py 20.00% 4 Missing ⚠️

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #814      +/-   ##
==========================================
- Coverage   87.07%   86.49%   -0.59%     
==========================================
  Files         288      292       +4     
  Lines       21525    21866     +341     
  Branches     3294     3329      +35     
==========================================
+ Hits        18742    18912     +170     
- Misses       1856     2023     +167     
- Partials      927      931       +4     
Flag Coverage Δ
backend 91.51% <86.81%> (-0.18%) ⬇️
frontend 80.52% <ø> (-1.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...d/packages/app/src/windup_app/server/mq/catalog.py 100.00% <100.00%> (ø)
.../src/windup_app/server/orchestrator/client_bake.py 93.13% <ø> (ø)
...app/src/windup_app/server/orchestrator/i2v_poll.py 93.10% <100.00%> (ø)
.../app/src/windup_app/server/orchestrator/recover.py 96.66% <100.00%> (+0.05%) ⬆️
.../packages/app/src/windup_app/web/api/generation.py 90.96% <ø> (ø)
...nd/packages/app/src/windup_app/bootstrap/worker.py 37.64% <20.00%> (-0.91%) ⬇️
...end/packages/app/src/windup_app/worker/consumer.py 90.03% <86.66%> (-1.20%) ⬇️

... and 16 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审查了本 PR 的 Stream 路由、恢复投递、延迟轮询、client bake 以及消费者并发控制。Stream 拆分和旧队列 drain 路径整体一致,但当前消费者的“空槽”判断仍会让独立线程池各自出现排队,无法完全满足避免 PEL 堆积的目标。

验证:尝试运行 PR 测试计划时,环境未安装 uvuv: command not found)。

Comment thread backend/packages/app/src/windup_app/worker/consumer.py Outdated
@johnnyzhang-eng
johnnyzhang-eng merged commit 0d5cbcf into 1024XEngineer:main Aug 27, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(mq): 图片与动作拆开独立 Stream,避免同队互堵放大上游 429

2 participants