fix: apply text-to-image when TTS dual output keeps text in the chain#9172
fix: apply text-to-image when TTS dual output keeps text in the chain#9172kldsjfas wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the text-to-image (t2i) processing stage to support TTS dual output by preserving voice segments (Record components) alongside the generated image, and adds comprehensive unit tests for these scenarios. The reviewer identified a critical issue where other message components (such as At or Face) in the message chain could be silently discarded during the reconstruction of result.chain. The reviewer provided actionable code suggestions to capture and append any remaining components to the end of the chain.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| parts = [] | ||
| records = [] | ||
| for comp in result.chain: | ||
| if isinstance(comp, Plain): | ||
| parts.append("\n\n" + comp.text) | ||
| elif isinstance(comp, Record): | ||
| # TTS 双输出时语音段与文本段交错,跳过语音段并在转图后保留 | ||
| records.append(comp) | ||
| else: | ||
| break |
There was a problem hiding this comment.
在当前实现中,如果 result.chain 中除了 Plain 和 Record 之外还包含其他消息段(例如 At、Face、Reply 等),循环遇到这些组件时会执行 break。然而,在文本成功转为图片后,result.chain 会被直接重写为 [*records, Image],这会导致 break 之后的所有其他消息段被静默丢弃。
为了避免消息段丢失,建议在循环中记录下未处理的剩余消息段(remain),并在重新组装 result.chain 时将它们追加到末尾。
| parts = [] | |
| records = [] | |
| for comp in result.chain: | |
| if isinstance(comp, Plain): | |
| parts.append("\n\n" + comp.text) | |
| elif isinstance(comp, Record): | |
| # TTS 双输出时语音段与文本段交错,跳过语音段并在转图后保留 | |
| records.append(comp) | |
| else: | |
| break | |
| parts = [] | |
| records = [] | |
| remain = [] | |
| for i, comp in enumerate(result.chain): | |
| if isinstance(comp, Plain): | |
| parts.append("\n\n" + comp.text) | |
| elif isinstance(comp, Record): | |
| # TTS 双输出时语音段与文本段交错,跳过语音段并在转图后保留 | |
| records.append(comp) | |
| else: | |
| remain = result.chain[i:] | |
| break |
| if url.startswith("http"): | ||
| result.chain = [Image.fromURL(url)] | ||
| result.chain = [*records, Image.fromURL(url)] | ||
| elif ( | ||
| self.ctx.astrbot_config["t2i_use_file_service"] | ||
| and self.ctx.astrbot_config["callback_api_base"] | ||
| ): | ||
| token = await file_token_service.register_file(url) | ||
| url = f"{self.ctx.astrbot_config['callback_api_base']}/api/file/{token}" | ||
| logger.debug(f"已注册:{url}") | ||
| result.chain = [Image.fromURL(url)] | ||
| result.chain = [*records, Image.fromURL(url)] | ||
| else: | ||
| result.chain = [Image.fromFileSystem(url)] | ||
| result.chain = [*records, Image.fromFileSystem(url)] |
There was a problem hiding this comment.
配合前面对剩余消息段(remain)的记录,在重新组装 result.chain 时,需要将 remain 追加到新链的末尾,以确保其他未转图的消息段(如 At、Face 等)不会丢失。
| if url.startswith("http"): | |
| result.chain = [Image.fromURL(url)] | |
| result.chain = [*records, Image.fromURL(url)] | |
| elif ( | |
| self.ctx.astrbot_config["t2i_use_file_service"] | |
| and self.ctx.astrbot_config["callback_api_base"] | |
| ): | |
| token = await file_token_service.register_file(url) | |
| url = f"{self.ctx.astrbot_config['callback_api_base']}/api/file/{token}" | |
| logger.debug(f"已注册:{url}") | |
| result.chain = [Image.fromURL(url)] | |
| result.chain = [*records, Image.fromURL(url)] | |
| else: | |
| result.chain = [Image.fromFileSystem(url)] | |
| result.chain = [*records, Image.fromFileSystem(url)] | |
| if url.startswith("http"): | |
| result.chain = [*records, Image.fromURL(url), *remain] | |
| elif ( | |
| self.ctx.astrbot_config["t2i_use_file_service"] | |
| and self.ctx.astrbot_config["callback_api_base"] | |
| ): | |
| token = await file_token_service.register_file(url) | |
| url = f"{self.ctx.astrbot_config['callback_api_base']}/api/file/{token}" | |
| logger.debug(f"已注册:{url}") | |
| result.chain = [*records, Image.fromURL(url), *remain] | |
| else: | |
| result.chain = [*records, Image.fromFileSystem(url), *remain] |
|
已按建议补上 remain 的处理:转图时把 break 之后的消息段记录下来,重组结果链时追加到图片之后([*records, Image, *remain]),At、表情等组件不再丢失。顺带新增 test_t2i_keeps_components_after_text 用例锁定该行为。 |
Motivation / 动机
修复 #8261。
开启 TTS 且勾选"同时输出语音和文字内容"(dual_output)后,文本转图像完全失效,超过阈值的长文本不再转成图片。
原因有两处:
result_decorate/stage.py里文本转图像分支是挂在 TTS 分支后面的elif——只要 TTS 执行了,T2I 连判断的机会都没有;if,T2I 收集文本的循环遇到非Plain组件就break,而 dual_output 产生的消息链是[Record(语音), Plain(文字), ...],开头的Record会直接让收集结果为空。Modifications / 改动点
astrbot/core/pipeline/result_decorate/stage.py:elif改为独立的if,TTS 之后仍会评估;Record段(记录下来),其余组件的break行为保持不变;[Record..., Image]),不再整链替换丢掉语音。行为保持不变的部分:未开启 TTS 时链的处理与原来一致;TTS 不带 dual_output 时链里没有
Plain,T2I 依旧不触发;低于阈值时链保持[Record, Plain]原样。tests/unit/test_result_decorate_t2i.py:新增 4 个用例覆盖上述四种组合(dual_output 长文本转图并保留语音、纯文本原行为、低于阈值不转、仅语音不转)。Screenshots or Test Results / 运行截图或测试结果
未修复时
test_t2i_still_applies_when_tts_dual_output失败、其余三个通过(即其余三个用例锁定的是需要保持的现有行为)。tests/unit全量与 master 基线对比无新增失败;ruff check/ruff format --check通过。Checklist / 检查清单
😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
/ 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。
👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
/ 我的更改经过了良好的测试,并已在上方提供了"验证步骤"和"运行截图"。
🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in
requirements.txtandpyproject.toml./ 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到了
requirements.txt和pyproject.toml文件相应位置。Summary by Sourcery
Ensure text-to-image conversion still runs when TTS dual output is enabled and preserves generated voice alongside the image.
New Features:
Bug Fixes:
Enhancements:
Tests: