Skip to content

fix: apply text-to-image when TTS dual output keeps text in the chain#9172

Open
kldsjfas wants to merge 3 commits into
AstrBotDevs:masterfrom
kldsjfas:fix/8261-t2i-with-tts-dual-output
Open

fix: apply text-to-image when TTS dual output keeps text in the chain#9172
kldsjfas wants to merge 3 commits into
AstrBotDevs:masterfrom
kldsjfas:fix/8261-t2i-with-tts-dual-output

Conversation

@kldsjfas

@kldsjfas kldsjfas commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Motivation / 动机

修复 #8261

开启 TTS 且勾选"同时输出语音和文字内容"(dual_output)后,文本转图像完全失效,超过阈值的长文本不再转成图片。

原因有两处:

  1. result_decorate/stage.py 里文本转图像分支是挂在 TTS 分支后面的 elif——只要 TTS 执行了,T2I 连判断的机会都没有;
  2. 即使改成 if,T2I 收集文本的循环遇到非 Plain 组件就 break,而 dual_output 产生的消息链是 [Record(语音), Plain(文字), ...],开头的 Record 会直接让收集结果为空。

Modifications / 改动点

astrbot/core/pipeline/result_decorate/stage.py

  • T2I 分支由 elif 改为独立的 if,TTS 之后仍会评估;
  • 收集文本时跳过 Record 段(记录下来),其余组件的 break 行为保持不变;
  • 转图成功后把语音段保留在结果链里([Record..., Image]),不再整链替换丢掉语音。

行为保持不变的部分:未开启 TTS 时链的处理与原来一致;TTS 不带 dual_output 时链里没有 Plain,T2I 依旧不触发;低于阈值时链保持 [Record, Plain] 原样。

tests/unit/test_result_decorate_t2i.py:新增 4 个用例覆盖上述四种组合(dual_output 长文本转图并保留语音、纯文本原行为、低于阈值不转、仅语音不转)。

  • This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

$ python -m pytest tests/unit/test_result_decorate_t2i.py -v
tests/unit/test_result_decorate_t2i.py::test_t2i_still_applies_when_tts_dual_output PASSED
tests/unit/test_result_decorate_t2i.py::test_t2i_without_tts_keeps_original_behavior PASSED
tests/unit/test_result_decorate_t2i.py::test_dual_output_below_threshold_keeps_voice_and_text PASSED
tests/unit/test_result_decorate_t2i.py::test_tts_without_dual_output_sends_voice_only PASSED
4 passed

未修复时 test_t2i_still_applies_when_tts_dual_output 失败、其余三个通过(即其余三个用例锁定的是需要保持的现有行为)。tests/unit 全量与 master 基线对比无新增失败;ruff check / ruff format --check 通过。


Checklist / 检查清单

  • 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
    / 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。

  • 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
    / 我的更改经过了良好的测试,并已在上方提供了"验证步骤"和"运行截图"

  • 🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in requirements.txt and pyproject.toml.
    / 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到了 requirements.txtpyproject.toml 文件相应位置。

Summary by Sourcery

Ensure text-to-image conversion still runs when TTS dual output is enabled and preserves generated voice alongside the image.

New Features:

  • Support combining TTS dual output with text-to-image so long texts are still converted to images while retaining the voice segment.

Bug Fixes:

  • Fix text-to-image not triggering when TTS dual output keeps text in the message chain.

Enhancements:

  • Preserve TTS voice records before text-to-image conversion so the final response includes both audio and image when applicable.

Tests:

  • Add unit tests covering interactions between TTS (with and without dual output) and text-to-image conversion, including threshold behavior and chain composition.

@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. area:core The bug / feature is about astrbot's core, backend labels Jul 6, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the text-to-image (t2i) processing stage to support TTS dual output by preserving voice segments (Record components) alongside the generated image, and adds comprehensive unit tests for these scenarios. The reviewer identified a critical issue where other message components (such as At or Face) in the message chain could be silently discarded during the reconstruction of result.chain. The reviewer provided actionable code suggestions to capture and append any remaining components to the end of the chain.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines 352 to 361
parts = []
records = []
for comp in result.chain:
if isinstance(comp, Plain):
parts.append("\n\n" + comp.text)
elif isinstance(comp, Record):
# TTS 双输出时语音段与文本段交错,跳过语音段并在转图后保留
records.append(comp)
else:
break

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

在当前实现中,如果 result.chain 中除了 PlainRecord 之外还包含其他消息段(例如 AtFaceReply 等),循环遇到这些组件时会执行 break。然而,在文本成功转为图片后,result.chain 会被直接重写为 [*records, Image],这会导致 break 之后的所有其他消息段被静默丢弃。

为了避免消息段丢失,建议在循环中记录下未处理的剩余消息段(remain),并在重新组装 result.chain 时将它们追加到末尾。

Suggested change
parts = []
records = []
for comp in result.chain:
if isinstance(comp, Plain):
parts.append("\n\n" + comp.text)
elif isinstance(comp, Record):
# TTS 双输出时语音段与文本段交错,跳过语音段并在转图后保留
records.append(comp)
else:
break
parts = []
records = []
remain = []
for i, comp in enumerate(result.chain):
if isinstance(comp, Plain):
parts.append("\n\n" + comp.text)
elif isinstance(comp, Record):
# TTS 双输出时语音段与文本段交错,跳过语音段并在转图后保留
records.append(comp)
else:
remain = result.chain[i:]
break

Comment on lines +380 to +391
if url.startswith("http"):
result.chain = [Image.fromURL(url)]
result.chain = [*records, Image.fromURL(url)]
elif (
self.ctx.astrbot_config["t2i_use_file_service"]
and self.ctx.astrbot_config["callback_api_base"]
):
token = await file_token_service.register_file(url)
url = f"{self.ctx.astrbot_config['callback_api_base']}/api/file/{token}"
logger.debug(f"已注册:{url}")
result.chain = [Image.fromURL(url)]
result.chain = [*records, Image.fromURL(url)]
else:
result.chain = [Image.fromFileSystem(url)]
result.chain = [*records, Image.fromFileSystem(url)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

配合前面对剩余消息段(remain)的记录,在重新组装 result.chain 时,需要将 remain 追加到新链的末尾,以确保其他未转图的消息段(如 AtFace 等)不会丢失。

Suggested change
if url.startswith("http"):
result.chain = [Image.fromURL(url)]
result.chain = [*records, Image.fromURL(url)]
elif (
self.ctx.astrbot_config["t2i_use_file_service"]
and self.ctx.astrbot_config["callback_api_base"]
):
token = await file_token_service.register_file(url)
url = f"{self.ctx.astrbot_config['callback_api_base']}/api/file/{token}"
logger.debug(f"已注册:{url}")
result.chain = [Image.fromURL(url)]
result.chain = [*records, Image.fromURL(url)]
else:
result.chain = [Image.fromFileSystem(url)]
result.chain = [*records, Image.fromFileSystem(url)]
if url.startswith("http"):
result.chain = [*records, Image.fromURL(url), *remain]
elif (
self.ctx.astrbot_config["t2i_use_file_service"]
and self.ctx.astrbot_config["callback_api_base"]
):
token = await file_token_service.register_file(url)
url = f"{self.ctx.astrbot_config['callback_api_base']}/api/file/{token}"
logger.debug(f"已注册:{url}")
result.chain = [*records, Image.fromURL(url), *remain]
else:
result.chain = [*records, Image.fromFileSystem(url), *remain]

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@kldsjfas

kldsjfas commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

已按建议补上 remain 的处理:转图时把 break 之后的消息段记录下来,重组结果链时追加到图片之后([*records, Image, *remain]),At、表情等组件不再丢失。顺带新增 test_t2i_keeps_components_after_text 用例锁定该行为。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core The bug / feature is about astrbot's core, backend size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant