Skip to content

refactor(parse): remove MinerU PDF strategy - #3966

Closed
qin-ctx wants to merge 1 commit into
mainfrom
refactor/remove-mineru
Closed

refactor(parse): remove MinerU PDF strategy#3966
qin-ctx wants to merge 1 commit into
mainfrom
refactor/remove-mineru

Conversation

@qin-ctx

@qin-ctx qin-ctx commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Description

OpenViking 当前同时保留了本地 pdfplumber 解析和 MinerU 远程转换。默认的 auto 模式先执行本地解析,失败后才可能调用 MinerU。仓库内没有为这条远程路径提供稳定的集成入口或契约覆盖,却额外引入了一套配置、回退逻辑和返回元数据。

修改前:

  • PDFConfig.strategy 支持 auto / local / mineru
  • 本地解析失败时,auto 可根据 mineru_endpoint 回退到 MinerU。
  • 解析结果会输出 pdf_strategystrategy 元数据。

修改后:

  • PDF 始终在本地使用 pdfplumber 提取文本、表格、图片和文档结构,再交给 MarkdownParser 构建上下文树。
  • 同步 PDF 转换仍在线程池中执行,不阻塞事件循环。
  • MinerU 实现、自动回退、配置、示例和策略元数据全部删除。

Human Involvement

  • A human participated in the implementation or review loop
  • This PR was generated entirely by AI agents without human participation in the loop

Related Issue

N/A

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • 删除 PDFParser 中的 MinerU 远程转换和 auto 回退分支,收敛为单一本地解析链路。
  • 删除 strategy 和全部 mineru_* PDF 配置,更新中英文文档和示例配置。
  • 更新现有 PDF 测试以调用唯一的转换入口;没有新增测试文件、测试函数或参数化用例。

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

66 passed:

tests/parse/test_pdf_bookmark_extraction.py
tests/parse/test_markdown_no_split.py
tests/parse/test_parser_config_wiring.py
tests/test_config_loader.py

同时通过 Ruff、git diff --check 和无新增测试用例检查。

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

N/A

Additional Notes

这是一个配置破坏性变更。升级后需从 parsers.pdf 中删除以下字段:

{
  "strategy": "auto",
  "mineru_endpoint": "...",
  "mineru_api_key": "...",
  "mineru_timeout": 300,
  "mineru_params": {}
}

这些字段不再被接受,配置加载时会明确报 Unknown config field

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant