diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 688c8de4f..a620ff1d0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,6 +2,21 @@ Thanks for helping improve FunASR. We especially welcome contributions that make the first successful transcription faster, improve deployment reliability, or make benchmarks easier to reproduce. +## Repository scope + +FunASR spans four repositories. Before opening an issue or PR, check which repo owns the area you are working on: + +| Repository | Owns | +|---|---| +| `modelscope/FunASR` | Toolkit, inference pipelines, deployment services, `funasr` PyPI package | +| `FunAudioLLM/Fun-ASR` | Fun-ASR-Nano / MLT model family, checkpoints, model cards, benchmarks, model-level integrations (Transformers, vLLM, GGUF) | +| `FunAudioLLM/SenseVoice` | SenseVoice model: ASR + emotion + audio events | +| `modelscope/FunClip` | Video transcription, subtitles, Gradio UI | + +**Quick test:** does the problem persist with a different model? If yes, it belongs in `modelscope/FunASR`. If it only affects one model, file it in that model's repo. + +Full details: [Repository roles & roadmap](./docs/repository_roles.md) ([中文](./docs/repository_roles_zh.md)) + ## High-impact areas - **Quick start reliability:** installation notes, CPU/GPU/MPS compatibility, dependency fixes, and runnable examples. diff --git a/README.md b/README.md index 1ded43143..229120a51 100644 --- a/README.md +++ b/README.md @@ -303,6 +303,7 @@ llama-funasr-sensevoice -m ./gguf/SenseVoiceSmall-f16.gguf --vad ./gguf/fsmn-vad | 📖 [Documentation](https://modelscope.github.io/FunASR/) | 🐛 [Issues](https://github.com/modelscope/FunASR/issues) | | 💬 [Discussions](https://github.com/modelscope/FunASR/discussions) | 🤗 [HuggingFace](https://huggingface.co/funasr) | | 🤝 [Contributing](./CONTRIBUTING.md) | 🌐 [funasr.com](https://www.funasr.com) | +| 🗺️ [Repository roles & roadmap](./docs/repository_roles.md) | 📈 [Growth plan](./docs/community_growth_20k.md) | | 🧩 [Community projects](./docs/community_projects.md) | 💡 [Use-case showcase](./docs/use_case_showcase.md) | ## Star History diff --git a/README_zh.md b/README_zh.md index 274ca8995..54fbf2c53 100644 --- a/README_zh.md +++ b/README_zh.md @@ -287,6 +287,7 @@ docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime- | 📖 [文档](https://modelscope.github.io/FunASR/zh/) | 🐛 [问题反馈](https://github.com/modelscope/FunASR/issues) | | 💬 [讨论](https://github.com/modelscope/FunASR/discussions) | 🤗 [HuggingFace](https://huggingface.co/funasr) | | 🤝 [贡献指南](./CONTRIBUTING.md) | 📈 [20k 增长计划](./docs/community_growth_20k.md) | +| 🗺️ [仓库职责与路线图](./docs/repository_roles_zh.md) | 🌐 [funasr.com](https://www.funasr.com) | | 🧩 [社区项目](./docs/community_projects.md) | 💡 [使用案例](./docs/use_case_showcase.md) | ## Star 趋势 diff --git a/docs/repository_roles.md b/docs/repository_roles.md new file mode 100644 index 000000000..d71d2b1f1 --- /dev/null +++ b/docs/repository_roles.md @@ -0,0 +1,99 @@ +# Repository Roles and Roadmap + +This document explains the responsibility boundaries, user entry points, and issue routing across the four FunASR ecosystem repositories, along with a directional roadmap. + +> **Versioned release roadmap: pending maintainer confirmation.** +> This document does not commit to version numbers or release dates. The repository has no established milestones; GitHub tags go up to `v1.3.13`, and PyPI has published `1.3.14`; the boundaries for `1.4 / 1.5 / 2.0` have not been confirmed by core maintainers. + +--- + +## Why this document exists + +The four repositories share models and tooling but their responsibility boundaries were never written down, causing two practical problems: + +1. **Misrouted issues** — model problems get filed against the toolkit, deployment questions land in model repos, and issues bounce between them. +2. **Duplicate implementation drift** — the same realtime service exists in multiple repositories, and fixes only land in one copy. [#3101](https://github.com/modelscope/FunASR/issues/3101) is a concrete example: an unbounded long-session state bug had to be fixed separately in [#3214](https://github.com/modelscope/FunASR/pull/3214) and [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135). + +--- + +## Repository responsibilities + +| Repository | Canonical responsibility | Not here | +|---|---|---| +| [modelscope/FunASR](https://github.com/modelscope/FunASR) (toolkit / runtime) | Framework and inference pipelines, training and fine-tuning, components (VAD / punctuation / ITN / speaker), **deployment services (including realtime WebSocket)**, `funasr` PyPI package | Model weights and model cards; application-layer UI | +| [FunAudioLLM/Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR) (model repo) | Fun-ASR-Nano / MLT model family and LLM-ASR identity: model documentation, weight releases, capability scope (languages / dialects / accents / hotwords / timestamps / speaker), benchmarks, fine-tuning, and model-level integrations (Transformers, vLLM, GGUF) | Service implementation (links to FunASR; no longer maintains its own authoritative copy) | +| [FunAudioLLM/SenseVoice](https://github.com/FunAudioLLM/SenseVoice) (model repo) | SenseVoice speech understanding foundation model: ASR / language identification (LID) / speech emotion recognition (SER) / audio event detection (AED), and model-side usage | General inference framework; deployment services | +| [modelscope/FunClip](https://github.com/modelscope/FunClip) (application layer) | FunASR-based video transcription, subtitle generation, and LLM-assisted clipping; local Gradio UI | Underlying ASR capabilities and model issues (upstream to FunASR / model repos) | + +--- + +## User entry points + +| I want to… | Go to | +|---|---| +| Use Python for speech recognition / training / fine-tuning | [modelscope/FunASR](https://github.com/modelscope/FunASR) | +| Deploy a realtime streaming ASR service, recommend **Fun-ASR-Nano + vLLM** | [modelscope/FunASR/fun_asr_nano](https://github.com/modelscope/FunASR/tree/main/examples/industrial_data_pretraining/fun_asr_nano) — **canonical implementation, see below** | +| Understand Fun-ASR-Nano / MLT capabilities, checkpoints, benchmarks, or use Transformers / vLLM / GGUF integrations | [FunAudioLLM/Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR) | +| Use emotion recognition / audio event detection | [FunAudioLLM/SenseVoice](https://github.com/FunAudioLLM/SenseVoice) | +| Generate video subtitles / clip videos | [modelscope/FunClip](https://github.com/modelscope/FunClip) | + +--- + +## Issue routing + +| Problem type | File it in | +|---|---| +| Framework, inference pipeline, training, fine-tuning | `modelscope/FunASR` | +| Deployment services: realtime WebSocket, offline service, SDK | `modelscope/FunASR` | +| VAD / punctuation / ITN / speaker component behavior | `modelscope/FunASR` | +| Fun-ASR model family recognition quality, language support, weights, benchmarks, or model-level integrations (Transformers / vLLM / GGUF) | `FunAudioLLM/Fun-ASR` | +| SenseVoice recognition / emotion / event detection quality | `FunAudioLLM/SenseVoice` | +| Video clipping, subtitle export, Gradio UI | `modelscope/FunClip` | + +**Quick test: does the problem persist if you swap in a different model?** + +- **Yes** → it is a framework / service issue → `modelscope/FunASR` +- **Only with a specific model** → it is a model issue → the corresponding model repo + +--- + +## Realtime WebSocket service: canonical source + +**The realtime WebSocket service in [Fun-ASR-Nano + vLLM realtime WebSocket service](https://github.com/modelscope/FunASR/blob/main/examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py) is the recommended implementation.** + +- Feature development, bug fixes, and behavior changes **always land in `modelscope/FunASR` first**. +- Model repos (`FunAudioLLM/Fun-ASR`) **link to the canonical implementation only** and no longer describe their own copy as the authoritative version. +- Related issues should all be filed in `modelscope/FunASR`. + +**Why:** two copies evolving independently means fixes land in only one. [#3101](https://github.com/modelscope/FunASR/issues/3101) demonstrated the cost — the same unbounded long-session state bug required separate fixes in [#3214](https://github.com/modelscope/FunASR/pull/3214) and [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135). Converging to a single canonical source is a Next roadmap item. + +--- + +## Roadmap (directional) + +> Each item links to an existing issue or PR where available. Items without an owner or acceptance evidence do not have completion dates. + +### Now + +- **Bounded realtime long-session state** — merged via [#3214](https://github.com/modelscope/FunASR/pull/3214), [#3101](https://github.com/modelscope/FunASR/issues/3101) closed; model-repo mirror fix [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135) also merged. +- **Fun-ASR-Nano native Transformers integration** — [huggingface/transformers#46180](https://github.com/huggingface/transformers/pull/46180); in review. See the linked PR for current CI and review status. +- **Clarify repository roles and issue routing** — [#3203](https://github.com/modelscope/FunASR/issues/3203); this document. + +### Next + +- **Converge duplicate realtime services to canonical source** (see above), preventing further drift. +- **Establish a smoke-tested support matrix**: Python / CLI / WebSocket / container. The goal is a single canonical entry point reachable from the top-level README, with pinned dependencies, a fixed test audio file and startup smoke test, and clear CPU/GPU support scope — rather than multiple scripts each claiming to be the recommended entry point. +- **Stable headless / API contract**: CLI / HTTP / gRPC / WebSocket paths that do not depend on Gradio or browser interaction; machine-parseable requests and responses; health checks, error codes, and compatibility tests suitable for services and agent integration. +- **Containerization (separate track)**: requires follow-up work to determine the canonical image, version tags, CPU/GPU support matrix, health checks, and build CI. This document does not provide installation steps or recommend any specific cluster solution; that work will be tracked separately by someone who can verify CPU/GPU images end to end. + +### Later + +- After the interfaces and compatibility tests above are stable, evaluate `2.x` for any breaking changes needed. +- **Version numbers and release plans are confirmed by core maintainers through milestones / release plans**, not predetermined in this document. + +--- + +## Related + +- 中文版: [`repository_roles_zh.md`](./repository_roles_zh.md) +- Contributing guide: [`CONTRIBUTING.md`](../CONTRIBUTING.md) diff --git a/docs/repository_roles_zh.md b/docs/repository_roles_zh.md new file mode 100644 index 000000000..218a7513f --- /dev/null +++ b/docs/repository_roles_zh.md @@ -0,0 +1,99 @@ +# 仓库职责与路线图 + +本文档说明 FunASR 生态四个仓库的职责边界、用户入口和 issue 路由,并给出一份方向性路线图。 + +> **版本化发布路线图：待维护者确认。** +> 本文档不承诺版本号和发布日期。当前仓库没有已建立的 milestone;GitHub tag 至 `v1.3.13`,PyPI 已发布 `1.3.14`;`1.4 / 1.5 / 2.0` 的边界尚未经核心 maintainer 确认。 + +--- + +## 为什么需要这份文档 + +四个仓库共享同一套模型和工具链,但职责边界此前没有写下来,带来两个实际问题: + +1. **Issue 路由错位** —— 模型问题提到工具包,部署问题提到模型仓,来回转派。 +2. **重复实现漂移** —— 同一个实时服务在多个仓库各有一份副本,修复只落在其中一处。[#3101](https://github.com/modelscope/FunASR/issues/3101) 就是这样产生的:长会话状态无界的缺陷需要在两个仓库分别修复([#3214](https://github.com/modelscope/FunASR/pull/3214) 与 [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135))。 + +--- + +## 四仓职责 + +| 仓库 | 核心职责 | 不在这里 | +|---|---|---| +| [modelscope/FunASR](https://github.com/modelscope/FunASR)（工具包 / 运行时） | 框架与推理管线 (pipelines)、训练与微调、VAD / 标点 / ITN / 说话人等组件、**部署服务(含实时 WebSocket 服务)**、`funasr` PyPI 包 | 模型权重与 model card;应用层 UI | +| [FunAudioLLM/Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR)（模型仓） | Fun-ASR-Nano / MLT 模型家族与 LLM-ASR 身份:模型说明、权重发布、能力范围(语言 / 方言 / 口音 / 热词 / 时间戳 / 说话人)、模型评测、微调，以及模型级集成(Transformers、vLLM、GGUF) | 服务实现(链接 FunASR,不再自带权威副本) | +| [FunAudioLLM/SenseVoice](https://github.com/FunAudioLLM/SenseVoice)（模型仓） | SenseVoice 语音理解基础模型:ASR / 语种识别(LID) / 情感识别(SER) / 音频事件检测(AED),及其模型侧用法 | 通用推理框架;部署服务 | +| [modelscope/FunClip](https://github.com/modelscope/FunClip)（应用层） | 基于 FunASR 的视频转写、字幕生成与 LLM 辅助剪辑;本地 Gradio UI | 底层 ASR 能力与模型问题(上游到 FunASR / 模型仓) | + +--- + +## 用户入口 + +| 我想… | 去哪里 | +|---|---| +| 用 Python 做语音识别 / 训练 / 微调 | [modelscope/FunASR](https://github.com/modelscope/FunASR) | +| 部署实时流式 ASR 服务，推荐 Fun-ASR-Nano + vLLM 做实时识别 | [modelscope/FunASR/fun_asr_nano](https://github.com/modelscope/FunASR/tree/main/examples/industrial_data_pretraining/fun_asr_nano) —— **推荐实现,见下节** | +| 了解 Fun-ASR-Nano / MLT 的能力范围、权重、评测,或使用 Transformers / vLLM / GGUF 集成 | [FunAudioLLM/Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR) | +| 需要情感识别 / 音频事件检测 | [FunAudioLLM/SenseVoice](https://github.com/FunAudioLLM/SenseVoice) | +| 做视频字幕 / 剪辑 | [modelscope/FunClip](https://github.com/modelscope/FunClip) | + +--- + +## Issue 路由 + +| 问题类型 | 提到 | +|---|---| +| 框架、推理管线 (pipelines)、训练、微调 | `modelscope/FunASR` | +| 部署服务:实时 WebSocket、离线服务、SDK | `modelscope/FunASR` | +| VAD / 标点 / ITN / 说话人 组件行为 | `modelscope/FunASR` | +| Fun-ASR 系列模型的识别效果、语言支持、权重、评测,以及 Transformers / vLLM / GGUF 等模型级集成 | `FunAudioLLM/Fun-ASR` | +| SenseVoice 的识别 / 情感 / 事件检测效果 | `FunAudioLLM/SenseVoice` | +| 视频剪辑、字幕导出、Gradio UI | `modelscope/FunClip` | + +**判断法则:换一个模型后问题是否还在?** + +- **还在** → 是框架 / 服务问题 → `modelscope/FunASR` +- **只在某个模型上出现** → 是模型问题 → 对应模型仓 + +--- + +## 实时 WebSocket 服务:推荐实现 + +**`modelscope/FunASR` 中的[Fun-ASR-Nano + vLLM 实时 WebSocket 服务](https://github.com/modelscope/FunASR/blob/main/examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py)是推荐实现。** + +- 功能开发、缺陷修复、行为变更,**一律先在 `modelscope/FunASR` 落地**。 +- 模型仓(`FunAudioLLM/Fun-ASR`)**只链接到权威实现**,不再把自带副本描述为权威实现。 +- 相关 issue 一律提到 `modelscope/FunASR`。 + +**为什么:** 两份副本各自演进,修复就会只落在其中一处。[#3101](https://github.com/modelscope/FunASR/issues/3101) 已经暴露过这个代价——同一个长会话状态无界的缺陷,需要分别在 [#3214](https://github.com/modelscope/FunASR/pull/3214) 和 [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135) 修两次。收敛到唯一权威实现是 Next 的一项工作。 + +--- + +## 路线图（方向性） + +> 每一项均链接到现有 issue / PR。没有 owner 或验收证据的条目不写完成日期。 + +### 当前 + +- **实时服务长会话有界状态** —— 已由 [#3214](https://github.com/modelscope/FunASR/pull/3214) 合并,[#3101](https://github.com/modelscope/FunASR/issues/3101) 已关闭;模型仓镜像修复 [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135) 亦已合并。 +- **Fun-ASR-Nano 的 Transformers 原生集成** —— [huggingface/transformers#46180](https://github.com/huggingface/transformers/pull/46180),正在审查中;当前 CI 与审查状态请以链接的 PR 为准。 +- **明确四仓职责与 issue 路由** —— [#3203](https://github.com/modelscope/FunASR/issues/3203);即本文档。 + +### 下一步 + +- **收敛重复的实时服务到唯一权威实现**(见上节),避免再次漂移。 +- **建立经过 smoke test 的支持矩阵**:Python / CLI / WebSocket / 容器。目标是从顶层 README 能找到一个权威入口、依赖锁定、有固定测试音频与启动 smoke test、CPU/GPU 支持范围写清楚——而不是多个脚本各自声称是推荐入口。 +- **稳定的 headless / API 契约**:不依赖 Gradio 或浏览器操作的 CLI / HTTP / gRPC / WebSocket 路径;请求与响应可机器解析;有健康检查、错误码和兼容性测试,适合服务与 agent 调用。 +- **容器化(独立方向)**:需要后续单独确定官方镜像、版本标签、CPU/GPU 支持矩阵、健康检查与构建 CI。本文档不提供安装步骤,也不推荐任何特定集群方案;该工作将另开任务,由能够实际验证 CPU/GPU 镜像的人负责。 + +### 下一阶段 + +- 在上述接口与兼容性测试稳定之后,再评估需要 breaking changes 的 `2.x`。 +- **版本号与发布计划由核心 maintainer 通过 milestone / release plan 确认**,不在本文档中预设。 + +--- + +## 相关链接 + +- English version: [`repository_roles.md`](./repository_roles.md) +- 贡献指南:[`CONTRIBUTING.md`](../CONTRIBUTING.md)