Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@

Thanks for helping improve FunASR. We especially welcome contributions that make the first successful transcription faster, improve deployment reliability, or make benchmarks easier to reproduce.

## Repository scope

FunASR spans four repositories. Before opening an issue or PR, check which repo owns the area you are working on:

| Repository | Owns |
|---|---|
| `modelscope/FunASR` | Toolkit, inference pipelines, deployment services, `funasr` PyPI package |
| `FunAudioLLM/Fun-ASR` | Fun-ASR-Nano / MLT model family, checkpoints, model cards, benchmarks, model-level integrations (Transformers, vLLM, GGUF) |
| `FunAudioLLM/SenseVoice` | SenseVoice model: ASR + emotion + audio events |
| `modelscope/FunClip` | Video transcription, subtitles, Gradio UI |

**Quick test:** does the problem persist with a different model? If yes, it belongs in `modelscope/FunASR`. If it only affects one model, file it in that model's repo.

Full details: [Repository roles & roadmap](./docs/repository_roles.md) ([中文](./docs/repository_roles_zh.md))

## High-impact areas

- **Quick start reliability:** installation notes, CPU/GPU/MPS compatibility, dependency fixes, and runnable examples.
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,7 @@ llama-funasr-sensevoice -m ./gguf/SenseVoiceSmall-f16.gguf --vad ./gguf/fsmn-vad
| 📖 [Documentation](https://modelscope.github.io/FunASR/) | 🐛 [Issues](https://github.com/modelscope/FunASR/issues) |
| 💬 [Discussions](https://github.com/modelscope/FunASR/discussions) | 🤗 [HuggingFace](https://huggingface.co/funasr) |
| 🤝 [Contributing](./CONTRIBUTING.md) | 🌐 [funasr.com](https://www.funasr.com) |
| 🗺️ [Repository roles & roadmap](./docs/repository_roles.md) | 📈 [Growth plan](./docs/community_growth_20k.md) |
| 🧩 [Community projects](./docs/community_projects.md) | 💡 [Use-case showcase](./docs/use_case_showcase.md) |

## Star History
Expand Down
1 change: 1 addition & 0 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,7 @@ docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-
| 📖 [文档](https://modelscope.github.io/FunASR/zh/) | 🐛 [问题反馈](https://github.com/modelscope/FunASR/issues) |
| 💬 [讨论](https://github.com/modelscope/FunASR/discussions) | 🤗 [HuggingFace](https://huggingface.co/funasr) |
| 🤝 [贡献指南](./CONTRIBUTING.md) | 📈 [20k 增长计划](./docs/community_growth_20k.md) |
| 🗺️ [仓库职责与路线图](./docs/repository_roles_zh.md) | 🌐 [funasr.com](https://www.funasr.com) |
| 🧩 [社区项目](./docs/community_projects.md) | 💡 [使用案例](./docs/use_case_showcase.md) |

## Star 趋势
Expand Down
99 changes: 99 additions & 0 deletions docs/repository_roles.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Repository Roles and Roadmap

This document explains the responsibility boundaries, user entry points, and issue routing across the four FunASR ecosystem repositories, along with a directional roadmap.

> **Versioned release roadmap: pending maintainer confirmation.**
> This document does not commit to version numbers or release dates. The repository has no established milestones; GitHub tags go up to `v1.3.13`, and PyPI has published `1.3.14`; the boundaries for `1.4 / 1.5 / 2.0` have not been confirmed by core maintainers.

---

## Why this document exists

The four repositories share models and tooling but their responsibility boundaries were never written down, causing two practical problems:

1. **Misrouted issues** — model problems get filed against the toolkit, deployment questions land in model repos, and issues bounce between them.
2. **Duplicate implementation drift** — the same realtime service exists in multiple repositories, and fixes only land in one copy. [#3101](https://github.com/modelscope/FunASR/issues/3101) is a concrete example: an unbounded long-session state bug had to be fixed separately in [#3214](https://github.com/modelscope/FunASR/pull/3214) and [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135).

---

## Repository responsibilities

| Repository | Canonical responsibility | Not here |
|---|---|---|
| [modelscope/FunASR](https://github.com/modelscope/FunASR) (toolkit / runtime) | Framework and inference pipelines, training and fine-tuning, components (VAD / punctuation / ITN / speaker), **deployment services (including realtime WebSocket)**, `funasr` PyPI package | Model weights and model cards; application-layer UI |
| [FunAudioLLM/Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR) (model repo) | Fun-ASR-Nano / MLT model family and LLM-ASR identity: model documentation, weight releases, capability scope (languages / dialects / accents / hotwords / timestamps / speaker), benchmarks, fine-tuning, and model-level integrations (Transformers, vLLM, GGUF) | Service implementation (links to FunASR; no longer maintains its own authoritative copy) |
| [FunAudioLLM/SenseVoice](https://github.com/FunAudioLLM/SenseVoice) (model repo) | SenseVoice speech understanding foundation model: ASR / language identification (LID) / speech emotion recognition (SER) / audio event detection (AED), and model-side usage | General inference framework; deployment services |
| [modelscope/FunClip](https://github.com/modelscope/FunClip) (application layer) | FunASR-based video transcription, subtitle generation, and LLM-assisted clipping; local Gradio UI | Underlying ASR capabilities and model issues (upstream to FunASR / model repos) |

---

## User entry points

| I want to… | Go to |
|---|---|
| Use Python for speech recognition / training / fine-tuning | [modelscope/FunASR](https://github.com/modelscope/FunASR) |
| Deploy a realtime streaming ASR service, recommend **Fun-ASR-Nano + vLLM** | [modelscope/FunASR/fun_asr_nano](https://github.com/modelscope/FunASR/tree/main/examples/industrial_data_pretraining/fun_asr_nano) — **canonical implementation, see below** |
| Understand Fun-ASR-Nano / MLT capabilities, checkpoints, benchmarks, or use Transformers / vLLM / GGUF integrations | [FunAudioLLM/Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR) |
| Use emotion recognition / audio event detection | [FunAudioLLM/SenseVoice](https://github.com/FunAudioLLM/SenseVoice) |
| Generate video subtitles / clip videos | [modelscope/FunClip](https://github.com/modelscope/FunClip) |

---

## Issue routing

| Problem type | File it in |
|---|---|
| Framework, inference pipeline, training, fine-tuning | `modelscope/FunASR` |
| Deployment services: realtime WebSocket, offline service, SDK | `modelscope/FunASR` |
| VAD / punctuation / ITN / speaker component behavior | `modelscope/FunASR` |
| Fun-ASR model family recognition quality, language support, weights, benchmarks, or model-level integrations (Transformers / vLLM / GGUF) | `FunAudioLLM/Fun-ASR` |
| SenseVoice recognition / emotion / event detection quality | `FunAudioLLM/SenseVoice` |
| Video clipping, subtitle export, Gradio UI | `modelscope/FunClip` |

**Quick test: does the problem persist if you swap in a different model?**

- **Yes** → it is a framework / service issue → `modelscope/FunASR`
- **Only with a specific model** → it is a model issue → the corresponding model repo

---

## Realtime WebSocket service: canonical source

**The realtime WebSocket service in [Fun-ASR-Nano + vLLM realtime WebSocket service](https://github.com/modelscope/FunASR/blob/main/examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py) is the recommended implementation.**

- Feature development, bug fixes, and behavior changes **always land in `modelscope/FunASR` first**.
- Model repos (`FunAudioLLM/Fun-ASR`) **link to the canonical implementation only** and no longer describe their own copy as the authoritative version.
- Related issues should all be filed in `modelscope/FunASR`.

**Why:** two copies evolving independently means fixes land in only one. [#3101](https://github.com/modelscope/FunASR/issues/3101) demonstrated the cost — the same unbounded long-session state bug required separate fixes in [#3214](https://github.com/modelscope/FunASR/pull/3214) and [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135). Converging to a single canonical source is a Next roadmap item.

---

## Roadmap (directional)

> Each item links to an existing issue or PR where available. Items without an owner or acceptance evidence do not have completion dates.

### Now

- **Bounded realtime long-session state** — merged via [#3214](https://github.com/modelscope/FunASR/pull/3214), [#3101](https://github.com/modelscope/FunASR/issues/3101) closed; model-repo mirror fix [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135) also merged.
- **Fun-ASR-Nano native Transformers integration** — [huggingface/transformers#46180](https://github.com/huggingface/transformers/pull/46180); in review. See the linked PR for current CI and review status.
- **Clarify repository roles and issue routing** — [#3203](https://github.com/modelscope/FunASR/issues/3203); this document.

### Next

- **Converge duplicate realtime services to canonical source** (see above), preventing further drift.
- **Establish a smoke-tested support matrix**: Python / CLI / WebSocket / container. The goal is a single canonical entry point reachable from the top-level README, with pinned dependencies, a fixed test audio file and startup smoke test, and clear CPU/GPU support scope — rather than multiple scripts each claiming to be the recommended entry point.
- **Stable headless / API contract**: CLI / HTTP / gRPC / WebSocket paths that do not depend on Gradio or browser interaction; machine-parseable requests and responses; health checks, error codes, and compatibility tests suitable for services and agent integration.
- **Containerization (separate track)**: requires follow-up work to determine the canonical image, version tags, CPU/GPU support matrix, health checks, and build CI. This document does not provide installation steps or recommend any specific cluster solution; that work will be tracked separately by someone who can verify CPU/GPU images end to end.

### Later

- After the interfaces and compatibility tests above are stable, evaluate `2.x` for any breaking changes needed.
- **Version numbers and release plans are confirmed by core maintainers through milestones / release plans**, not predetermined in this document.

---

## Related

- 中文版: [`repository_roles_zh.md`](./repository_roles_zh.md)
- Contributing guide: [`CONTRIBUTING.md`](../CONTRIBUTING.md)
99 changes: 99 additions & 0 deletions docs/repository_roles_zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# 仓库职责与路线图

本文档说明 FunASR 生态四个仓库的职责边界、用户入口和 issue 路由,并给出一份方向性路线图。

> **版本化发布路线图:待维护者确认。**
> 本文档不承诺版本号和发布日期。当前仓库没有已建立的 milestone;GitHub tag 至 `v1.3.13`,PyPI 已发布 `1.3.14`;`1.4 / 1.5 / 2.0` 的边界尚未经核心 maintainer 确认。

---

## 为什么需要这份文档

四个仓库共享同一套模型和工具链,但职责边界此前没有写下来,带来两个实际问题:

1. **Issue 路由错位** —— 模型问题提到工具包,部署问题提到模型仓,来回转派。
2. **重复实现漂移** —— 同一个实时服务在多个仓库各有一份副本,修复只落在其中一处。[#3101](https://github.com/modelscope/FunASR/issues/3101) 就是这样产生的:长会话状态无界的缺陷需要在两个仓库分别修复([#3214](https://github.com/modelscope/FunASR/pull/3214) 与 [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135))。

---

## 四仓职责

| 仓库 | 核心职责 | 不在这里 |
|---|---|---|
| [modelscope/FunASR](https://github.com/modelscope/FunASR)(工具包 / 运行时) | 框架与推理管线 (pipelines)、训练与微调、VAD / 标点 / ITN / 说话人等组件、**部署服务(含实时 WebSocket 服务)**、`funasr` PyPI 包 | 模型权重与 model card;应用层 UI |
| [FunAudioLLM/Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR)(模型仓) | Fun-ASR-Nano / MLT 模型家族与 LLM-ASR 身份:模型说明、权重发布、能力范围(语言 / 方言 / 口音 / 热词 / 时间戳 / 说话人)、模型评测、微调,以及模型级集成(Transformers、vLLM、GGUF) | 服务实现(链接 FunASR,不再自带权威副本) |
| [FunAudioLLM/SenseVoice](https://github.com/FunAudioLLM/SenseVoice)(模型仓) | SenseVoice 语音理解基础模型:ASR / 语种识别(LID) / 情感识别(SER) / 音频事件检测(AED),及其模型侧用法 | 通用推理框架;部署服务 |
| [modelscope/FunClip](https://github.com/modelscope/FunClip)(应用层) | 基于 FunASR 的视频转写、字幕生成与 LLM 辅助剪辑;本地 Gradio UI | 底层 ASR 能力与模型问题(上游到 FunASR / 模型仓) |

---

## 用户入口

| 我想… | 去哪里 |
|---|---|
| 用 Python 做语音识别 / 训练 / 微调 | [modelscope/FunASR](https://github.com/modelscope/FunASR) |
| 部署实时流式 ASR 服务,推荐 Fun-ASR-Nano + vLLM 做实时识别 | [modelscope/FunASR/fun_asr_nano](https://github.com/modelscope/FunASR/tree/main/examples/industrial_data_pretraining/fun_asr_nano) —— **推荐实现,见下节** |
| 了解 Fun-ASR-Nano / MLT 的能力范围、权重、评测,或使用 Transformers / vLLM / GGUF 集成 | [FunAudioLLM/Fun-ASR](https://github.com/FunAudioLLM/Fun-ASR) |
| 需要情感识别 / 音频事件检测 | [FunAudioLLM/SenseVoice](https://github.com/FunAudioLLM/SenseVoice) |
| 做视频字幕 / 剪辑 | [modelscope/FunClip](https://github.com/modelscope/FunClip) |

---

## Issue 路由

| 问题类型 | 提到 |
|---|---|
| 框架、推理管线 (pipelines)、训练、微调 | `modelscope/FunASR` |
| 部署服务:实时 WebSocket、离线服务、SDK | `modelscope/FunASR` |
| VAD / 标点 / ITN / 说话人 组件行为 | `modelscope/FunASR` |
| Fun-ASR 系列模型的识别效果、语言支持、权重、评测,以及 Transformers / vLLM / GGUF 等模型级集成 | `FunAudioLLM/Fun-ASR` |
| SenseVoice 的识别 / 情感 / 事件检测效果 | `FunAudioLLM/SenseVoice` |
| 视频剪辑、字幕导出、Gradio UI | `modelscope/FunClip` |

**判断法则:换一个模型后问题是否还在?**

- **还在** → 是框架 / 服务问题 → `modelscope/FunASR`
- **只在某个模型上出现** → 是模型问题 → 对应模型仓

---

## 实时 WebSocket 服务:推荐实现

**`modelscope/FunASR` 中的[Fun-ASR-Nano + vLLM 实时 WebSocket 服务](https://github.com/modelscope/FunASR/blob/main/examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py)是推荐实现。**

- 功能开发、缺陷修复、行为变更,**一律先在 `modelscope/FunASR` 落地**。
- 模型仓(`FunAudioLLM/Fun-ASR`)**只链接到权威实现**,不再把自带副本描述为权威实现。
- 相关 issue 一律提到 `modelscope/FunASR`。

**为什么:** 两份副本各自演进,修复就会只落在其中一处。[#3101](https://github.com/modelscope/FunASR/issues/3101) 已经暴露过这个代价——同一个长会话状态无界的缺陷,需要分别在 [#3214](https://github.com/modelscope/FunASR/pull/3214) 和 [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135) 修两次。收敛到唯一权威实现是 Next 的一项工作。

---

## 路线图(方向性)

> 每一项均链接到现有 issue / PR。没有 owner 或验收证据的条目不写完成日期。

### 当前

- **实时服务长会话有界状态** —— 已由 [#3214](https://github.com/modelscope/FunASR/pull/3214) 合并,[#3101](https://github.com/modelscope/FunASR/issues/3101) 已关闭;模型仓镜像修复 [FunAudioLLM/Fun-ASR#135](https://github.com/FunAudioLLM/Fun-ASR/pull/135) 亦已合并。
- **Fun-ASR-Nano 的 Transformers 原生集成** —— [huggingface/transformers#46180](https://github.com/huggingface/transformers/pull/46180),正在审查中;当前 CI 与审查状态请以链接的 PR 为准。
- **明确四仓职责与 issue 路由** —— [#3203](https://github.com/modelscope/FunASR/issues/3203);即本文档。

### 下一步

- **收敛重复的实时服务到唯一权威实现**(见上节),避免再次漂移。
- **建立经过 smoke test 的支持矩阵**:Python / CLI / WebSocket / 容器。目标是从顶层 README 能找到一个权威入口、依赖锁定、有固定测试音频与启动 smoke test、CPU/GPU 支持范围写清楚——而不是多个脚本各自声称是推荐入口。
- **稳定的 headless / API 契约**:不依赖 Gradio 或浏览器操作的 CLI / HTTP / gRPC / WebSocket 路径;请求与响应可机器解析;有健康检查、错误码和兼容性测试,适合服务与 agent 调用。
- **容器化(独立方向)**:需要后续单独确定官方镜像、版本标签、CPU/GPU 支持矩阵、健康检查与构建 CI。本文档不提供安装步骤,也不推荐任何特定集群方案;该工作将另开任务,由能够实际验证 CPU/GPU 镜像的人负责。

### 下一阶段

- 在上述接口与兼容性测试稳定之后,再评估需要 breaking changes 的 `2.x`。
- **版本号与发布计划由核心 maintainer 通过 milestone / release plan 确认**,不在本文档中预设。

---

## 相关链接

- English version: [`repository_roles.md`](./repository_roles.md)
- 贡献指南:[`CONTRIBUTING.md`](../CONTRIBUTING.md)