ホーム エクスプローラ ヘルプ
サインイン
Server
/
fish-speech
41
0
フォーク 0
ファイル 課題 0 プルリクエスト 0 Wiki

説明なし

ツリー: v1.4.3
ブランチ タグ
fish-speech
ZIP TAR.GZ
Leng Yue 1359896b06 Update README.md 1 年間 前
.github ecaa69e7fc Update docs (#626) 1 年間 前
docs 172bcc5f35 [docs]Fix bugs in Chinese docs and add translated docs of agent for other language. (#656) 1 年間 前
fish_speech 911e037855 Fix original shape matching with crop (#673) 1 年間 前
tools 672000d594 Streaming Agent (#659) 1 年間 前
.dockerignore e413df7145 perf: Optimizing docker builds (#547) 1 年間 前
.gitignore 834b07257c Agent inference (#650) 1 年間 前
.pre-commit-config.yaml 97625fb8e7 [pre-commit.ci] pre-commit autoupdate (#599) 1 年間 前
.project-root 5707699dfd Handle adaptive number of codebooks 2 年 前
.readthedocs.yaml fe293ca492 Use readthedocs instead of github action 2 年 前
API_FLAGS.txt 834b07257c Agent inference (#650) 1 年間 前
LICENSE b91815e074 Switch to CC-BY-NC-SA 4.0 license 1 年間 前
README.md 1359896b06 Update README.md 1 年間 前
docker-compose.dev.yml f6c56c68d4 Update docker-compose.dev.yml 1 年間 前
dockerfile 23fa4d7e38 Fix dockerfile for `pyaudio` (#623) 1 年間 前
dockerfile.dev 23fa4d7e38 Fix dockerfile for `pyaudio` (#623) 1 年間 前
entrypoint.sh e413df7145 perf: Optimizing docker builds (#547) 1 年間 前
inference.ipynb dad516d86d update checkpoint path 1 年間 前
install_env.bat f15d9f23a9 feat: enable more workers in `api.py` (#621) 1 年間 前
mkdocs.yml 172bcc5f35 [docs]Fix bugs in Chinese docs and add translated docs of agent for other language. (#656) 1 年間 前
pyproject.toml bbddbdfbbc fix: Install cachetools dependencies (#676) 1 年間 前
pyrightconfig.json 6d57066e52 Update pre-commit hook 2 年 前
run_cmd.bat 8702c61100 From whisper to sensevoice (#482) 1 年間 前
start.bat 46440f25be 对脚本的一点小修改 (#414) 1 年間 前

README.md

Fish Speech

**English** | [简体中文](docs/README.zh.md) | [Portuguese](docs/README.pt-BR.md) | [日本語](docs/README.ja.md) | [한국어](docs/README.ko.md)




This codebase and all models are released under CC-BY-NC-SA-4.0 License. Please refer to LICENSE for more details.

Fish Agent

We are very excited to annoce that we have made our self-research agent demo open source, you can now try our agent demo online at demo for instant English chat and English and Chinese chat locally by following the docs.

You should mention that the content is released under a CC BY-NC-SA 4.0 licence. And the demo is an early alpha test version, the inference speed needs to be optimised, and there are a lot of bugs waiting to be fixed. If you've found a bug or want to fix it, we'd be very happy to receive an issue or a pull request.

Features

Fish Speech

  1. Zero-shot & Few-shot TTS: Input a 10 to 30-second vocal sample to generate high-quality TTS output. For detailed guidelines, see Voice Cloning Best Practices.

  2. Multilingual & Cross-lingual Support: Simply copy and paste multilingual text into the input box—no need to worry about the language. Currently supports English, Japanese, Korean, Chinese, French, German, Arabic, and Spanish.

  3. No Phoneme Dependency: The model has strong generalization capabilities and does not rely on phonemes for TTS. It can handle text in any language script.

  4. Highly Accurate: Achieves a low CER (Character Error Rate) and WER (Word Error Rate) of around 2% for 5-minute English texts.

  5. Fast: With fish-tech acceleration, the real-time factor is approximately 1:5 on an Nvidia RTX 4060 laptop and 1:15 on an Nvidia RTX 4090.

  6. WebUI Inference: Features an easy-to-use, Gradio-based web UI compatible with Chrome, Firefox, Edge, and other browsers.

  7. GUI Inference: Offers a PyQt6 graphical interface that works seamlessly with the API server. Supports Linux, Windows, and macOS. See GUI.

  8. Deploy-Friendly: Easily set up an inference server with native support for Linux, Windows and MacOS, minimizing speed loss.

Fish Agent

  1. Completely End to End: Automatically integrates ASR and TTS parts, no need to plug-in other models, i.e., true end-to-end, not three-stage (ASR+LLM+TTS).

  2. Timbre Control: Can use reference audio to control the speech timbre.

  3. Emotional: The model can generate speech with strong emotion.

Disclaimer

We do not hold any responsibility for any illegal usage of the codebase. Please refer to your local laws about DMCA and other related laws.

Online Demo

Fish Audio

Fish Agent

Quick Start for Local Inference

inference.ipynb

Videos

V1.4 Demo Video: Youtube

Documents

Samples (2024/10/02 V1.4)

Credits

Tech Report (V1.4)

@misc{fish-speech-v1.4,
      title={Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesis}, 
      author={Shijia Liao and Yuxuan Wang and Tianyu Li and Yifan Cheng and Ruoyi Zhang and Rongzhi Zhou and Yijin Xing},
      year={2024},
      eprint={2411.01156},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2411.01156}, 
}

Sponsor