Sākums Izpētīt Palīdzība
Pierakstīties
Server
/
fish-speech
41
0
Atdalīts 0
Faili Problēmas 0 Izmaiņu pieprasījumi 0 Vikivietne
Koks: 75d7ecb5b5
Atzari Tagi
fish-speech
/
docs
/
en
/
index.md

index.md 6.3 KB
Vēsture Neapstrādāts

OpenAudio (formerly Fish-Speech)

OpenAudio

Advanced Text-to-Speech Model Series

Try it now: Fish Audio Playground | Learn more: OpenAudio Website

!!! warning "Legal Notice"

We assume no responsibility for any illegal use of the codebase. Please refer to the local laws regarding DMCA (Digital Millennium Copyright Act) and other relevant laws in your area.

**License:** This codebase is released under Apache 2.0 license and all models are released under the CC-BY-NC-SA-4.0 license.

Introduction

We are excited to announce that we have rebranded to OpenAudio - introducing a brand new series of advanced Text-to-Speech models that builds upon the foundation of Fish-Speech with significant improvements and new capabilities.

Openaudio-S1-mini: Video; Hugging Face;

Fish-Speech v1.5: Video; Hugging Face;

Highlights ✨

Emotion Control

OpenAudio S1 supports a variety of emotional, tone, and special markers to enhance speech synthesis:

  • Basic emotions:

    (angry) (sad) (excited) (surprised) (satisfied) (delighted) 
(scared) (worried) (upset) (nervous) (frustrated) (depressed)
(empathetic) (embarrassed) (disgusted) (moved) (proud) (relaxed)
(grateful) (confident) (interested) (curious) (confused) (joyful)

  • Advanced emotions:

    (disdainful) (unhappy) (anxious) (hysterical) (indifferent) 
(impatient) (guilty) (scornful) (panicked) (furious) (reluctant)
(keen) (disapproving) (negative) (denying) (astonished) (serious)
(sarcastic) (conciliative) (comforting) (sincere) (sneering)
(hesitating) (yielding) (painful) (awkward) (amused)

  • Tone markers:

    (in a hurry tone) (shouting) (screaming) (whispering) (soft tone)

  • Special audio effects:

    (laughing) (chuckling) (sobbing) (crying loudly) (sighing) (panting)
(groaning) (crowd laughing) (background laughter) (audience laughing)

You can also use Ha,ha,ha to control, there's many other cases waiting to be explored by yourself.

Excellent TTS quality

We use Seed TTS Eval Metrics to evaluate the model performance, and the results show that OpenAudio S1 achieves 0.008 WER and 0.004 CER on English text, which is significantly better than previous models. (English, auto eval, based on OpenAI gpt-4o-transcribe, speaker distance using Revai/pyannote-wespeaker-voxceleb-resnet34-LM)

Model Word Error Rate (WER) Character Error Rate (CER) Speaker Distance
S1 0.008 0.004 0.332
S1-mini 0.011 0.005 0.380

Two Type of Models

Model Size Availability Features
S1 4B parameters Avaliable on fish.audio Full-featured flagship model
S1-mini 0.5B parameters Avaliable on huggingface hf space Distilled version with core capabilities

Both S1 and S1-mini incorporate online Reinforcement Learning from Human Feedback (RLHF).

Features

  1. Zero-shot & Few-shot TTS: Input a 10 to 30-second vocal sample to generate high-quality TTS output. For detailed guidelines, see Voice Cloning Best Practices.

  2. Multilingual & Cross-lingual Support: Simply copy and paste multilingual text into the input box—no need to worry about the language. Currently supports English, Japanese, Korean, Chinese, French, German, Arabic, and Spanish.

  3. No Phoneme Dependency: The model has strong generalization capabilities and does not rely on phonemes for TTS. It can handle text in any language script.

  4. Highly Accurate: Achieves a low CER (Character Error Rate) of around 0.4% and WER (Word Error Rate) of around 0.8% for Seed-TTS Eval.

  5. Fast: With fish-tech acceleration, the real-time factor is approximately 1:5 on an Nvidia RTX 4060 laptop and 1:15 on an Nvidia RTX 4090.

  6. WebUI Inference: Features an easy-to-use, Gradio-based web UI compatible with Chrome, Firefox, Edge, and other browsers.

  7. GUI Inference: Offers a PyQt6 graphical interface that works seamlessly with the API server. Supports Linux, Windows, and macOS. See GUI.

  8. Deploy-Friendly: Easily set up an inference server with native support for Linux, Windows (MacOS comming soon), minimizing speed loss.

Disclaimer

We do not hold any responsibility for any illegal usage of the codebase. Please refer to your local laws about DMCA and other related laws.

Media & Demos

🚧 Coming Soon

Video demonstrations and tutorials are currently in development.

Documentation

Quick Start

Community & Support