Update lora document

docs/en/finetune.md

@@ -166,11 +166,14 @@ huggingface-cli download fishaudio/speech-lm-v1 text2semantic-400m-v0.2-4k.pth -
 
 Finally, you can start the fine-tuning by running the following command:
 ```bash
-python fish_speech/train.py --config-name text2semantic_finetune_spk
+python fish_speech/train.py --config-name text2semantic_finetune
 ```
 
+!!! info
+    If you want to use lora, please use `--config-name text2semantic_finetune_lora` to start fine-tuning.
+
 !!! note
-    You can modify the training parameters such as `batch_size`, `gradient_accumulation_steps`, etc. to fit your GPU memory by modifying `fish_speech/configs/text2semantic_finetune_spk.yaml`.
+    You can modify the training parameters such as `batch_size`, `gradient_accumulation_steps`, etc. to fit your GPU memory by modifying `fish_speech/configs/text2semantic_finetune.yaml`.
 
 After training is complete, you can refer to the [inference](inference.md) section, and use `--speaker SPK1` to generate speech.
 

docs/en/index.md

@@ -33,6 +33,7 @@ pip3 install -e .
 
 ## Changelog
 
+- 2023/12/28: Added `lora` fine-tuning support.
 - 2023/12/27: Add `gradient checkpointing`, `causual sampling`, and `flash-attn` support.
 - 2023/12/19: Updated webui and HTTP API.
 - 2023/12/18: Updated fine-tuning documentation and related examples.

docs/en/inference.md

@@ -51,6 +51,10 @@ This command will create a `codes_N` file in the working directory, where N is a
 !!! info
     For GPUs that do not support bf16, you may need to use the `--half` parameter.
 
+!!! warning
+    If you are using your own fine-tuned model, please be sure to carry the `--speaker` parameter to ensure the stability of pronunciation.  
+    If you are using lora, please use `--config-name text2semantic_finetune_lora` to load the model.
+
 ### 3. Generate vocals from semantic tokens:
 ```bash
 python tools/vqgan/inference.py \

docs/zh/finetune.md

@@ -186,11 +186,14 @@ HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/speech-lm-v
 最后, 你可以运行以下命令来启动微调:
 
 ```bash
-python fish_speech/train.py --config-name text2semantic_finetune_spk
+python fish_speech/train.py --config-name text2semantic_finetune
 ```
 
 !!! note
-    你可以通过修改 `fish_speech/configs/text2semantic_finetune_spk.yaml` 来修改训练参数如 `batch_size`, `gradient_accumulation_steps` 等, 来适应你的显存.
+    如果你想使用 lora, 请使用 `--config-name text2semantic_finetune_lora` 来启动微调.
+
+!!! note
+    你可以通过修改 `fish_speech/configs/text2semantic_finetune.yaml` 来修改训练参数如 `batch_size`, `gradient_accumulation_steps` 等, 来适应你的显存.
 
 训练结束后, 你可以参考 [推理](inference.md) 部分, 并携带 `--speaker SPK1` 参数来测试你的模型.
 

docs/zh/index.md

@@ -33,6 +33,7 @@ pip3 install -e .
 
 ## 更新日志
 
+- 2023/12/28: 添加了 `lora` 微调支持.
 - 2023/12/27: 添加了 `gradient checkpointing`, `causual sampling` 和 `flash-attn` 支持.
 - 2023/12/19: 更新了 Webui 和 HTTP API.
 - 2023/12/18: 更新了微调文档和相关例子.

docs/zh/inference.md

@@ -57,7 +57,8 @@ python tools/llama/generate.py \
     对于不支持 bf16 的 GPU, 你可能需要使用 `--half` 参数.
 
 !!! warning
-    如果你在使用自己微调的模型, 请务必携带 `--speaker` 参数来保证发音的稳定性.
+    如果你在使用自己微调的模型, 请务必携带 `--speaker` 参数来保证发音的稳定性.  
+    如果你使用了 lora, 请使用 `--config-name text2semantic_finetune_lora` 来加载模型.
 
 ### 3. 从语义 token 生成人声: 
 ```bash

pyproject.toml

@@ -34,6 +34,7 @@ dependencies = [
     "zibai-server>=0.9.0",
     "loguru>=0.6.0",
     "WeTextProcessing>=0.1.10",
+    "loralib>=0.1.2"
 ]
 
 [project.optional-dependencies]