1
0
mirror of https://github.com/fumiama/Retrieval-based-Voice-Conversion-WebUI.git synced 2026-06-07 19:40:44 +08:00

feat(audio): use PyAV instead of ffmpeg (#31)

* feat(audio): use PyAV instead of ffmpeg

replaced usage of ffmpeg in favor of PyAV (`av`)

* refactor(audio): store all of the audio related functions in the `infer.lib.audio`

refactors previous commit to have singular functions for each task, all located in `infer.lib.audio`

* fix(audio): remove downsample_audio from mdxnet.py

it is no longer needed, since it's imported from infer.lib.audio

* docs: remove every ffmpeg mention in the documentation to avoid confusion

* chore(requirements): remove ffmpeg-python and ffmpy from all requirements

* fix(audio): fix loading for UVR

wrapped gathering of META info from the stream into a function

fixes loading for UVR

* fix(audio): use np.frombuffer() instead of direct conversion of the resampled frames

this fixes traceback on preprocessing

* feat(audio): pre-allocate decoded_audio array in the load_audio function

this should improve performance, even if just a little

* Revert "docs: remove every ffmpeg mention in the documentation to avoid confusion"

This reverts commit 1e05bbce03.

* chore(format): run black on dev

* fix(requirements): revert removal of ffmpeg in unitest.yml and Dockerfile

* Revert "fix(requirements): revert removal of ffmpeg in unitest.yml and Dockerfile"

This reverts commit e28a0eebb2.

* feat(audio): pre-allocate numpy array to store the AudioFrame data in ndarray of dtype float32

* chore(format): run black on dev

* fix(audio): fix the decoded_audio size estimation

in estimated_total_samples we multiply by `sr` instead of `container.streams.audio[0].rate` since we want to estimate size of the OUTPUT file, not the input one. - Added dynamic resizing, in case something goes wrong and the size of decoded_audio is estimated incorrectly

Fixed function `load_audio` when the input audio's samplerate does not match the desired samplerate (`sr`)

* chore(format): run black on dev

* refactor(audio): remove `clean_path()` function as it serves no purpose anymore

* docs: remove everything related to ffmpeg

this includes everything except for formats support specification in the training_tips docs, since it has nothing to do with what ffmpeg does/did but rather what audio formats are supported (all the ones that ffmpeg supports!)

* docs: fix order of the steps in preparation in the READMEs

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This commit is contained in:
Alex Murkoff
2024-06-12 18:13:26 +07:00
committed by GitHub
parent aec56ec0b4
commit 1e22d468ea
28 changed files with 233 additions and 366 deletions

View File

@@ -126,27 +126,7 @@ sh ./run.sh
rvcmd assets/v2 # RVC-Models-Downloader command
```
### 2. 安装 ffmpeg 工具
若已安装`ffmpeg`和`ffprobe`则可跳过此步骤。
#### Ubuntu/Debian 用户
```bash
sudo apt install ffmpeg
```
#### MacOS 用户
```bash
brew install ffmpeg
```
#### Windows 用户
下载后放置在根目录。
```bash
rvcmd tools/ffmpeg # RVC-Models-Downloader command
```
- 下载[ffmpeg.exe](https://huggingface.co/lj1995/VoiceConversionWebUI/blob/main/ffmpeg.exe)
- 下载[ffprobe.exe](https://huggingface.co/lj1995/VoiceConversionWebUI/blob/main/ffprobe.exe)
### 3. 下载 rmvpe 人声音高提取算法所需文件
### 2. 下载 rmvpe 人声音高提取算法所需文件
如果你想使用最新的RMVPE人声音高提取算法则你需要下载音高提取模型参数并放置于`assets/rmvpe`。
@@ -162,7 +142,7 @@ rvcmd tools/ffmpeg # RVC-Models-Downloader command
rvcmd assets/rmvpe # RVC-Models-Downloader command
```
### 4. AMD显卡Rocm(可选, 仅Linux)
### 3. AMD显卡Rocm(可选, 仅Linux)
如果你想基于AMD的Rocm技术在Linux系统上运行RVC请先在[这里](https://rocm.docs.amd.com/en/latest/deploy/linux/os-native/install.html)安装所需的驱动。
@@ -207,7 +187,6 @@ rvcmd packs/general/latest # RVC-Models-Downloader command
+ [VITS](https://github.com/jaywalnut310/vits)
+ [HIFIGAN](https://github.com/jik876/hifi-gan)
+ [Gradio](https://github.com/gradio-app/gradio)
+ [FFmpeg](https://github.com/FFmpeg/FFmpeg)
+ [Ultimate Vocal Remover](https://github.com/Anjok07/ultimatevocalremovergui)
+ [audio-slicer](https://github.com/openvpi/audio-slicer)
+ [Vocal pitch extraction:RMVPE](https://github.com/Dream-High/RMVPE)