替影片配上另一種語言的聲音,常得分別處理字幕、配音和時間對齊。
- 社群開發者做了 VoxaDub,能先轉錄對話並對齊時間,再用 Hy-MT2 翻譯,接著以 VoxCPM2 參考各說話者的乾淨音訊來生成配音,並調整語句長度以配合畫面。完成後可把配音和原本的背景音合在一起,也能另外選擇唇形同步。模型設定完成後,配音可全程在本機執行。
- 直播配音仍在實驗階段,時間調整有助於對齊,但不保證完全同步。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
🎬 A new language. A familiar voice.
Community developer @kadirrbrtt built VoxaDub, a local-first video dubbing tool that brings VoxCPM2 voice cloning into a multilingual workflow. 🌍
From source video to dubbed output:
📝 Transcribe & align: Transcribe dialogue and align timestamps, with optional speaker diarization to distinguish who speaks when.
🧠 Translate: Local Hy-MT2 translation uses surrounding context and optional terminology.
🎙️ Generate voices: Selected clean audio clips from each speaker guide VoxCPM2 synthesis, helping preserve distinct voice characteristics in the translated dialogue.
⏱️ Fit the timing: Duration-aware translation refinement and bounded audio adjustments help fit speech into the original scene.
🎞️ Rebuild the video: Combine dubbed speech with the original background audio. Optional lip-sync is available as a separate step.
🔒 Offline dubbing can run fully locally after model setup, with optional cloud compute choices.
⚡ VoxaDub also explores live broadcast dubbing, currently experimental. Timing safeguards help improve alignment without promising perfect synchronization.
We love seeing our community turn VoxCPM2’s expressive speech generation into tools that help stories reach new audiences. 💙
🛠️ Explore VoxaDub and its multilingual demos:
https://t.co/8YaTwW5OWI
🚀 VoxCPM GitHub:
https://t.co/VjpJxoyoLr
🤗 VoxCPM2 on Hugging Face:
https://t.co/NE1t03jOXz