Towards Film-Making Production Dialogue, Narration, Monologue Adaptive Moving Dubbing Benchmarks

Movie dubbing has advanced significantly, yet assessing the real-world effectiveness of these models remains challenging. A comprehensive evaluation benchmark is crucial for two key reasons: 1) Existing metrics fail to fully capture the complexities of dialogue, narration, monologue, and actor adaptability in movie dubbing. 2) A practical evaluation system should offer valuable insights to improve movie dubbing quality and advancement in film production. To this end, we introduce Talking Adaptive Dubbing Benchmarks (TA-Dubbing), designed to improve film production by adapting to dialogue, narration, monologue, and actors in movie dubbing. TA-Dubbing offers several key advantages: 1) Comprehensive Dimensions: TA-Dubbing covers a variety of dimensions of movie dubbing, incorporating metric evaluations for both movie understanding and speech generation. 2) Versatile Benchmarking: TA-Dubbing is designed to evaluate state-of-the-art movie dubbing models and advanced multi-modal large language models. 3) Full Open-Sourcing: We fully open-source TA-Dubbing atthis https URL0a/DeepDubber- V1 including all video suits, evaluation methods, annotations. We also continuously integrate new movie dubbing models into the TA-Dubbing leaderboard atthis https URL0a/DeepDubber-V1 to drive forward the field of movie dubbing.

@article{wang2025_2505.01450,
  title={ Towards Film-Making Production Dialogue, Narration, Monologue Adaptive Moving Dubbing Benchmarks },
  author={ Chaoyi Wang and Junjie Zheng and Zihao Chen and Shiyu Xia and Chaofan Ding and Xiaohao Zhang and Xi Tao and Xiaoming He and Xinhan Di },
  journal={arXiv preprint arXiv:2505.01450},
  year={ 2025 }
}