VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translation

25 May 2023

Papers citing "VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translation"

24 / 24 papers shown

Title
Continuous Speech Tokenizer in Text To Speech Yixing Li Ruobing Xie Xingchen Sun Yu Cheng Zhanhui Kang AuLLM CLL 63 2 0 22 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 73 3 0 20 Oct 2024
F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching Yushen Chen Zhikang Niu Ziyang Ma Keqi Deng Chunhui Wang Jian Zhao Kai Yu Xie Chen 35 55 0 09 Oct 2024
LLaMA-Omni: Seamless Speech Interaction with Large Language Models Qingkai Fang Shoutao Guo Yan Zhou Zhengrui Ma Shaolei Zhang Yang Feng AuLLM 33 32 0 10 Sep 2024
MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders Feiyu Xiong Shuo Sun Bin Wang Xunlong Zou Zhuohan Liu Yingxu He Geyu Lin Nancy F. Chen Ai Ti Aw AuLLM 67 1 0 10 Sep 2024
Language Model Can Listen While Speaking Ziyang Ma Yakun Song Chenpeng Du Jian Cong Zhuo Chen Yuping Wang Yansen Wang Xie Chen AuLLM 50 24 0 05 Aug 2024
dMel: Speech Tokenization made Simple Richard He Bai Tatiana Likhomanenko Ruixiang Zhang Zijin Gu Zakaria Aldeneh Navdeep Jaitly 48 4 0 22 Jul 2024
Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization Yuchen Hu Chen Chen Siyin Wang Eng Siong Chng C. Zhang 48 3 0 02 Jul 2024
AudioBench: A Universal Benchmark for Audio Large Language Models Bin Wang Xunlong Zou Geyu Lin Shri Kiran Srinivasan Zhuohan Liu Wenyu Zhang Zhengyuan Liu AiTi Aw Nancy F. Chen AuLLM ELM LM&MA 92 23 0 23 Jun 2024
How Should We Extract Discrete Audio Tokens from Self-Supervised Models? Pooneh Mousavi J. Duret Salah Zaiem Luca Della Libera Artem Ploujnikov Cem Subakan Mirco Ravanelli 42 10 0 15 Jun 2024
PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Models Runyan Yang Huibao Yang Xiqing Zhang Tiantian Ye Ying Liu Yingying Gao Shilei Zhang Chao Deng Junlan Feng 36 0 0 12 Jun 2024
Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model Siyang Wang Éva Székely 47 4 0 16 May 2024
SpeechVerse: A Large-scale Generalizable Audio Language Model Nilaksh Das Saket Dingliwal S. Ronanki Rohit Paturi David Huang ... Monica Sunkara S. Srinivasan Kyu J. Han Katrin Kirchhoff Katrin Kirchhoff 41 38 0 14 May 2024
VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editing Philip Anastassiou Zhenyu Tang Kainan Peng Dongya Jia Jiaxin Li Ming Tu Yuping Wang Yuxuan Wang Mingbo Ma 42 4 0 10 Apr 2024
Towards audio language modeling -- an overview Haibin Wu Xuanjun Chen Yi-Cheng Lin Kai-Wei Chang Ho-Lam Chung Alexander H. Liu Hung-yi Lee AuLLM 38 28 0 20 Feb 2024
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations Guan-Ting Lin Cheng-Han Chiang Hung-yi Lee 34 24 0 20 Feb 2024
OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identification Yifan Peng Yui Sudo Muhammad Shakeel Shinji Watanabe VLM 46 17 0 20 Feb 2024
Not My Voice! A Taxonomy of Ethical and Safety Harms of Speech Generators Wiebke Hutiri Orestis Papakyriakopoulos Alice Xiang 36 16 0 25 Jan 2024
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models Yunfei Chu Jin Xu Xiaohuan Zhou Qian Yang Shiliang Zhang Zhijie Yan Chang Zhou Jingren Zhou AuLLM 42 280 0 14 Nov 2023
Few-Shot Spoken Language Understanding via Joint Speech-Text Models Chung-Ming Chien Mingjiamei Zhang Ju-Chieh Chou Karen Livescu 34 3 0 09 Oct 2023
Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition Krishna C. Puvvada Nithin Rao Koluguri Kunal Dhawan Jagadeesh Balam Boris Ginsburg 34 13 0 19 Sep 2023
FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codec Zhihao Du Shiliang Zhang Kai Hu Siqi Zheng 34 55 0 14 Sep 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers Chengyi Wang Sanyuan Chen Yu-Huan Wu Zi-Hua Zhang Long Zhou ... Huaming Wang Jinyu Li Lei He Sheng Zhao Furu Wei 48 648 0 05 Jan 2023
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing Junyi Ao Rui Wang Long Zhou Chengyi Wang Shuo Ren ... Yu Zhang Zhihua Wei Yao Qian Jinyu Li Furu Wei 118 193 0 14 Oct 2021