Speech Resynthesis from Discrete Disentangled Self-Supervised Representations

1 April 2021

Yossi Adi

Papers citing "Speech Resynthesis from Discrete Disentangled Self-Supervised Representations"

50 / 79 papers shown

Title
Voila: Voice-Language Foundation Models for Real-Time Autonomous Interaction and Voice Role-Play Yemin Shi Yu Shu Siwei Dong Guangyi Liu Jaward Sesay Jingwen Li Zhiting Hu AuLLM VLM 50 0 0 05 May 2025
Mitigating Timbre Leakage with Universal Semantic Mapping Residual Block for Voice Conversion Na Li Chuke Wang Yu Gu Zhifeng Li 59 0 0 11 Apr 2025
A Unit-based System and Dataset for Expressive Direct Speech-to-Speech Translation Anna Min Chenxu Hu Yi Ren Hang Zhao 66 0 0 01 Feb 2025
When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation Anna Min Chenxu Hu Yi Ren Hang Zhao 74 0 0 01 Feb 2025
Towards Unsupervised Speech Recognition Without Pronunciation Models Junrui Ni Liming Wang Yang Zhang Kaizhi Qian Heting Gao Mark Hasegawa-Johnson Chang D. Yoo SSL OffRL 94 0 0 10 Jan 2025
Bird Vocalization Embedding Extraction Using Self-Supervised Disentangled Representation Learning Runwu Shi Katsutoshi Itoyama K. Nakadai SSL DRL 46 1 0 31 Dec 2024
DiFiC: Your Diffusion Model Holds the Secret to Fine-Grained Clustering Ruohong Yang Peng Hu Xi Peng Xiting Liu Yunfan Li 39 0 0 25 Dec 2024
Sylber: Syllabic Embedding Representation of Speech from Raw Audio Cheol Jun Cho Nicholas Lee Akshat Gupta Dhruv Agarwal Ethan Chen Alan W Black Gopala K. Anumanchipalli 34 0 0 09 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 61 17 0 01 Oct 2024
Learning Source Disentanglement in Neural Audio Codec Xiaoyu Bie Xubo Liu Gaël Richard 34 1 0 17 Sep 2024
Salmon: A Suite for Acoustic Language Model Evaluation Gallil Maimon Amit Roth Yossi Adi ELM AuLLM 66 5 0 11 Sep 2024
LLaMA-Omni: Seamless Speech Interaction with Large Language Models Qingkai Fang Shoutao Guo Yan Zhou Zhengrui Ma Shaolei Zhang Yang Feng AuLLM 33 34 0 10 Sep 2024
A Benchmark for Multi-speaker Anonymization Xiaoxiao Miao Ruijie Tao Chang Zeng Xin Wang 49 1 0 08 Jul 2024
How Should We Extract Discrete Audio Tokens from Self-Supervised Models? Pooneh Mousavi J. Duret Salah Zaiem Luca Della Libera Artem Ploujnikov Cem Subakan Mirco Ravanelli 42 10 0 15 Jun 2024
The Interspeech 2024 Challenge on Speech Processing Using Discrete Units Xuankai Chang Jiatong Shi Jinchuan Tian Yuning Wu Yuxun Tang Yihan Wu Shinji Watanabe Yossi Adi Xie Chen Qin Jin 49 16 0 11 Jun 2024
A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Any Translation Zhengrui Ma Qingkai Fang Shaolei Zhang Shoutao Guo Yang Feng Min Zhang 53 10 0 11 Jun 2024
Fill in the Gap! Combining Self-supervised Representation Learning with Neural Audio Synthesis for Speech Inpainting Ihab Asaad Maxime Jacquelin Olivier Perrotin Laurent Girin Thomas Hueber 33 0 0 30 May 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 56 1 0 16 Apr 2024
VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editing Philip Anastassiou Zhenyu Tang Kainan Peng Dongya Jia Jiaxin Li Ming Tu Yuping Wang Yuxuan Wang Mingbo Ma 42 4 0 10 Apr 2024
Towards audio language modeling -- an overview Haibin Wu Xuanjun Chen Yi-Cheng Lin Kai-Wei Chang Ho-Lam Chung Alexander H. Liu Hung-yi Lee AuLLM 38 28 0 20 Feb 2024
SEF-VC: Speaker Embedding Free Zero-Shot Voice Conversion with Cross Attention Junjie Li Yiwei Guo Xie Chen Kai Yu 47 13 0 14 Dec 2023
OpenVoice: Versatile Instant Voice Cloning Zengyi Qin Wenliang Zhao Xumin Yu Xin Sun VLM 37 20 0 03 Dec 2023
Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation Haram Choi Sang-Hoon Lee Seong-Whan Lee DiffM 34 24 0 08 Nov 2023
SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERT Cheol Jun Cho Abdelrahman Mohamed Shang-Wen Li Alan W. Black Gopala K. Anumanchipalli 39 8 0 16 Oct 2023
Toward Joint Language Modeling for Speech Units and Text Ju-Chieh Chou Chung-Ming Chien Wei-Ning Hsu Karen Livescu Arun Babu Alexis Conneau Alexei Baevski Michael Auli VLM 28 20 0 12 Oct 2023
Few-Shot Spoken Language Understanding via Joint Speech-Text Models Chung-Ming Chien Mingjiamei Zhang Ju-Chieh Chou Karen Livescu 36 3 0 09 Oct 2023
Lip Reading for Low-resource Languages by Learning and Combining General Speech Knowledge and Language-specific Knowledge Minsu Kim Jeong Hun Yeo J. Choi Y. Ro 34 16 0 18 Aug 2023
HierVST: Hierarchical Adaptive Zero-shot Voice Style Transfer Sang-Hoon Lee Haram Choi H. Oh Seong-Whan Lee BDL 30 9 0 30 Jul 2023
Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis Hubert Siuzdak 37 79 0 01 Jun 2023
Intelligible Lip-to-Speech Synthesis with Speech Units J. Choi Minsu Kim Y. Ro 32 24 0 31 May 2023
Speaker anonymization using orthogonal Householder neural network Xiaoxiao Miao Xin Wang Erica Cooper Junichi Yamagishi N. Tomashenko BDL 26 18 0 30 May 2023
Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target Guanyong Wu Guan-Ting Lin Shang-Wen Li Hung-yi Lee 29 5 0 29 May 2023
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation Rongjie Huang Huadai Liu Xize Cheng Yi Ren Lin Li ... Jinzheng He Lichao Zhang Jinglin Liu Xiaoyue Yin Zhou Zhao 78 8 0 24 May 2023
Textually Pretrained Speech Language Models Michael Hassid Tal Remez Tu Nguyen Itai Gat Alexis Conneau ... Alexandre Défossez Gabriel Synnaeve Emmanuel Dupoux Roy Schwartz Yossi Adi VLM SyDa 44 54 0 22 May 2023
Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspaces Oli Danyi Liu Hao Tang Sharon Goldwater SSL 33 12 0 21 May 2023
MParrotTTS: Multilingual Multi-speaker Text to Speech Synthesis in Low Resource Setting Neil Shah Vishal Tambrahalli Saiteja Kosgi N. Pedanekar Vineet Gandhi 44 0 0 19 May 2023
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities Dong Zhang Shimin Li Xin Zhang Jun Zhan Pengyu Wang Yaqian Zhou Xipeng Qiu AuLLM MLLM 62 302 0 18 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 40 13 0 15 May 2023
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit Brian Yan Jiatong Shi Yun Tang Hirofumi Inaguma Yifan Peng ... Zhaoheng Ni Moto Hira Soumi Maiti J. Pino Shinji Watanabe 21 20 0 10 Apr 2023
Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representations Yuma Koizumi Heiga Zen Shigeki Karita Yifan Ding Kohei Yatabe Nobuyuki Morioka Yu Zhang Wei Han Ankur Bapna M. Bacchiani 39 22 0 03 Mar 2023
PITS: Variational Pitch Inference without Fundamental Frequency for End-to-End Pitch-controllable TTS Junhyeok Lee Wonbin Jung Hyunjae Cho Jaeyeon Kim Jaehwan Kim 22 3 0 24 Feb 2023
ACE-VC: Adaptive and Controllable Voice Conversion using Explicitly Disentangled Self-supervised Speech Representations Shehzeen Samarah Hussain Paarth Neekhara Jocelyn Huang Jason Chun Lok Li Boris Ginsburg 13 21 0 16 Feb 2023
Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision Eugene Kharitonov Damien Vincent Zalan Borsos Raphaël Marinier Sertan Girgin Olivier Pietquin Matthew Sharifi Marco Tagliasacchi Neil Zeghidour 24 191 0 07 Feb 2023
Unearthing InSights into Mars: Unsupervised Source Separation with Limited Data Ali Siahkoohi Rudy Morel Maarten V. de Hoop Erwan Allys G. Sainton Taichi Kawamura 42 4 0 27 Jan 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers Chengyi Wang Sanyuan Chen Yu-Huan Wu Zi-Hua Zhang Long Zhou ... Huaming Wang Jinyu Li Lei He Sheng Zhao Furu Wei 48 654 0 05 Jan 2023
ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement Wei-Ning Hsu Tal Remez Bowen Shi Jacob Donley Yossi Adi DiffM 27 12 0 21 Dec 2022
Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Units Gallil Maimon Yossi Adi 34 13 0 19 Dec 2022
Voice-preserving Zero-shot Multiple Accent Conversion Mumin Jin Prashant Serai Jilong Wu Andros Tjandra Vimal Manohar Qing He 19 12 0 23 Nov 2022
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations Paul-Ambroise Duquenne Hongyu Gong Ning Dong Jingfei Du Ann Lee Vedanuj Goswani Changhan Wang J. Pino Benoît Sagot Holger Schwenk 45 34 0 08 Nov 2022
Audio Language Modeling using Perceptually-Guided Discrete Representations Felix Kreuk Yaniv Taigman Adam Polyak Jade Copet Gabriel Synnaeve Alexandre Défossez Yossi Adi 32 4 0 02 Nov 2022