BEATs: Audio Pre-Training with Acoustic Tokenizers

18 December 2022

Papers citing "BEATs: Audio Pre-Training with Acoustic Tokenizers"

50 / 56 papers shown

Title
Assessing the Utility of Audio Foundation Models for Heart and Respiratory Sound Analysis Daisuke Niizumi Daiki Takeuchi Masahiro Yasuda Binh Thien Nguyen Yasunori Ohishi N. Harada 32 0 0 25 Apr 2025
Temporal Attention Pooling for Frequency Dynamic Convolution in Sound Event Detection Hyeonuk Nam Yong-Hwa Park 31 0 0 17 Apr 2025
Can Masked Autoencoders Also Listen to Birds? Lukas Rauch Ilyass Moummad René Heinrich Alexis Joly Bernhard Sick Christoph Scholz 29 0 0 17 Apr 2025
SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction Fine-Tuning Prabhat Pandey R. Swaminathan K V Vijay Girish Arunasish Sen Jian Xie Grant P. Strimel Andreas Schwarz 116 0 0 12 Apr 2025
Formula-Supervised Sound Event Detection: Pre-Training Without Real Data Yuto Shibata Keitaro Tanaka Yoshiaki Bando Keisuke Imoto Hirokatsu Kataoka Yoshimitsu Aoki 26 0 0 06 Apr 2025
QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions Siyin Wang Wenyi Yu Xianzhao Chen Xiaohai Tian J. Zhang Lu Lu Yu Tsao Junichi Yamagishi Y. Wang Chao Zhang AuLLM 76 0 0 26 Mar 2025
Toward Foundation Models for Online Complex Event Detection in CPS-IoT: A Case Study Liying Han Gaofeng Dong Xiaomin Ouyang Lance M. Kaplan Federico Cerutti Mani B. Srivastava 55 0 0 15 Mar 2025
KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation Antoni Bigata Michał Stypułkowski Rodrigo Mira Stella Bounareli Konstantinos Vougioukas Zoe Landgraf Nikita Drobyshev Maciej Ziȩba Stavros Petridis M. Pantic DiffM VGen 65 2 0 03 Mar 2025
AAD-LLM: Neural Attention-Driven Auditory Scene Understanding Xilin Jiang Sukru Samet Dindar Vishal B. Choudhari Stephan Bickel A. Mehta Guy M McKhann A. Flinker D. Friedman N. Mesgarani 32 2 0 24 Feb 2025
How do Multimodal Foundation Models Encode Text and Speech? An Analysis of Cross-Lingual and Cross-Modal Representations Hyunji Lee Danni Liu Supriti Sinhamahapatra Jan Niehues 106 0 0 21 Feb 2025
Leveraging Broadcast Media Subtitle Transcripts for Automatic Speech Recognition and Subtitling Jakob Poncelet Hugo Van hamme 69 0 0 05 Feb 2025
SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words Junyi Ao Yuancheng Wang Xiaohai Tian Dekun Chen J. Zhang Lu Lu Y. Wang Haizhou Li Z. Wu AuLLM 80 17 0 17 Jan 2025
HAAQI-Net: A Non-intrusive Neural Music Audio Quality Assessment Model for Hearing Aids Dyah A. M. G. Wisnu Stefano Rini Ryandhimas E. Zezario Hsin-Min Wang Yu Tsao 52 0 0 10 Jan 2025
Do Language Models Understand Time? Xi Ding Lei Wang 178 0 0 18 Dec 2024
Gramian Multimodal Representation Learning and Alignment Giordano Cicchetti Eleonora Grassucci Luigi Sigillo Danilo Comminiello 91 0 0 16 Dec 2024
Gotta Hear Them All: Sound Source Aware Vision to Audio Generation Wei Guo Heng Wang Jianbo Ma Weidong Cai DiffM 90 3 0 23 Nov 2024
StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identification Yichen He Yuan Lin Jianchao Wu Hanchong Zhang Yuchen Zhang Ruicheng Le VGen VLM 142 2 0 11 Nov 2024
Leveraging LLM and Text-Queried Separation for Noise-Robust Sound Event Detection Han Yin Yang Xiao Jisheng Bai Rohan Kumar Das 31 0 0 02 Nov 2024
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models Kim Sung-Bin Oh Hyun-Bin JungMok Lee Arda Senocak Joon Son Chung Tae-Hyun Oh MLLM VLM 40 3 0 23 Oct 2024
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 59 14 0 01 Oct 2024
MT2KD: Towards A General-Purpose Encoder for Speech, Speaker, and Audio Events Xiaoyu Yang Qiujia Li Chao Zhang P. Woodland 18 0 0 25 Sep 2024
Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation Siyin Wang Wenyi Yu Yudong Yang Changli Tang Yixuan Li ... Jun Zhang Guangzhi Sun Lu Lu Yuxuan Wang Chao Zhang AuLLM LM&MA 67 5 0 25 Sep 2024
OmniBench: Towards The Future of Universal Omni-Language Models Yizhi Li Ge Zhang Yinghao Ma Ruibin Yuan Kang Zhu ... Zhaoxiang Zhang Zachary Liu Emmanouil Benetos Wenhao Huang Chenghua Lin LRM 44 11 0 23 Sep 2024
A Comprehensive Survey with Critical Analysis for Deepfake Speech Detection Lam Pham Phat Lam Dat Tran Hieu Tang Tin Nguyen Alexander Schindler Canh Vu Alexander Polonsky Canh Vu 46 3 0 23 Sep 2024
What Are They Doing? Joint Audio-Speech Co-Reasoning Yingzhi Wang Pooneh Mousavi Artem Ploujnikov Mirco Ravanelli AuLLM 44 0 0 22 Sep 2024
Effective Pre-Training of Audio Transformers for Sound Event Detection Florian Schmid T. Morocutti Francesco Foscarin Jan Schluter Paul Primus Gerhard Widmer ViT 23 2 0 14 Sep 2024
MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders W. Zhang Shuo Sun Bin Wang Xunlong Zou Zhuohan Liu Yingxu He Geyu Lin Nancy F. Chen A. Aw AuLLM 67 1 0 10 Sep 2024
ICSD: An Open-source Dataset for Infant Cry and Snoring Detection Qingyu Liu Longfei Song Dongxing Xu Yanhua Long 37 0 0 20 Aug 2024
Improving Audio Spectrogram Transformers for Sound Event Detection Through Multi-Stage Training Florian Schmid Paul Primus T. Morocutti Jonathan Greif Gerhard Widmer 32 5 0 17 Jul 2024
Sequential Contrastive Audio-Visual Learning Ioannis Tsiamas Santiago Pascual Chunghsin Yeh Joan Serra 35 2 0 08 Jul 2024
AND: Audio Network Dissection for Interpreting Deep Acoustic Models Tung-Yu Wu Yu-Xiang Lin Tsui-Wei Weng 50 1 0 24 Jun 2024
AnoPatch: Towards Better Consistency in Machine Anomalous Sound Detection Anbai Jiang Bing Han Zhiqiang Lv Yufeng Deng Wei-Qiang Zhang Xie Chen Yanmin Qian Jia Liu Pingyi Fan 32 3 0 17 Jun 2024
GameVibe: A Multimodal Affective Game Corpus M. Barthet Maria Kaselimi Kosmas Pinitas Konstantinos Makantasis Antonios Liapis Georgios N. Yannakakis 30 3 0 17 Jun 2024
DCASE 2024 Task 4: Sound Event Detection with Heterogeneous Data and Missing Labels Samuele Cornell Janek Ebbers Constance Douwes Irene Martín-Morató Manu Harju A. Mesaros Romain Serizel 32 13 0 12 Jun 2024
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts Yunxin Li Shenyuan Jiang Baotian Hu Longyue Wang Wanqi Zhong Wenhan Luo Lin Ma Min-Ling Zhang MoE 39 28 0 18 May 2024
uaMix-MAE: Efficient Tuning of Pretrained Audio Transformers with Unsupervised Audio Mixtures Afrina Tabassum Dung N. Tran Trung D. Q. Dang Ismini Lourentzou K. Koishida 47 0 0 14 Mar 2024
CREMA: Generalizable and Efficient Video-Language Reasoning via Multimodal Modular Fusion Shoubin Yu Jaehong Yoon Mohit Bansal 77 4 0 08 Feb 2024
Audio-Visual Instance Segmentation Ruohao Guo Yaru Chen Yanyu Qi Wenzhen Yue Dantong Niu ... Wenzhen Yue Ji Shi Qixun Wang Peiliang Zhang Buwen Liang VLM VOS 28 2 0 28 Oct 2023
SALMONN: Towards Generic Hearing Abilities for Large Language Models Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang LM&MA AuLLM 35 199 0 20 Oct 2023
Cross-modal Cognitive Consensus guided Audio-Visual Segmentation Zhaofeng Shi Qingbo Wu Fanman Meng Linfeng Xu Hongliang Li VOS 30 3 0 10 Oct 2023
Efficient Supervised Training of Audio Transformers for Music Representation Learning Pablo Alonso-Jiménez Xavier Serra Dmitry Bogdanov ViT 35 3 0 28 Sep 2023
Semantic Proximity Alignment: Towards Human Perception-consistent Audio Tagging by Aligning with Label Text Description Youbin Jeon Yanzhen Ren VLM 26 0 0 28 Sep 2023
Exploring Self-Supervised Contrastive Learning of Spatial Sound Event Representation Xilin Jiang Cong Han Yinghao Aaron Li N. Mesgarani SSL 23 1 0 27 Sep 2023
AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codes Zhaohui Li Haitao Wang Xinghua Jiang 40 1 0 14 Aug 2023
Noise-aware Speech Enhancement using Diffusion Probabilistic Model Yuchen Hu Cheng Chen Ruizhe Li Qiu-shi Zhu E. Chng DiffM 16 9 0 16 Jul 2023
A Multimodal Prototypical Approach for Unsupervised Sound Classification Saksham Singh Kushwaha Magdalena Fuentes 22 8 0 21 Jun 2023
Streaming Audio Transformers for Online Audio Tagging Heinrich Dinkel Zhiyong Yan Yongqing Wang Junbo Zhang Yujun Wang Bin Wang 28 4 0 29 May 2023
Pengi: An Audio Language Model for Audio Tasks Soham Deshmukh Benjamin Elizalde Rita Singh Huaming Wang MLLM AuLLM 34 157 0 19 May 2023
MMViT: Multiscale Multiview Vision Transformers Yuchen Liu Natasha Ong Kaiyan Peng Bo Xiong Qifan Wang ... Madian Khabsa Kaiyue Yang David C. Liu Donald Williamson Hanchao Yu ViT 22 4 0 28 Apr 2023
HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection Ke Chen Xingjian Du Bilei Zhu Zejun Ma Taylor Berg-Kirkpatrick Shlomo Dubnov ViT 118 264 0 02 Feb 2022