43
v1v2 (latest)

Woosh: A Sound Effects Foundation Model

Gaëtan Hadjeres
Marc Ferras
Khaled Koutini
Benno Weck
Alexandre Bittar
Thomas Hummel
Zineb Lahrici
Hakim Missoum
Joan Serrà
Yuki Mitsufuji
Main:15 Pages
5 Figures
Bibliography:4 Pages
9 Tables
Appendix:2 Pages
Abstract

The audio research community depends on open generative models as foundational tools for building novel approaches and establishing baselines. In this report, we present Woosh, Sony AI's publicly released sound effect foundation model, detailing its architecture, training process, and an evaluation against other popular open models. Being optimized for sound effects, we provide (1) a high-quality audio encoder/decoder model and (2) a text-audio alignment model for conditioning, together with (3) text-to-audio and (4) video-to-audio generative models. Distilled text-to-audio and video-to-audio models are also included in the release, allowing for low-resource operation and fast inference. Our evaluation on both public and private data shows competitive or better performance for each module when compared to existing open alternatives like StableAudio-Open and TangoFlux. Inference code and model weights are available atthis https URL. Demo samples can be found atthis https URL.

View on arXiv
Comments on this paper