Spoken Moments: Learning Joint Audio-Visual Representations from Video
Descriptions

Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions

10 May 2021

Alexander H. Liu

Papers citing "Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions"

18 / 18 papers shown

Title
VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling Xinhao Li Yi Wang Jiashuo Yu Xiangyu Zeng Yuhan Zhu ... Yinan He Chenting Wang Yu Qiao Yali Wang L. Wang VLM 89 26 0 31 Dec 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 89 26 0 04 Oct 2024
Tarsier: Recipes for Training and Evaluating Large Video Description Models Jiawei Wang Liping Yuan Yuchen Zhang 49 52 0 30 Jun 2024
Enhancing Multimodal Unified Representations for Cross Modal Generalization Hai Huang Yan Xia Shengpeng Ji Shulei Wang Hanting Wang Minghui Fang Jieming Zhu Zhenhua Dong Sashuai Zhou Zhou Zhao 42 6 0 08 Mar 2024
VideoPrism: A Foundational Visual Encoder for Video Understanding Long Zhao N. B. Gundavarapu Liangzhe Yuan Hao Zhou Shen Yan ... Huisheng Wang Hartwig Adam Mikhail Sirotenko Ting Liu Boqing Gong VGen 50 29 0 20 Feb 2024
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang ... Jilan Xu Guo Chen Ping Luo Limin Wang Yu Qiao VLM MLLM 87 413 0 28 Nov 2023
Learning Human Action Recognition Representations Without Real Humans Howard Zhong Samarth Mishra Donghyun Kim SouYoung Jin Yikang Shen Hildegard Kuehne Leonid Karlinsky Venkatesh Saligrama Aude Oliva Rogerio Feris 29 3 0 10 Nov 2023
PaLI-3 Vision Language Models: Smaller, Faster, Stronger Xi Chen Xiao Wang Lucas Beyer Alexander Kolesnikov Jialin Wu ... Keran Rong Tianli Yu Daniel Keysers Xiao-Qi Zhai Radu Soricut MLLM VLM 41 94 0 13 Oct 2023
PaLI-X: On Scaling up a Multilingual Vision and Language Model Xi Chen Josip Djolonga Piotr Padlewski Basil Mustafa Soravit Changpinyo ... Mojtaba Seyedhosseini A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut VLM 76 190 0 29 May 2023
Deep Neural Networks in Video Human Action Recognition: A Review Zihan Wang Yang Yang Zhi Liu Y. Zheng 61 4 0 25 May 2023
Connecting Vision and Language with Video Localized Narratives P. Voigtlaender Soravit Changpinyo Jordi Pont-Tuset Radu Soricut V. Ferrari VGen 54 21 0 22 Feb 2023
UAVM: Towards Unifying Audio and Visual Models Yuan Gong Alexander H. Liu Andrew Rouditchenko James R. Glass 33 21 0 29 Jul 2022
What's in a Caption? Dataset-Specific Linguistic Diversity and Its Effect on Visual Description Models and Metrics David M. Chan Austin Myers Sudheendra Vijayanarasimhan David A. Ross Bryan Seybold John F. Canny 33 6 0 12 May 2022
i-Code: An Integrative and Composable Multimodal Learning Framework Ziyi Yang Yuwei Fang Chenguang Zhu Reid Pryzant DongDong Chen ... Bin Xiao Yuanxun Lu Takuya Yoshioka Michael Zeng Xuedong Huang 40 46 0 03 May 2022
Cascaded Multilingual Audio-Visual Learning from Videos Andrew Rouditchenko Angie Boggust David Harwath Samuel Thomas Hilde Kuehne ... Yikang Shen Rogerio Feris Brian Kingsbury M. Picheny James R. Glass 143 8 0 08 Nov 2021
Spoken ObjectNet: A Bias-Controlled Spoken Caption Dataset Ian Palmer Andrew Rouditchenko Andrei Barbu Boris Katz James R. Glass 11 4 0 14 Oct 2021
Contrastive Attraction and Contrastive Repulsion for Representation Learning Huangjie Zheng Xu Chen Jiangchao Yao Hongxia Yang Chunyuan Li Ya Zhang Hao Zhang Ivor Tsang Jingren Zhou Mingyuan Zhou SSL 42 12 0 08 May 2021
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos Andrew Rouditchenko Angie Boggust David Harwath Brian Chen D. Joshi ... Rogerio Feris Brian Kingsbury M. Picheny Antonio Torralba James R. Glass SSL 22 141 0 16 Jun 2020