OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation

1 July 2021

Jing Liu

Papers citing "OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation"

22 / 22 papers shown

Title
Pedestrian Attribute Recognition: A New Benchmark Dataset and A Large Language Model Augmented Framework Jiandong Jin Xiao Wang Qian Zhu Haiyang Wang Chenglong Li VLM 31 4 0 19 Aug 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 46 8 0 13 Apr 2024
CoAVT: A Cognition-Inspired Unified Audio-Visual-Text Pre-Training Model for Multimodal Processing Xianghu Yue Xiaohai Tian Lu Lu Malu Zhang Zhizheng Wu Haizhou Li 39 0 0 22 Jan 2024
Detecting Multimedia Generated by Large AI Models: A Survey Li Lin Neeraj Gupta Yue Zhang Hainan Ren Chun-Hao Liu Feng Ding Xin Wang Xin Li Luisa Verdoliva Shu Hu 88 58 0 22 Jan 2024
OmniVec: Learning robust representations with cross modal sharing Siddharth Srivastava Gaurav Sharma SSL 37 64 0 07 Nov 2023
Conversational Speech Recognition by Learning Audio-textual Cross-modal Contextual Representation Kun Wei Bei Li Hang Lv Quan Lu Ning Jiang Lei Xie 44 3 0 22 Oct 2023
SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training Kazem Meidani Parshin Shojaee Chandan K. Reddy A. Farimani 26 18 0 03 Oct 2023
At Which Training Stage Does Code Data Help LLMs Reasoning? Xiaogang Jia Yue Liu Yue Yu Yuanliang Zhang Yu Jiang Changjian Wang Shanshan Li LRM SyDa 24 59 0 28 Sep 2023
EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE Junyi Chen Longteng Guo Jianxiang Sun Shuai Shao Zehuan Yuan Liang Lin Dongyu Zhang MLLM VLM MoE 60 9 0 23 Aug 2023
LANISTR: Multimodal Learning from Structured and Unstructured Data Sayna Ebrahimi Sercan Ö. Arik Yihe Dong Tomas Pfister 20 4 0 26 May 2023
LLMDet: A Third Party Large Language Models Generated Text Detection Tool Kangxi Wu Liang Pang Huawei Shen Xueqi Cheng Tat-Seng Chua DeLMO 35 38 0 24 May 2023
Self-Supervised Multimodal Learning: A Survey Yongshuo Zong Oisin Mac Aodha Timothy M. Hospedales SSL 24 44 0 31 Mar 2023
Accommodating Audio Modality in CLIP for Multimodal Processing Ludan Ruan Anwen Hu Yuqing Song Liang Zhang S. Zheng Qin Jin VLM 24 10 0 12 Mar 2023
Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey Tianlin Li Guangyao Chen Guangwu Qian Pengcheng Gao Xiaoyong Wei Yaowei Wang Yonghong Tian Wen Gao AI4CE VLM 48 204 0 20 Feb 2023
VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning Qiu-shi Zhu Long Zhou Zi-Hua Zhang Shujie Liu Binxing Jiao Jie Zhang Lirong Dai Daxin Jiang Jinyu Li Furu Wei 33 37 0 21 Nov 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 46 55 0 20 Aug 2022
Leveraging Acoustic Contextual Representation by Audio-textual Cross-modal Learning for Conversational ASR Kun Wei Yike Zhang Sining Sun Lei Xie Long Ma 26 9 0 03 Jul 2022
HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrieval Feilong Chen Xiuyi Chen Jiaxin Shi Duzhen Zhang Jianlong Chang Qi Tian VLM CLIP 34 6 0 24 May 2022
VLP: A Survey on Vision-Language Pre-training Feilong Chen Duzhen Zhang Minglun Han Xiuyi Chen Jing Shi Shuang Xu Bo Xu VLM 82 213 0 18 Feb 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 45 207 0 07 Jan 2022
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 255 4,796 0 24 Feb 2021
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 252 927 0 24 Sep 2019