Survey: Transformer-based Models in Data Modality Conversion

8 August 2024

Papers citing "Survey: Transformer-based Models in Data Modality Conversion"

50 / 74 papers shown

Title
Towards Practical and Efficient Image-to-Speech Captioning with Vision-Language Pre-training and Multi-modal Tokens Minsu Kim J. Choi Soumi Maiti Jeong Hun Yeo Shinji Watanabe Y. Ro VLM 59 6 0 15 Sep 2023
Transformers in Speech Processing: A Survey S. Latif Aun Zaidi Heriberto Cuayáhuitl Fahad Shamshad Moazzam Shoukat Muhammad Usama Junaid Qadir 113 48 0 21 Mar 2023
MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentions Shengkui Zhao Bin Ma 73 54 0 23 Feb 2023
NLIP: Noise-robust Language-Image Pre-training Runhu Huang Yanxin Long Jianhua Han Hang Xu Xiwen Liang Chunjing Xu Xiaodan Liang VLM 83 30 0 14 Dec 2022
A Comprehensive Survey of Transformers for Computer Vision Sonain Jamil Md. Jalil Piran Oh-Jin Kwon ViT 58 53 0 11 Nov 2022
Make-A-Video: Text-to-Video Generation without Text-Video Data Uriel Singer Adam Polyak Thomas Hayes Xiaoyue Yin Jie An ... Oron Ashual Oran Gafni Devi Parikh Sonal Gupta Yaniv Taigman DiffM VGen 81 1,409 0 29 Sep 2022
PaLI: A Jointly-Scaled Multilingual Language-Image Model Xi Chen Tianlin Li Soravit Changpinyo A. Piergiovanni Piotr Padlewski ... Andreas Steiner A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut MLLM VLM 92 720 0 14 Sep 2022
Multimodal Learning with Transformers: A Survey Peng Xu Xiatian Zhu David Clifton ViT 153 566 0 13 Jun 2022
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers Wenyi Hong Ming Ding Wendi Zheng Xinghan Liu Jie Tang DiffM 303 608 0 29 May 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 401 6,866 0 13 Apr 2022
Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors Oran Gafni Adam Polyak Oron Ashual Shelly Sheynin Devi Parikh Yaniv Taigman DiffM 71 521 0 24 Mar 2022
SeMask: Semantically Masked Transformers for Semantic Segmentation Jitesh Jain Anukriti Singh Nikita Orlov Zilong Huang Jiachen Li Steven Walton Humphrey Shi ViT 68 97 0 23 Dec 2021
NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion Chenfei Wu Jian Liang Lei Ji Fan Yang Yuejian Fang Daxin Jiang Nan Duan ViT VGen 70 294 0 24 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 138 350 0 11 Nov 2021
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing Sanyuan Chen Chengyi Wang Zhengyang Chen Yu-Huan Wu Shujie Liu ... Yao Qian Jian Wu Micheal Zeng Xiangzhan Yu Furu Wei SSL 244 1,873 0 26 Oct 2021
PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization Wen Xiao Iz Beltagy Giuseppe Carenini Arman Cohan CVBM 124 118 0 16 Oct 2021
Survey: Transformer based Video-Language Pre-training Ludan Ruan Qin Jin VLM ViT 104 45 0 21 Sep 2021
CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows Xiaoyi Dong Jianmin Bao Dongdong Chen Weiming Zhang Nenghai Yu Lu Yuan Dong Chen B. Guo ViT 142 982 0 01 Jul 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 180 2,966 0 14 Jun 2021
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers Enze Xie Wenhai Wang Zhiding Yu Anima Anandkumar J. Álvarez Ping Luo ViT 279 5,030 0 31 May 2021
CogView: Mastering Text-to-Image Generation via Transformers Ming Ding Zhuoyi Yang Wenyi Hong Wendi Zheng Chang Zhou ... Junyang Lin Xu Zou Zhou Shao Hongxia Yang Jie Tang ViT VLM 107 781 0 26 May 2021
Segmenter: Transformer for Semantic Segmentation Robin Strudel Ricardo Garcia Pinel Ivan Laptev Cordelia Schmid ViT 203 1,467 0 12 May 2021
ISTR: End-to-End Instance Segmentation with Transformers Jie Hu Liujuan Cao Yao Lu Shengchuan Zhang Yan Wang Ke Li Feiyue Huang Ling Shao Rongrong Ji ISeg 52 95 0 03 May 2021
GODIVA: Generating Open-DomaIn Videos from nAtural Descriptions Chenfei Wu Lun Huang Qianxi Zhang Binyang Li Lei Ji Fan Yang Guillermo Sapiro Nan Duan DiffM VGen 74 241 0 30 Apr 2021
SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network William Chan Daniel S. Park Chris A. Lee Yu Zhang Quoc V. Le Mohammad Norouzi AI4TS 75 138 0 05 Apr 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 447 21,418 0 25 Mar 2021
Scaling Local Self-Attention for Parameter Efficient Visual Backbones Ashish Vaswani Prajit Ramachandran A. Srinivas Niki Parmar Blake A. Hechtman Jonathon Shlens 90 400 0 23 Mar 2021
TSTNN: Two-stage Transformer based Neural Network for Speech Enhancement in the Time Domain Kai Wang Bengbeng He Weiping Zhu 69 169 0 18 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 527 3,722 0 24 Feb 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 397 4,953 0 24 Feb 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 302 2,516 0 04 Jan 2021
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units Wei-Ning Hsu David Harwath Christopher Song James R. Glass CLIP 79 67 0 31 Dec 2020
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 384 6,768 0 23 Dec 2020
A Survey on Visual Transformer Kai Han Yunhe Wang Hanting Chen Xinghao Chen Jianyuan Guo ... Chunjing Xu Yixing Xu Zhaohui Yang Yiman Zhang Dacheng Tao ViT 200 2,230 0 23 Dec 2020
Pre-Trained Image Processing Transformer Hanting Chen Yunhe Wang Tianyu Guo Chang Xu Yiping Deng Zhenhua Liu Siwei Ma Chunjing Xu Chao Xu Wen Gao VLM ViT 136 1,676 0 01 Dec 2020
Attention is All You Need in Speech Separation Cem Subakan Mirco Ravanelli Samuele Cornell Mirko Bronzi Jianyuan Zhong 95 557 0 25 Oct 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 651 41,103 0 22 Oct 2020
Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation Jing-jing Chen Qi-rong Mao Dong Liu 75 286 0 28 Jul 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 282 5,790 0 20 Jun 2020
FastPitch: Parallel Text-to-speech with Pitch Prediction Adrian Lañcucki 73 340 0 11 Jun 2020
MultiSpeech: Multi-Speaker Text to Speech with Transformer Mingjian Chen Xu Tan Yi Ren Jin Xu Hao Sun Sheng Zhao Tao Qin Tie-Yan Liu 62 110 0 08 Jun 2020
FastSpeech 2: Fast and High-Quality End-to-End Text to Speech Yi Ren Chenxu Hu Xu Tan Tao Qin Sheng Zhao Zhou Zhao Tie-Yan Liu 105 1,396 0 08 Jun 2020
Learning Texture Transformer Network for Image Super-Resolution Fuzhi Yang Huan Yang Jianlong Fu Hongtao Lu B. Guo SupR ViT 74 724 0 07 Jun 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 399 13,035 0 26 May 2020
A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformer Vladimir E. Iashin Esa Rahtu 59 129 0 17 May 2020
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 223 3,139 0 16 May 2020
FLAT: Chinese NER Using Flat-Lattice Transformer Xiaonan Li Hang Yan Xipeng Qiu Xuanjing Huang 60 396 0 24 Apr 2020
TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue Chien-Sheng Wu Guosheng Lin R. Socher Caiming Xiong 58 323 0 15 Apr 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 168 4,071 0 10 Apr 2020
Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss Qian Zhang Han Lu Hasim Sak Anshuman Tripathi Erik McDermott Stephen Koo Shankar Kumar 81 480 0 07 Feb 2020