Adapting LLaMA Decoder to Vision Transformer

10 April 2024

Yong Liu

Ping Luo

Papers citing "Adapting LLaMA Decoder to Vision Transformer"

50 / 69 papers shown

Title
Wormhole Memory: A Rubik's Cube for Cross-Dialogue Retrieval Libo Wang 334 0 0 24 Jan 2025
Data-efficient Large Vision Models through Sequential Autoregression Jianyuan Guo Zhiwei Hao Chengcheng Wang Yehui Tang Han Wu Han Hu Kai Han Chang Xu VLM 67 10 0 07 Feb 2024
Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities Yiyuan Zhang Xiaohan Ding Kaixiong Gong Yixiao Ge Ying Shan Xiangyu Yue ViT 95 7 0 25 Jan 2024
Scalable Pre-training of Large Autoregressive Image Models Alaaeldin El-Nouby Michal Klein Shuangfei Zhai Miguel Angel Bautista Alexander Toshev Vaishaal Shankar J. Susskind Armand Joulin VLM 51 73 0 16 Jan 2024
Sequential Modeling Enables Scalable Learning for Large Vision Models Yutong Bai Xinyang Geng K. Mangalam Amir Bar Alan Yuille Trevor Darrell Jitendra Malik Alexei A. Efros MLLM VLM 54 158 0 01 Dec 2023
Initializing Models with Larger Ones Zhiqiu Xu Yanjie Chen Kirill Vishniakov Yida Yin Zhiqiang Shen Trevor Darrell Lingjie Liu Zhuang Liu 56 19 0 30 Nov 2023
ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracy Kirill Vishniakov Zhiqiang Shen Zhuang Liu CLIP 68 16 0 15 Nov 2023
OtterHD: A High-Resolution Multi-modality Model Yue Liu Peiyuan Zhang Jingkang Yang Yuanhan Zhang Fanyi Pu Ziwei Liu VLM MLLM 57 65 0 07 Nov 2023
Frozen Transformers in Language Models Are Effective Visual Encoder Layers Ziqi Pang Ziyang Xie Yunze Man Yu-Xiong Wang 91 26 0 19 Oct 2023
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 203 457 0 14 Oct 2023
Mistral 7B Albert Q. Jiang Alexandre Sablayrolles A. Mensch Chris Bamford Devendra Singh Chaplot ... Teven Le Scao Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LRM 42 2,102 0 10 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 89 2,593 0 05 Oct 2023
Code Llama: Open Foundation Models for Code Baptiste Rozière Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat ... Hugo Touvron Louis Martin Nicolas Usunier Thomas Scialom Gabriel Synnaeve ELM ALM 82 1,990 0 24 Aug 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 206 11,636 0 18 Jul 2023
VanillaNet: the Power of Minimalism in Deep Learning Hanting Chen Yunhe Wang Jianyuan Guo Dacheng Tao VLM 48 88 0 22 May 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 101 1,978 0 20 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 347 4,607 0 17 Apr 2023
Dropout Reduces Underfitting Zhuang Liu Zhi-Qin John Xu Joseph Jin Zhiqiang Shen Trevor Darrell 80 38 0 02 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 819 12,840 0 27 Feb 2023
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders Sanghyun Woo Shoubhik Debnath Ronghang Hu Xinlei Chen Zhuang Liu In So Kweon Saining Xie SyDa 128 760 0 02 Jan 2023
Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer Yanjing Li Sheng Xu Baochang Zhang Xianbin Cao Penglei Gao Guodong Guo MQ ViT 57 92 0 13 Oct 2022
OPT: Open Pre-trained Transformer Language Models Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen ... Daniel Simig Punit Singh Koura Anjali Sridhar Tianlu Wang Luke Zettlemoyer VLM OSLM AI4CE 284 3,583 0 02 May 2022
DeiT III: Revenge of the ViT Hugo Touvron Matthieu Cord Hervé Jégou ViT 103 402 0 14 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 340 6,132 0 05 Apr 2022
A ConvNet for the 2020s Zhuang Liu Hanzi Mao Chaozheng Wu Christoph Feichtenhofer Trevor Darrell Saining Xie ViT 86 5,102 0 10 Jan 2022
MetaFormer Is Actually What You Need for Vision Weihao Yu Mi Luo Pan Zhou Chenyang Si Yichen Zhou Xinchao Wang Jiashi Feng Shuicheng Yan 130 896 0 22 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 375 7,600 0 11 Nov 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 175 2,790 0 15 Jun 2021
Are Convolutional Neural Networks or Transformers more like human vision? Shikhar Tuli Ishita Dasgupta Erin Grant Thomas Griffiths ViT FaML 41 185 0 15 May 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 275 692 0 22 Apr 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 141 2,307 0 20 Apr 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 339 21,175 0 25 Mar 2021
Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth Yihe Dong Jean-Baptiste Cordonnier Andreas Loukas 83 376 0 05 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 450 3,678 0 24 Feb 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 306 6,657 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 408 40,217 0 22 Oct 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 175 1,678 0 08 Jun 2020
Shortcut Learning in Deep Neural Networks Robert Geirhos J. Jacobsen Claudio Michaelis R. Zemel Wieland Brendel Matthias Bethge Felix Wichmann 188 2,023 0 16 Apr 2020
Non-Autoregressive Machine Translation with Latent Alignments Chitwan Saharia William Chan Saurabh Saxena Mohammad Norouzi 38 158 0 16 Apr 2020
GLU Variants Improve Transformer Noam M. Shazeer 107 968 0 12 Feb 2020
Understanding Knowledge Distillation in Non-autoregressive Machine Translation Chunting Zhou Graham Neubig Jiatao Gu 49 220 0 07 Nov 2019
Root Mean Square Layer Normalization Biao Zhang Rico Sennrich 53 712 0 16 Oct 2019
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 193 3,458 0 30 Sep 2019
KERMIT: Generative Insertion-Based Modeling for Sequences William Chan Nikita Kitaev Kelvin Guu Mitchell Stern Jakob Uszkoreit VLM 41 65 0 04 Jun 2019
Generating Diverse High-Fidelity Images with VQ-VAE-2 Ali Razavi Aaron van den Oord Oriol Vinyals DRL BDL 108 1,788 0 02 Jun 2019
Levenshtein Transformer Jiatao Gu Changhan Wang Jake Zhao 105 359 0 27 May 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 587 4,735 0 13 May 2019
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness Robert Geirhos Patricia Rubisch Claudio Michaelis Matthias Bethge Felix Wichmann Wieland Brendel 89 2,647 0 29 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.0K 93,936 0 11 Oct 2018
Unified Perceptual Parsing for Scene Understanding Tete Xiao Yingcheng Liu Bolei Zhou Yuning Jiang Jian Sun OCL VOS 124 1,859 0 26 Jul 2018