The Evolved Transformer

30 January 2019

Papers citing "The Evolved Transformer"

50 / 113 papers shown

Title
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 91 154 0 17 Sep 2021
The NiuTrans System for WNGT 2020 Efficiency Task Chi Hu Bei Li Ye Lin Yinqiao Li Yanyang Li Chenglong Wang Tong Xiao Jingbo Zhu 25 7 0 16 Sep 2021
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation Chenhe Dong Guangrun Wang Hang Xu Jiefeng Peng Xiaozhe Ren Xiaodan Liang 24 28 0 15 Sep 2021
BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation Haoran Xu Benjamin Van Durme Kenton W. Murray 50 57 0 09 Sep 2021
Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation Shangshang Yang Ye Tian Xiaoshu Xiang Shichen Peng Xing-yi Zhang 29 20 0 10 Aug 2021
Learning to Rank Ace Neural Architectures via Normalized Discounted Cumulative Gain Yuge Zhang Quan Zhang Li Zhang Yaming Yang Chenqian Yan Xiaotian Gao Yuqing Yang 33 0 0 06 Aug 2021
Residual Tree Aggregation of Layers for Neural Machine Translation Guoliang Li Yiyang Li 45 0 0 19 Jul 2021
AutoBERT-Zero: Evolving BERT Backbone from Scratch Jiahui Gao Hang Xu Han Shi Xiaozhe Ren Philip L. H. Yu Xiaodan Liang Xin Jiang Zhenguo Li 21 37 0 15 Jul 2021
The Benchmark Lottery Mostafa Dehghani Yi Tay A. Gritsenko Zhe Zhao N. Houlsby Fernando Diaz Donald Metzler Oriol Vinyals 42 89 0 14 Jul 2021
Learning Algebraic Recombination for Compositional Generalization Chenyao Liu Shengnan An Zeqi Lin Qian Liu Bei Chen Jian-Guang Lou Lijie Wen Nanning Zheng Dongmei Zhang CoGe 196 36 0 14 Jul 2021
AutoFormer: Searching Transformers for Visual Recognition Minghao Chen Houwen Peng Jianlong Fu Haibin Ling ViT 52 259 0 01 Jul 2021
Multi-head or Single-head? An Empirical Comparison for Transformer Training Liyuan Liu Jialu Liu Jiawei Han 23 32 0 17 Jun 2021
A Survey of Transformers Tianyang Lin Yuxin Wang Xiangyang Liu Xipeng Qiu ViT 53 1,089 0 08 Jun 2021
Scalable Transformers for Neural Machine Translation Peng Gao Shijie Geng Ping Luo Xiaogang Wang Jifeng Dai Hongsheng Li 31 13 0 04 Jun 2021
Memory-Efficient Differentiable Transformer Architecture Search Yuekai Zhao Li Dong Yelong Shen Zhihua Zhang Furu Wei Weizhu Chen ViT 32 17 0 31 May 2021
Towards a Universal NLG for Dialogue Systems and Simulators with Future Bridging Philipp Ennen Yen-Ting Lin Ali Girayhan Ozbay F. Insalata Maolin Li Ye Tian Sepehr Jalali Da-Shan Shiu 26 2 0 21 May 2021
Dynamic Multi-Branch Layers for On-Device Neural Machine Translation Zhixing Tan Zeyuan Yang Meng Zhang Qun Liu Maosong Sun Yang Liu AI4CE 24 4 0 14 May 2021
Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations Jonathan Herzig Peter Shaw Ming-Wei Chang Kelvin Guu Panupong Pasupat Yuan Zhang AI4CE 30 67 0 15 Apr 2021
Towards Accurate and Compact Architectures via Neural Architecture Transformer Yong Guo Yin Zheng Mingkui Tan Qi Chen Zhipeng Li Jian Chen P. Zhao Junzhou Huang ViT MQ 16 38 0 20 Feb 2021
MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records Zhen Xu David R. So Andrew M. Dai Mamba 61 51 0 03 Feb 2021
Evolving Reinforcement Learning Algorithms John D. Co-Reyes Yingjie Miao Daiyi Peng Esteban Real Sergey Levine Quoc V. Le Honglak Lee Aleksandra Faust 46 73 0 08 Jan 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 233 2,434 0 04 Jan 2021
AutoCaption: Image Captioning with Neural Architecture Search Xinxin Zhu Weining Wang Longteng Guo Jing Liu 32 9 0 16 Dec 2020
Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement Risheng Liu Long Ma Jiaao Zhang Xin-Yue Fan Zhongxuan Luo 38 568 0 10 Dec 2020
Physics-Informed Neural State Space Models via Learning and Evolution Elliott Skomski Ján Drgoňa Aaron Tuor PINN AI4CE 27 9 0 26 Nov 2020
Revisiting Modularized Multilingual NMT to Meet Industrial Demands Sungwon Lyu Bokyung Son Kichang Yang Jaekyoung Bae MoE 15 20 0 19 Oct 2020
Automated Concatenation of Embeddings for Structured Prediction Xinyu Wang Yong-jia Jiang Nguyen Bach Tao Wang Zhongqiang Huang Fei Huang Kewei Tu 35 172 0 10 Oct 2020
A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation Dinghan Shen Ming Zheng Yelong Shen Yanru Qu Weizhu Chen AAML 29 130 0 29 Sep 2020
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 114 1,104 0 14 Sep 2020
AutoTrans: Automating Transformer Design via Reinforced Architecture Search Wei-wei Zhu Xiaoling Wang Xipeng Qiu Yuan Ni Guotong Xie 32 18 0 04 Sep 2020
Very Deep Transformers for Neural Machine Translation Xiaodong Liu Kevin Duh Liyuan Liu Jianfeng Gao 19 102 0 18 Aug 2020
ConvBERT: Improving BERT with Span-based Dynamic Convolution Zihang Jiang Weihao Yu Daquan Zhou Yunpeng Chen Jiashi Feng Shuicheng Yan 43 157 0 06 Aug 2020
Evolving Multi-Resolution Pooling CNN for Monaural Singing Voice Separation Weitao Yuan Bofei Dong Shengbei Wang M. Unoki Wenwu Wang 22 12 0 03 Aug 2020
Proof of Learning (PoLe): Empowering Machine Learning with Consensus Building on Blockchains Yixiao Lan Yuan Liu Boyang Albert Li 19 20 0 29 Jul 2020
Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures Daniel Furrer Marc van Zee Nathan Scales Nathanael Scharli CoGe 26 113 0 17 Jul 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhehuai Chen MoE 43 1,118 0 30 Jun 2020
AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System Pengyu Zhao Kecheng Xiao Yuanxing Zhang Kaigui Bian Wei Yan 21 16 0 10 Jun 2020
Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges E. Galván P. Mooney 29 129 0 09 Jun 2020
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing Zihang Dai Guokun Lai Yiming Yang Quoc V. Le 48 230 0 05 Jun 2020
Lite Transformer with Long-Short Range Attention Zhanghao Wu Zhijian Liu Ji Lin Chengyue Wu Song Han 23 318 0 24 Apr 2020
How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAS Kaicheng Yu René Ranftl Mathieu Salzmann 27 34 0 09 Mar 2020
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch Esteban Real Chen Liang David R. So Quoc V. Le 39 220 0 06 Mar 2020
NAS-Count: Counting-by-Density with Neural Architecture Search Yutao Hu Xiaolong Jiang Xuhui Liu Baochang Zhang Jungong Han Xianbin Cao David Doermann 38 89 0 29 Feb 2020
Sparse Sinkhorn Attention Yi Tay Dara Bahri Liu Yang Donald Metzler Da-Cheng Juan 23 331 0 26 Feb 2020
Semi-Supervised Neural Architecture Search Renqian Luo Xu Tan Rui Wang Tao Qin Enhong Chen Tie-Yan Liu 13 88 0 24 Feb 2020
A Survey of Deep Learning Techniques for Neural Machine Translation Shu Yang Yuxin Wang Xiaowen Chu VLM AI4TS AI4CE 24 138 0 18 Feb 2020
How to 0wn NAS in Your Spare Time Sanghyun Hong Michael Davinroy Yigitcan Kaya Dana Dachman-Soled Tudor Dumitras 33 36 0 17 Feb 2020
Best of Both Worlds: AutoML Codesign of a CNN and its Hardware Accelerator Mohamed S. Abdelfattah L. Dudziak Thomas C. P. Chau Royson Lee Hyeji Kim Nicholas D. Lane 17 80 0 11 Feb 2020
Towards a Human-like Open-Domain Chatbot Daniel De Freitas Minh-Thang Luong David R. So Jamie Hall Noah Fiedel ... Zi Yang Apoorv Kulshreshtha Gaurav Nemade Yifeng Lu Quoc V. Le 42 924 0 27 Jan 2020
MixPath: A Unified Approach for One-shot Neural Architecture Search Xiangxiang Chu Shun Lu Xudong Li Bo Zhang 24 21 0 16 Jan 2020