QuadraNet: Improving High-Order Neural Interaction Efficiency with
Hardware-Aware Quadratic Neural Networks

QuadraNet: Improving High-Order Neural Interaction Efficiency with Hardware-Aware Quadratic Neural Networks

29 November 2023

Zirui Xu

Jinjun Xiong

ArXiv (abs)PDF HTML

Papers citing "QuadraNet: Improving High-Order Neural Interaction Efficiency with Hardware-Aware Quadratic Neural Networks"

16 / 16 papers shown

Title
On The Computational Complexity of Self-Attention Feyza Duman Keles Pruthuvi Maheshakya Wijewardena Chinmay Hegde 135 128 0 11 Sep 2022
HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions Yongming Rao Wenliang Zhao Yansong Tang Jie Zhou Ser-Nam Lim Jiwen Lu ViT 103 256 0 28 Jul 2022
QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration Zirui Xu Fuxun Yu Jinjun Xiong Xiang Chen 51 25 0 01 Apr 2022
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 288 1,285 0 05 Oct 2021
Global Filter Networks for Image Classification Yongming Rao Wenliang Zhao Zheng Zhu Jiwen Lu Jie Zhou ViT 71 470 0 01 Jul 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 467 21,603 0 25 Mar 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 391 1,574 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 535 3,740 0 24 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 146 1,942 0 28 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,805 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 682 41,483 0 22 Oct 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 219 1,716 0 08 Jun 2020
$Π-$ nets: Deep Polynomial Neural Networks Grigorios G. Chrysos Stylianos Moschoglou Giorgos Bouritsas Yannis Panagakis Jiankang Deng Stefanos Zafeiriou 63 60 0 08 Mar 2020
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware Han Cai Ligeng Zhu Song Han 105 1,876 0 02 Dec 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 209 19,335 0 13 Jan 2018
A New Type of Neurons for Machine Learning Fenglei Fan W. Cong Ge Wang 47 73 0 26 Apr 2017