FNet: Mixing Tokens with Fourier Transforms

9 May 2021

James Lee-Thorp

Joshua Ainslie

Ilya Eckstein

Santiago Ontanon

ArXiv PDF HTML

Papers citing "FNet: Mixing Tokens with Fourier Transforms"

50 / 84 papers shown

Title
Non-Stationary Time Series Forecasting Based on Fourier Analysis and Cross Attention Mechanism Yuqi Xiong Yang Wen AI4TS 26 0 0 11 May 2025
FIM: Frequency-Aware Multi-View Interest Modeling for Local-Life Service Recommendation Guoquan Wang Qiang Luo Weisong Hu Pengfei Yao Wencong Zeng Guorui Zhou Kun Gai AI4TS 51 0 0 23 Apr 2025
FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation Andrew Kiruluta Eric Lundy Andreas Lemos AI4TS 39 0 0 04 Mar 2025
Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction Xiang Xu Hao Wang Wei Guo L. Zhang Wanshan Yang R. Yu Y. Liu Defu Lian Enhong Chen 35 5 0 22 Nov 2024
FAN: Fourier Analysis Networks Yihong Dong Ge Li Yongding Tao Xue Jiang Kechi Zhang Jia Li Jing Su Jing Su Jun Zhang Jingjing Xu AI4TS 13 4 0 03 Oct 2024
Hypergraph Learning based Recommender System for Anomaly Detection, Control and Optimization Sakhinana Sagar Srinivas Rajat Kumar Sarkar Venkataramana Runkana AI4TS 24 2 0 21 Aug 2024
Repurformer: Transformers for Repurposing-Aware Molecule Generation Changhun Lee Gyumin Lee 49 0 0 16 Jul 2024
Multi-State-Action Tokenisation in Decision Transformers for Multi-Discrete Action Spaces Perusha Moodley Pramod S. Kaushik Dhillu Thambi Mark Trovinger Praveen Paruchuri Xia Hong Benjamin Rosman 50 0 0 01 Jul 2024
GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization Y. Chen X. Huang Quan Zhang Wei Li Mingjian Zhu ... Hanting Chen Hailin Hu J. Yang W. Liu Jie Hu EGVM 58 1 0 24 Jun 2024
LOLAMEME: Logic, Language, Memory, Mechanistic Framework Jay Desai Xiaobo Guo Srinivasan H. Sengamedu 19 0 0 31 May 2024
SWAT: Scalable and Efficient Window Attention-based Transformers Acceleration on FPGAs Zhenyu Bai Pranav Dangi Huize Li Tulika Mitra 29 5 0 27 May 2024
MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolution Yuxuan Jiang Chen Feng Fan Zhang David Bull SupR 51 11 0 15 Apr 2024
Model Selection with Model Zoo via Graph Learning Ziyu Li Hilco van der Wilk Danning Zhan Megha Khosla A. Bozzon Rihan Hai 46 1 0 05 Apr 2024
SpiralMLP: A Lightweight Vision MLP Architecture Haojie Mu Burhan Ul Tayyab Nicholas Chua 43 0 0 31 Mar 2024
DiJiang: Efficient Large Language Models through Compact Kernelization Hanting Chen Zhicheng Liu Xutao Wang Yuchuan Tian Yunhe Wang VLM 26 5 0 29 Mar 2024
Learning with SASQuaTCh: a Novel Variational Quantum Transformer Architecture with Kernel-Based Self-Attention Ethan N. Evans Matthew G. Cook Zachary P. Bradshaw Margarite L. LaBorde 40 5 0 21 Mar 2024
Windowed-FourierMixer: Enhancing Clutter-Free Room Modeling with Fourier Transform Bruno Henriques Benjamin Allaert Jean-Philippe Vandeborre 3DV 29 0 0 28 Feb 2024
Learning to See Through Dazzle Xiaopeng Peng Erin F. Fleet A. Watnik Grover A. Swartzlander GAN AAML 25 4 0 24 Feb 2024
FViT: A Focal Vision Transformer with Gabor Filter Yulong Shi Mingwei Sun Yongshuai Wang Rui Wang 52 4 0 17 Feb 2024
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 26 1 0 18 Dec 2023
Graph Convolutions Enrich the Self-Attention in Transformers! Jeongwhan Choi Hyowon Wi Jayoung Kim Yehjin Shin Kookjin Lee Nathaniel Trask Noseong Park 25 4 0 07 Dec 2023
Improving Interpretation Faithfulness for Vision Transformers Lijie Hu Yixin Liu Ninghao Liu Mengdi Huai Lichao Sun Di Wang 27 5 0 29 Nov 2023
Transformer-VQ: Linear-Time Transformers via Vector Quantization Albert Mohwald 28 15 0 28 Sep 2023
Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers Tobias Christian Nauen Sebastián M. Palacio Federico Raue Andreas Dengel 42 3 0 18 Aug 2023
MultiWave: Multiresolution Deep Architectures through Wavelet Decomposition for Multivariate Time Series Prediction I. Deznabi M. Fiterau AI4TS 30 5 0 16 Jun 2023
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Ziwei He Meng-Da Yang Minwei Feng Jingcheng Yin X. Wang Jingwen Leng Zhouhan Lin ViT 29 11 0 24 May 2023
Cascaded Local Implicit Transformer for Arbitrary-Scale Super-Resolution Haoming Chen Yu-Syuan Xu Minui Hong Yi-Min Tsai Hsien-Kai Kuo Chun-Yi Lee OffRL 33 46 0 29 Mar 2023
EdgeTran: Co-designing Transformers for Efficient Inference on Mobile Edge Platforms Shikhar Tuli N. Jha 34 3 0 24 Mar 2023
TemporalMaxer: Maximize Temporal Context with only Max Pooling for Temporal Action Localization Tuan N. Tang Kwonyoung Kim K. Sohn 16 29 0 16 Mar 2023
MITFAS: Mutual Information based Temporal Feature Alignment and Sampling for Aerial Video Action Recognition Ruiqi Xian Xijun Wang Dinesh Manocha 19 10 0 05 Mar 2023
A Federated Approach for Hate Speech Detection Jay Gala Deep Gandhi Jash Mehta Zeerak Talat 13 4 0 18 Feb 2023
Efficiency 360: Efficient Vision Transformers Badri N. Patro Vijay Srinivas Agneeswaran 26 6 0 16 Feb 2023
Simple Hardware-Efficient Long Convolutions for Sequence Modeling Daniel Y. Fu Elliot L. Epstein Eric N. D. Nguyen A. Thomas Michael Zhang Tri Dao Atri Rudra Christopher Ré 16 52 0 13 Feb 2023
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment Jared Fernandez Jacob Kahn Clara Na Yonatan Bisk Emma Strubell FedML 25 10 0 13 Feb 2023
MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing Zhe Li Zhongwen Rao Lujia Pan Zenglin Xu AI4TS 27 60 0 09 Feb 2023
Efficient Joint Learning for Clinical Named Entity Recognition and Relation Extraction Using Fourier Networks: A Use Case in Adverse Drug Events A. Yazdani D. Proios H. Rouhizadeh Douglas Teodoro 19 7 0 08 Feb 2023
Scaling Up Computer Vision Neural Networks Using Fast Fourier Transform Siddharth Agrawal 13 0 0 02 Feb 2023
WLD-Reg: A Data-dependent Within-layer Diversity Regularizer Firas Laakom Jenni Raitoharju Alexandros Iosifidis M. Gabbouj AI4CE 26 7 0 03 Jan 2023
Pretraining Without Attention Junxiong Wang J. Yan Albert Gu Alexander M. Rush 25 48 0 20 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 120 36 0 15 Dec 2022
Paraphrase Identification with Deep Learning: A Review of Datasets and Methods Chao Zhou Cheng Qiu Daniel Ernesto Acuna 29 25 0 13 Dec 2022
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities Zhe Zhao Yudong Li Cheng-An Hou Jing-xin Zhao Rong Tian ... Xingwu Sun Zhanhui Kang Xiaoyong Du Linlin Shen Kimmo Yan VLM 33 23 0 13 Dec 2022
BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch? Joel Niklaus Daniele Giofré 27 11 0 30 Nov 2022
Lightweight Structure-Aware Attention for Visual Understanding Heeseung Kwon F. M. Castro M. Marín-Jiménez N. Guil Alahari Karteek 28 2 0 29 Nov 2022
Efficient Transformers with Dynamic Token Pooling Piotr Nawrot J. Chorowski Adrian Lañcucki E. Ponti 14 42 0 17 Nov 2022
An FNet based Auto Encoder for Long Sequence News Story Generation Paul K. Mandal Rakeshkumar V. Mahto 19 0 0 15 Nov 2022
Contextual Learning in Fourier Complex Field for VHR Remote Sensing Images Yan Zhang Xiyuan Gao Qingyan Duan Jiaxu Leng Xiao Pu Xinbo Gao ViT 16 1 0 28 Oct 2022
MetaFormer Baselines for Vision Weihao Yu Chenyang Si Pan Zhou Mi Luo Yichen Zhou Jiashi Feng Shuicheng Yan Xinchao Wang MoE 25 156 0 24 Oct 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 41 9 0 14 Oct 2022
FCT-GAN: Enhancing Table Synthesis via Fourier Transform Zilong Zhao Robert Birke L. Chen 13 7 0 12 Oct 2022