Re-ttention: Ultra Sparse Visual Generation via Attention Statistical Reshape

28 May 2025

Papers citing "Re-ttention: Ultra Sparse Visual Generation via Attention Statistical Reshape"

41 / 41 papers shown

Title
DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformers Hao Zhang R. Su Zhihang Yuan Pengtao Chen Mingzhu Shen Yibo Fan Shengen Yan Guohao Dai Yu Wang 82 2 0 28 Mar 2025
XAttention: Block Sparse Attention with Antidiagonal Scoring Ruyi Xu Guangxuan Xiao Haofeng Huang Junxian Guo Enze Xie 110 11 0 20 Mar 2025
Predicting Team Performance from Communications in Simulated Search-and-Rescue Ali Jalal-Kamali Nikolos Gurney David Pynadath AI4TS 157 14 0 05 Mar 2025
FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference Xunhao Lai Jianqiao Lu Yao Luo Yiyuan Ma Xun Zhou 100 9 0 28 Feb 2025
Open-Sora: Democratizing Efficient Video Production for All Zangwei Zheng Xiangyu Peng Tianji Yang Chenhui Shen Shenggui Li Hongxin Liu Yukun Zhou Tianyi Li Yang You VGen 119 238 0 31 Dec 2024
Open-Sora Plan: Open-Source Large Video Generation Model Bin Lin Yunyang Ge Xinhua Cheng Zongjian Li Bin Zhu ... Zhang Pan Xing Zhou Shaoling Dong Yonghong Tian Li-xin Yuan VLM VGen 159 80 0 28 Nov 2024
Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model Feng Liu Shiwei Zhang Xiaofeng Wang Yujie Wei Haonan Qiu Yuzhong Zhao Yingya Zhang Qixiang Ye Fang Wan VGen AI4TS 169 22 0 28 Nov 2024
Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study Xibo Sun Jiarui Fang Aoyu Li Jinzhe Pan 81 5 0 18 Nov 2024
Real-Time Video Generation with Pyramid Attention Broadcast Xuanlei Zhao Xiaolong Jin Kai Wang Yang You VGen DiffM 114 40 0 22 Aug 2024
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer Zhuoyi Yang Jiayan Teng Wendi Zheng Ming Ding Shiyu Huang ... Weihan Wang Yean Cheng Xiaotao Gu Yuxiao Dong Jie Tang DiffM VGen 199 507 0 12 Aug 2024
MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention Huiqiang Jiang Yucheng Li Chengruidong Zhang Qianhui Wu Xufang Luo ... Amir H. Abdi Dongsheng Li Chin-Yew Lin Yuqing Yang L. Qiu 129 110 0 02 Jul 2024
DiTFastAttn: Attention Compression for Diffusion Transformer Models Zhihang Yuan Pu Lu Hanling Zhang Xuefei Ning Linfeng Zhang Tianchen Zhao Shengen Yan Guohao Dai Yu Wang 78 28 0 12 Jun 2024
$Δ$ -DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers Pengtao Chen Mingzhu Shen Peng Ye Jianjian Cao Chongjun Tu C. Bouganis Yiren Zhao Tao Chen 86 39 0 03 Jun 2024
Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysis Yao Fu 52 23 0 14 May 2024
Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding Zhimin Li Jianwei Zhang Qin Lin Jiangfeng Xiong Yanxin Long ... Wei Liu Dingyong Wang Yong Yang Jie Jiang Qinglin Lu ViT 107 108 0 14 May 2024
PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation Junsong Chen Chongjian Ge Enze Xie Yue Wu Lewei Yao Xiaozhe Ren Zhongdao Wang Ping Luo Huchuan Lu Zhenguo Li 171 109 0 07 Mar 2024
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis Patrick Esser Sumith Kulal A. Blattmann Rahim Entezari Jonas Muller ... Zion English Kyle Lacey Alex Goodwin Yannik Marek Robin Rombach DiffM 251 1,305 0 05 Mar 2024
DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models Zhekai Zhang Tianle Cai Jiaxin Cao Qinsheng Zhang Han Cai Junjie Bai Yangqing Jia Ming-Yu Liu Kai Li Song Han DiffM 56 47 0 29 Feb 2024
Latte: Latent Diffusion Transformer for Video Generation Xin Ma Yaohui Wang Gengyun Jia Xinyuan Chen Ziqiang Liu Yuan-Fang Li Cunjian Chen Yu Qiao DiffM VGen 212 269 0 05 Jan 2024
VBench: Comprehensive Benchmark Suite for Video Generative Models Ziqi Huang Yinan He Jiashuo Yu Fan Zhang Chenyang Si ... Xinyuan Chen Limin Wang Dahua Lin Yu Qiao Ziwei Liu VGen 154 410 0 29 Nov 2023
GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment Dhruba Ghosh Hanna Hajishirzi Ludwig Schmidt 78 179 0 17 Oct 2023
Mistral 7B Albert Q. Jiang Alexandre Sablayrolles A. Mensch Chris Bamford Devendra Singh Chaplot ... Teven Le Scao Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LRM 70 2,192 0 10 Oct 2023
PixArt- $α$ : Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis Junsong Chen Jincheng Yu Chongjian Ge Lewei Yao Enze Xie ... Zhongdao Wang James T. Kwok Ping Luo Huchuan Lu Zhenguo Li DiffM 74 437 0 30 Sep 2023
Efficient Streaming Language Models with Attention Sinks Michel Lang Yuandong Tian Beidi Chen Song Han Mike Lewis AI4TS RALM 119 750 0 29 Sep 2023
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis Dustin Podell Zion English Kyle Lacey A. Blattmann Tim Dockhorn Jonas Muller Joe Penna Robin Rombach 205 2,351 0 04 Jul 2023
Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis Xiaoshi Wu Yiming Hao Keqiang Sun Yixiong Chen Feng Zhu Rui Zhao Hongsheng Li 75 286 0 15 Jun 2023
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation Jiazheng Xu Xiao Liu Yuchen Wu Yuxuan Tong Qinkai Li Ming Ding Jie Tang Yuxiao Dong 115 371 0 12 Apr 2023
Scalable Diffusion Models with Transformers William S. Peebles Saining Xie GNN 86 2,298 0 19 Dec 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 393 15,454 0 20 Dec 2021
CLIPScore: A Reference-free Evaluation Metric for Image Captioning Jack Hessel Ari Holtzman Maxwell Forbes Ronan Le Bras Yejin Choi CLIP 127 1,557 0 18 Apr 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 434 21,392 0 25 Mar 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 601 40,961 0 22 Oct 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 525 2,080 0 28 Jul 2020
Understanding SSIM J. Nilsson T. Akenine-Möller 41 143 0 24 Jun 2020
Denoising Diffusion Probabilistic Models Jonathan Ho Ajay Jain Pieter Abbeel DiffM 561 18,008 0 19 Jun 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 145 4,061 0 10 Apr 2020
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 107 1,896 0 23 Apr 2019
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric Richard Y. Zhang Phillip Isola Alexei A. Efros Eli Shechtman Oliver Wang EGVM 355 11,784 0 11 Jan 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 677 131,414 0 12 Jun 2017
U-Net: Convolutional Networks for Biomedical Image Segmentation Olaf Ronneberger Philipp Fischer Thomas Brox SSeg 3DV 1.8K 77,099 0 18 May 2015
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 403 43,619 0 01 May 2014