Directly Fine-Tuning Diffusion Models on Differentiable Rewards

29 September 2023

Amita Gajewar

Paul Vicol

G. Bansal

David J Fleet

ArXiv PDF HTML

Papers citing "Directly Fine-Tuning Diffusion Models on Differentiable Rewards"

50 / 124 papers shown

Title
Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models Fu-Yun Wang Yunhao Shui Jingtan Piao Keqiang Sun Hongsheng Li 22 0 0 16 May 2025
Towards Self-Improvement of Diffusion Models via Group Preference Optimization Renjie Chen Wenfeng Lin Yichen Zhang Jiangchuan Wei Boyuan Liu Chao Feng Jiao Ran Mingyu Guo 12 0 0 16 May 2025
CompAlign: Improving Compositional Text-to-Image Generation with a Complex Benchmark and Fine-Grained Feedback Yixin Wan Kai-Wei Chang EGVM CoGe 25 0 0 16 May 2025
You Only Look One Step: Accelerating Backpropagation in Diffusion Sampling with Gradient Shortcuts Hongkun Dou Zeyu Li Xingyu Jiang Hao Li Lijun Yang Wen Yao Yue Deng DiffM 38 0 0 12 May 2025
Flow-GRPO: Training Flow Matching Models via Online RL Jie Liu Gongye Liu Jiajun Liang Yongqian Li Jiaheng Liu Xinyu Wang Pengfei Wan Di Zhang Wanli Ouyang AI4CE 68 0 0 08 May 2025
Augmenting Perceptual Super-Resolution via Image Quality Predictors Fengjia Zhang Samrudhdhi B. Rangrej Tristan Aumentado-Armstrong Afsaneh Fazly Alex Levinshtein SupR 72 0 0 25 Apr 2025
SUDO: Enhancing Text-to-Image Diffusion Models with Self-Supervised Direct Preference Optimization Liang Peng Boxi Wu Haoran Cheng Yibo Zhao Xiaofei He 36 0 0 20 Apr 2025
Diffusion Distillation With Direct Preference Optimization For Efficient 3D LiDAR Scene Completion An Zhao Shengyuan Zhang Ling Yang Z. Li Jiale Wu Haoran Xu AnYang Wei Perry Pengyun GU Lingyun Sun 24 0 0 15 Apr 2025
ADT: Tuning Diffusion Models with Adversarial Supervision Dazhong Shen Guanglu Song Yang Zhang Bingqi Ma Lujundong Li D. Jiang Zhuofan Zong Y. Liu DiffM 50 0 0 15 Apr 2025
Aligning Anime Video Generation with Human Feedback Bingwen Zhu Yudong Jiang Baohan Xu Siqian Yang Mingyu Yin Yidi Wu Huyang Sun Zuxuan Wu EGVM VGen 48 0 0 14 Apr 2025
InstructEngine: Instruction-driven Text-to-Image Alignment Xingyu Lu Yihan Hu Yang Zhang Kaiyu Jiang Changyi Liu ... Bin Wen C. Yuan Fan Yang Tingting Gao Di Zhang 48 0 0 14 Apr 2025
Discriminator-Free Direct Preference Optimization for Video Diffusion Haoran Cheng Qide Dong Liang Peng Zhizhou Sha Weiguo Feng Jinghui Xie Zhao Song Shilei Wen Xiaofei He Boxi Wu VGen 122 0 0 11 Apr 2025
Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data Zhiyuan Ma Xinyue Liang Rongyuan Wu Xiangyu Zhu Zhen Lei Lei Zhang 71 0 0 27 Mar 2025
IPGO: Indirect Prompt Gradient Optimization for Parameter-Efficient Prompt-level Fine-Tuning on Text-to-Image Models Jianping Ye Michel Wedel Kunpeng Zhang 39 0 0 25 Mar 2025
Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing Jaihoon Kim Taehoon Yoon Jisung Hwang Minhyuk Sung DiffM 54 1 0 25 Mar 2025
InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment Yaojie Lu Qichao Wang H. Cao Xierui Wang Xiaoyin Xu Min Zhang 61 0 0 24 Mar 2025
EvolvingGrasp: Evolutionary Grasp Generation via Efficient Preference Alignment Yufei Zhu Yiming Zhong Zemin Yang Peishan Cong Jingyi Yu X. Zhu Y. Ma 55 1 0 18 Mar 2025
Rewards Are Enough for Fast Photo-Realistic Text-to-image Generation Yihong Luo Tianyang Hu Weijian Luo Kenji Kawaguchi Jing Tang EGVM 159 0 0 17 Mar 2025
Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards Zijing Hu Fengda Zhang Long Chen Kun Kuang Jiahui Li Kaifeng Gao Jun Xiao X. Wang Wenwu Zhu EGVM 56 0 0 14 Mar 2025
Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenization Kyle Sargent Kyle Hsu Justin Johnson L. Fei-Fei Jiajun Wu DiffM MU 58 3 0 14 Mar 2025
Adding Additional Control to One-Step Diffusion with Joint Distribution Matching Yihong Luo Tianyang Hu Yifan Song Jiacheng Sun ZeLin Li Jing Tang DiffM 78 1 0 13 Mar 2025
DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture Design in Text to Image Generation Chen Chen Rui Qian Wenze Hu Tsu-jui Fu Jialing Tong ... Lezhi Li Bowen Zhang A. Schwing Wei Liu Yuqing Yang 64 0 0 13 Mar 2025
Learning to Match Unpaired Data with Minimum Entropy Coupling Mustapha Bounoua Giulio Franzese Pietro Michiardi 39 0 0 11 Mar 2025
ROCM: RLHF on consistency models Shivanshu Shekhar Tong Zhang 40 0 0 08 Mar 2025
Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising Trajectory Sharpening Ye Tian L. Yang Xinchen Zhang Yunhai Tong Mengdi Wang Bin Cui 67 1 0 17 Feb 2025
Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation Taeyoung Yun Dinghuai Zhang Jinkyoo Park Ling Pan DiffM 84 2 0 17 Feb 2025
Maximize Your Diffusion: A Study into Reward Maximization and Alignment for Diffusion-based Control Dom Huh P. Mohapatra 86 1 0 16 Feb 2025
Skill Expansion and Composition in Parameter Space Tenglong Liu J. Li Yinan Zheng Haoyi Niu Yixing Lan Xin Xu Xianyuan Zhan 58 4 0 09 Feb 2025
Dual Caption Preference Optimization for Diffusion Models Amir Saeidi Yiran Luo Agneet Chatterjee Shamanthak Hegde Bimsara Pathiraja Yezhou Yang Chitta Baral DiffM 63 0 0 09 Feb 2025
IPO: Iterative Preference Optimization for Text-to-Video Generation Xiaomeng Yang Zhiyu Tan Xuecheng Nie VGen 106 1 0 04 Feb 2025
Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods Oussama Zekri Nicolas Boullé DiffM 65 3 0 03 Feb 2025
Refining Alignment Framework for Diffusion Models with Intermediate-Step Preference Ranking Jie Ren Yuhang Zhang Dongrui Liu Xiaopeng Zhang Qi Tian 70 0 0 01 Feb 2025
Improving Video Generation with Human Feedback Jie Liu Gongye Liu Jiajun Liang Ziyang Yuan Xiaokun Liu ... Pengfei Wan Di Zhang Kun Gai Yujiu Yang Wanli Ouyang VGen EGVM 61 13 0 23 Jan 2025
DiffDoctor: Diagnosing Image Diffusion Models Before Treating Yiyang Wang Xi Chen Xiaogang Xu S. Ji Y. Liu Yujun Shen Hengshuang Zhao DiffM 49 0 0 21 Jan 2025
Focus-N-Fix: Region-Aware Fine-Tuning for Text-to-Image Generation Xiaoying Xing Avinab Saha Junfeng He Susan Hao Paul Vicol ... Sahil Singla Sarah Young Yinxiao Li Feng Yang Deepak Ramachandran DiffM 52 0 0 11 Jan 2025
Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets Zhen Liu Tim Z. Xiao Weiyang Liu Yoshua Bengio Dinghuai Zhang 123 2 0 10 Dec 2024
RFSR: Improving ISR Diffusion Models via Reward Feedback Learning Xiaopeng Sun Q. Lin Yu Gao Yujie Zhong Chengjian Feng Dengjie Li Zheng Zhao Jie Hu Lin Ma EGVM 77 1 0 04 Dec 2024
DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling Xin Xie Dong Gong 82 1 0 01 Dec 2024
Reward Incremental Learning in Text-to-Image Generation Maorong Wang Jiafeng Mao Xueting Wang Toshihiko Yamasaki EGVM 103 0 0 26 Nov 2024
Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward Zhiwei Jia Yuesong Nan Huixi Zhao Gengdai Liu EGVM 88 0 0 22 Nov 2024
Aligning Few-Step Diffusion Models with Dense Reward Difference Learning Ziyi Zhang Li Shen Sen Zhang Deheng Ye Yong Luo Miaojing Shi Bo Du Dacheng Tao 94 0 0 18 Nov 2024
Controlling Language and Diffusion Models by Transporting Activations P. Rodríguez Arno Blaas Michal Klein Luca Zappella N. Apostoloff Marco Cuturi Xavier Suau LLMSV 37 4 0 30 Oct 2024
Diff-Instruct*: Towards Human-Preferred One-step Text-to-image Generative Models Weijian Luo C. Zhang Debing Zhang Zhengyang Geng 28 3 0 28 Oct 2024
Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferences Weijian Luo EGVM 36 6 0 24 Oct 2024
Scalable Ranked Preference Optimization for Text-to-Image Generation Shyamgopal Karthik Huseyin Coskun Zeynep Akata Sergey Tulyakov J. Ren Anil Kag EGVM 57 5 0 23 Oct 2024
Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design Chenyu Wang Masatoshi Uehara Yichun He Amy Wang Tommaso Biancalani Avantika Lal Tommi Jaakkola Sergey Levine Hanchen Wang Aviv Regev 53 8 0 17 Oct 2024
Improving Long-Text Alignment for Text-to-Image Diffusion Models Luping Liu Chao Du Tianyu Pang Zehan Wang Chongxuan Li Dong Xu VLM 53 4 0 15 Oct 2024
IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generation Xinchen Zhang Ling Yang Bernard Ghanem Yaqi Cai Jiake Xie Yong Tang Yujiu Yang Mengdi Wang Bin Cui EGVM CoGe 44 5 0 09 Oct 2024
T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design Jiachen Li Qian Long Jian Zheng Xiaofeng Gao Robinson Piramuthu Wenhu Chen William Yang Wang VGen 29 22 0 08 Oct 2024
Training-free Diffusion Model Alignment with Sampling Demons Po-Hung Yeh Kuang-Huei Lee Jun-Cheng Chen 39 4 0 08 Oct 2024