Directly Fine-Tuning Diffusion Models on Differentiable Rewards

29 September 2023

Amita Gajewar

Paul Vicol

G. Bansal

David J Fleet

ArXiv PDF HTML

Papers citing "Directly Fine-Tuning Diffusion Models on Differentiable Rewards"

50 / 124 papers shown

Title
An Efficient On-Policy Deep Learning Framework for Stochastic Optimal Control Mengjian Hua Matthieu Laurière Eric Vanden-Eijnden 31 3 0 07 Oct 2024
HERO: Human-Feedback Efficient Reinforcement Learning for Online Diffusion Model Finetuning Ayano Hiranaka Shang-Fu Chen Chieh-Hsin Lai Dongjun Kim Naoki Murata Takashi Shibuya Wei-Hsiang Liao Shao-Hua Sun Yuki Mitsufuji 47 1 0 07 Oct 2024
ComfyGen: Prompt-Adaptive Workflows for Text-to-Image Generation Rinon Gal Adi Haviv Yuval Alaluf Amit H. Bermano Daniel Cohen-Or Gal Chechik DiffM 26 3 0 02 Oct 2024
A Taxonomy of Loss Functions for Stochastic Optimal Control Carles Domingo-Enrich 35 3 0 01 Oct 2024
Task-agnostic Pre-training and Task-guided Fine-tuning for Versatile Diffusion Planner Chenyou Fan Chenjia Bai Zhao Shan Haoran He Yang Zhang Zhen Wang 33 3 0 30 Sep 2024
Physics-aligned Schrödinger bridge Zeyu Li Hongkun Dou Shen Fang Wang Han Yue Deng Lijun Yang AI4CE DiffM 30 0 0 26 Sep 2024
Latent Diffusion Models for Controllable RNA Sequence Generation Kaixuan Huang Yukang Yang Kaidi Fu Yanyi Chu Le Cong Mengdi Wang 47 1 0 15 Sep 2024
Generalizing Alignment Paradigm of Text-to-Image Generation with Preferences through $f$ -divergence Minimization Haoyuan Sun Bo Xia Yongzhe Chang Xueqian Wang EGVM 35 2 0 15 Sep 2024
FreeEnhance: Tuning-Free Image Enhancement via Content-Consistent Noising-and-Denoising Process Yang Luo Y. Zhang Zhaofan Qiu Ting Yao Zhineng Chen Yu-Gang Jiang Tao Mei DiffM 40 4 0 11 Sep 2024
Alignment of Diffusion Models: Fundamentals, Challenges, and Future Buhua Liu Shitong Shao Bao Li Lichen Bai Zhiqiang Xu Haoyi Xiong James Kwok Sumi Helal Zeke Xie 45 12 0 11 Sep 2024
Elucidating Optimal Reward-Diversity Tradeoffs in Text-to-Image Diffusion Models Rohit Jena Ali Taghibakhshi Sahil Jain Gerald Shen Nima Tajbakhsh Arash Vahdat 42 3 0 09 Sep 2024
Reward-Directed Score-Based Diffusion Models via q-Learning Xuefeng Gao Jiale Zha X. Zhou DiffM 39 2 0 07 Sep 2024
Diffusion Policy Policy Optimization Allen Z. Ren Justin Lidard Lars L. Ankile Anthony Simeonov Pulkit Agrawal Anirudha Majumdar Benjamin Burchfiel Hongkai Dai Max Simchowitz 45 36 0 01 Sep 2024
Constrained Diffusion Models via Dual Training Shervin Khalafi Dongsheng Ding Alejandro Ribeiro 42 3 0 27 Aug 2024
Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding Xiner Li Yulai Zhao Chenyu Wang Gabriele Scalia Gökçen Eraslan Surag Nair Tommaso Biancalani Aviv Regev Sergey Levine Masatoshi Uehara 54 23 0 15 Aug 2024
Towards Reliable Advertising Image Generation Using Human Feedback Thorben Werner Wei Feng Haohan Wang Yaoyu Li Jingsen Wang ... Maximilian Stubbemann Junsheng Jin Lars Schmidt-Thieme Zhangang Lin Jingping Shao 50 3 0 01 Aug 2024
Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review Masatoshi Uehara Yulai Zhao Tommaso Biancalani Sergey Levine 63 22 0 18 Jul 2024
Exploring the Potentials and Challenges of Deep Generative Models in Product Design Conception Phillip Mueller Lars Mikelsons AI4CE 41 1 0 15 Jul 2024
Powerful and Flexible: Personalized Text-to-Image Generation via Reinforcement Learning Fanyue Wei Wei Zeng Zhenyang Li Dawei Yin Lixin Duan Wen Li EGVM 39 2 0 09 Jul 2024
Maximum Entropy Inverse Reinforcement Learning of Diffusion Models with Energy-Based Models Sangwoong Yoon Himchan Hwang Dohyun Kwon Yung-Kyun Noh Frank C. Park 34 3 0 30 Jun 2024
Prompt Refinement with Image Pivot for Text-to-Image Generation Jingtao Zhan Qingyao Ai Yiqun Liu Yingwei Pan Ting Yao Jiaxin Mao Shaoping Ma Tao Mei EGVM 30 4 0 28 Jun 2024
Aligning Diffusion Models with Noise-Conditioned Perception Alexander Gambashidze Anton Kulikov Yuriy Sosnin Ilya Makarov 47 5 0 25 Jun 2024
VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation Xuan He Dongfu Jiang Ge Zhang Max W.F. Ku Achint Soni ... Yaswanth Narsupalli Rongqi Fan Zhiheng Lyu Yuchen Lin Wenhu Chen EGVM VGen ALM 48 42 0 21 Jun 2024
Adding Conditional Control to Diffusion Models with Reinforcement Learning Yulai Zhao Masatoshi Uehara Gabriele Scalia Tommaso Biancalani Sergey Levine Ehsan Hajiramezanali Ehsan Hajiramezanali AI4CE 57 3 0 17 Jun 2024
PID: Prompt-Independent Data Protection Against Latent Diffusion Models Ang Li Yichuan Mo Mingjie Li Yisen Wang AAML 46 2 0 14 Jun 2024
Margin-aware Preference Optimization for Aligning Diffusion Models without Reference Jiwoo Hong Sayak Paul Noah Lee Kashif Rasul James Thorne Jongheon Jeong 43 13 0 10 Jun 2024
Diffusion-RPO: Aligning Diffusion Models through Relative Preference Optimization Yi Gu Zhendong Wang Yueqin Yin Yujia Xie Mingyuan Zhou 38 15 0 10 Jun 2024
ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization L. Eyring Shyamgopal Karthik Karsten Roth Alexey Dosovitskiy Zeynep Akata 78 17 0 06 Jun 2024
Improving GFlowNets for Text-to-Image Diffusion Alignment Dinghuai Zhang Yizhe Zhang Jiatao Gu Ruixiang Zhang J. Susskind Navdeep Jaitly Shuangfei Zhai EGVM 98 7 0 02 Jun 2024
Information Theoretic Text-to-Image Alignment Chao Wang Giulio Franzese A. Finamore Massimo Gallo Pietro Michiardi 75 0 0 31 May 2024
Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models Masatoshi Uehara Yulai Zhao Ehsan Hajiramezanali Gabriele Scalia Gökçen Eraslan Avantika Lal Sergey Levine Tommaso Biancalani 53 13 0 30 May 2024
T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback Jiachen Li Weixi Feng Tsu-jui Fu Xinyi Wang Sugato Basu Wenhu Chen William Yang Wang VGen 34 27 0 29 May 2024
Deep Reward Supervisions for Tuning Text-to-Image Diffusion Models Xiaoshi Wu Yiming Hao Manyuan Zhang Keqiang Sun Zhaoyang Huang Guanglu Song Yu Liu Hongsheng Li EGVM 76 18 0 01 May 2024
PuLID: Pure and Lightning ID Customization via Contrastive Alignment Zinan Guo Yanze Wu Zhuowei Chen Lang Chen Qian He DiffM 41 58 0 24 Apr 2024
Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation Xun Wu Shaohan Huang Furu Wei 44 8 0 23 Apr 2024
Gradient Guidance for Diffusion Models: An Optimization Perspective Yingqing Guo Hui Yuan Yukang Yang Minshuo Chen Mengdi Wang 27 20 0 23 Apr 2024
ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback Ming Li Taojiannan Yang Huafeng Kuang Jie Wu Zhaoning Wang Xuefeng Xiao Cheng Chen 37 63 0 11 Apr 2024
An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization Minshuo Chen Song Mei Jianqing Fan Mengdi Wang VLM MedIm DiffM 37 48 0 11 Apr 2024
YaART: Yet Another ART Rendering Technology Sergey Kastryulin Artem Konev Alexander Shishenya Eugene Lyapustin Artem Khurshudov ... Dmitrii Kornilov Mikhail Romanov Artem Babenko Sergei Ovcharenko Valentin Khrulkov EGVM 38 1 0 08 Apr 2024
Aligning Diffusion Models by Optimizing Human Utility Shufan Li Konstantinos Kallidromitis Akash Gokul Yusuke Kato Kazuki Kozuka 107 29 0 06 Apr 2024
Identity Decoupling for Multi-Subject Personalization of Text-to-Image Models Sang-Sub Jang Jaehyeong Jo Kimin Lee Sung Ju Hwang 29 15 0 05 Apr 2024
CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching Dongzhi Jiang Guanglu Song Xiaoshi Wu Renrui Zhang Dazhong Shen Zhuofan Zong Yu Liu Hongsheng Li VLM 32 20 0 04 Apr 2024
TextCraftor: Your Text Encoder Can be Image Quality Controller Yanyu Li Xian Liu Anil Kag Ju Hu Yerlan Idelbayev Dhritiman Sagar Yanzhi Wang Sergey Tulyakov Jian Ren 45 15 0 27 Mar 2024
Reward Guided Latent Consistency Distillation Jiachen Li Weixi Feng Wenhu Chen William Yang Wang EGVM 28 11 0 16 Mar 2024
SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data Jialu Li Jaemin Cho Yi-Lin Sung Jaehong Yoon Mohit Bansal MoMe DiffM 47 8 0 11 Mar 2024
Feedback Efficient Online Fine-Tuning of Diffusion Models Masatoshi Uehara Yulai Zhao Kevin Black Ehsan Hajiramezanali Gabriele Scalia N. Diamant Alex Tseng Sergey Levine Tommaso Biancalani 36 21 0 26 Feb 2024
Graph Diffusion Policy Optimization Yijing Liu Chao Du Tianyu Pang Chongxuan Li Wei Chen Min-Bin Lin 34 7 0 26 Feb 2024
Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Control Masatoshi Uehara Yulai Zhao Kevin Black Ehsan Hajiramezanali Gabriele Scalia N. Diamant Alex Tseng Tommaso Biancalani Sergey Levine 42 42 0 23 Feb 2024
Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation Huizhuo Yuan Zixiang Chen Kaixuan Ji Quanquan Gu 65 24 0 15 Feb 2024
PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models Fei Deng Qifei Wang Wei Wei Matthias Grundmann Tingbo Hou EGVM 19 15 0 13 Feb 2024