Directly Fine-Tuning Diffusion Models on Differentiable Rewards

29 September 2023

Amita Gajewar

Paul Vicol

G. Bansal

David J Fleet

ArXiv PDF HTML

Papers citing "Directly Fine-Tuning Diffusion Models on Differentiable Rewards"

24 / 124 papers shown

Title
Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases Ziyi Zhang Sen Zhang Yibing Zhan Yong Luo Yonggang Wen Dacheng Tao EGVM 41 8 0 13 Feb 2024
A Dense Reward View on Aligning Text-to-Image Diffusion with Preference Shentao Yang Tianqi Chen Mingyuan Zhou EGVM 34 22 0 13 Feb 2024
Implicit Diffusion: Efficient Optimization through Stochastic Sampling Pierre Marion Anna Korba Peter Bartlett Mathieu Blondel Valentin De Bortoli Arnaud Doucet Felipe Llinares-López Courtney Paquette Quentin Berthet 79 12 0 08 Feb 2024
DITTO: Diffusion Inference-Time T-Optimization for Music Generation Zachary Novack Julian McAuley Taylor Berg-Kirkpatrick Nicholas J. Bryan DiffM 34 33 0 22 Jan 2024
Large-scale Reinforcement Learning for Diffusion Models Yinan Zhang Eric Tzeng Yilun Du Dmitry Kislyuk VLM 33 31 0 20 Jan 2024
SCoFT: Self-Contrastive Fine-Tuning for Equitable Image Generation Zhixuan Liu Peter Schaldenbrand Beverley-Claire Okogwu Wenxuan Peng Youngsik Yun Andrew Hundt Jihie Kim Jean Oh 39 16 0 16 Jan 2024
Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation Seung Hyun Lee Yinxiao Li Junjie Ke Innfarn Yoo Han Zhang ... Junfeng He Gang Li Sangpil Kim Irfan Essa Feng Yang EGVM 38 18 0 11 Jan 2024
Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning Desai Xie Jiahao Li Hao Tan Xin Sun Zhixin Shu Yi Zhou Sai Bi Soren Pirk Arie E. Kaufman 37 8 0 21 Dec 2023
InstructVideo: Instructing Video Diffusion Models with Human Feedback Hangjie Yuan Shiwei Zhang Xiang Wang Yujie Wei Tao Feng Yining Pan Yingya Zhang Ziwei Liu Samuel Albanie Dong Ni VGen 24 42 0 19 Dec 2023
Optimizing Diffusion Noise Can Serve As Universal Motion Priors Korrawe Karunratanakul Konpat Preechakul Emre Aksan Thabo Beeler Supasorn Suwajanakorn Siyu Tang DiffM 31 37 0 19 Dec 2023
Generalized Contrastive Divergence: Joint Training of Energy-Based Model and Diffusion Model through Inverse Reinforcement Learning Sangwoong Yoon Dohyun Kwon Himchan Hwang Yung-Kyun Noh Frank C. Park 33 0 0 06 Dec 2023
InstructBooth: Instruction-following Personalized Text-to-Image Generation Daewon Chae Nokyung Park Jinkyu Kim Kimin Lee DiffM 24 11 0 04 Dec 2023
Enhancing Diffusion Models with Text-Encoder Reinforcement Learning Chaofeng Chen Annan Wang Haoning Wu Liang Liao Wenxiu Sun Qiong Yan Weisi Lin 33 10 0 27 Nov 2023
Diffusion Model Alignment Using Direct Preference Optimization Bram Wallace Meihua Dang Rafael Rafailov Linqi Zhou Aaron Lou Senthil Purushwalkam Stefano Ermon Caiming Xiong Shafiq R. Joty Nikhil Naik EGVM 50 227 0 21 Nov 2023
Behavior Optimized Image Generation Varun Khurana Yaman Kumar Singla J. Subramanian R. Shah Changyou Chen Zhiqiang Xu Balaji Krishnamurthy EGVM 8 4 0 18 Nov 2023
Finetuning Text-to-Image Diffusion Models for Fairness Xudong Shen Chao Du Tianyu Pang Min-Bin Lin Yongkang Wong Mohan S. Kankanhalli 26 50 0 11 Nov 2023
Aligning Text-to-Image Diffusion Models with Reward Backpropagation Mihir Prabhudesai Anirudh Goyal Deepak Pathak Katerina Fragkiadaki 37 111 0 05 Oct 2023
Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation Yuval Kirstain Adam Polyak Uriel Singer Shahbuland Matiana Joe Penna Omer Levy EGVM 168 351 0 02 May 2023
End-to-End Diffusion Latent Optimization Improves Classifier Guidance Bram Wallace Akash Gokul Stefano Ermon Nikhil Naik 124 70 0 23 Mar 2023
UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance Wei Li Xue Xu Xinyan Xiao Jiacheng Liu Hu Yang ... Zhanpeng Wang Zhifan Feng Qiaoqiao She Yajuan Lyu Hua-Hong Wu 121 29 0 28 Oct 2022
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 227 502 0 28 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 319 11,953 0 04 Mar 2022
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 255 4,781 0 24 Feb 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 286 1,595 0 18 Sep 2019