GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos

12 December 2023

Dima Damen

Papers citing "GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos"

41 / 41 papers shown

Title
What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation Learning Chi-Hsi Kung Frangil Ramirez Juhyung Ha Yi-Ting Chen David J. Crandall Yi-Hsuan Tsai 107 1 0 27 Mar 2025
Latent Beam Diffusion Models for Decoding Image Sequences Guilherme Fernandes Vasco Ramos Regev Cohen Idan Szpektor João Magalhães 141 1 0 26 Mar 2025
Adding Conditional Control to Text-to-Image Diffusion Models Lvmin Zhang Anyi Rao Maneesh Agrawala AI4CE 148 4,113 1 10 Feb 2023
InstructPix2Pix: Learning to Follow Image Editing Instructions Tim Brooks Aleksander Holynski Alexei A. Efros DiffM 200 1,799 0 17 Nov 2022
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation Nataniel Ruiz Yuanzhen Li Varun Jampani Yael Pritch Michael Rubinstein Kfir Aberman 279 2,861 0 25 Aug 2022
Prompt-to-Prompt Image Editing with Cross Attention Control Amir Hertz Ron Mokady J. Tenenbaum Kfir Aberman Yael Pritch Daniel Cohen-Or DiffM 186 1,769 0 02 Aug 2022
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion Rinon Gal Yuval Alaluf Yuval Atzmon Or Patashnik Amit H. Bermano Gal Chechik Daniel Cohen-Or 155 1,876 0 02 Aug 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 193 3,889 0 26 Jul 2022
Compositional Visual Generation with Composable Diffusion Models Nan Liu Shuang Li Yilun Du Antonio Torralba J. Tenenbaum DiffM CoGe 167 521 0 03 Jun 2022
Pretraining is All You Need for Image-to-Image Translation Tengfei Wang Ting Zhang Bo Zhang Hao Ouyang Dong Chen Qifeng Chen Fang Wen DiffM 243 178 0 25 May 2022
P3IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervision Henghui Zhao Isma Hadji Nikita Dvornik Konstantinos G. Derpanis Richard P. Wildes Allan D. Jepson 75 46 0 04 May 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 401 6,866 0 13 Apr 2022
Temporal Alignment Networks for Long-term Video Tengda Han Weidi Xie Andrew Zisserman AI4TS 81 87 0 06 Apr 2022
Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors Oran Gafni Adam Polyak Oron Ashual Shelly Sheynin Devi Parikh Yaniv Taigman DiffM 71 521 0 24 Mar 2022
Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videos Tomávs Souvcek Jean-Baptiste Alayrac Antoine Miech Ivan Laptev Josef Sivic 58 33 0 22 Mar 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 532 4,360 0 28 Jan 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 419 15,515 0 20 Dec 2021
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models Alex Nichol Prafulla Dhariwal Aditya A. Ramesh Pranav Shyam Pamela Mishkin Bob McGrew Ilya Sutskever Mark Chen 338 3,600 0 20 Dec 2021
Blended Diffusion for Text-driven Editing of Natural Images Omri Avrahami Dani Lischinski Ohad Fried DiffM 113 947 0 29 Nov 2021
Palette: Image-to-Image Diffusion Models Chitwan Saharia William Chan Huiwen Chang Chris A. Lee Jonathan Ho Tim Salimans David J. Fleet Mohammad Norouzi DiffM VLM 481 1,640 0 10 Nov 2021
DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation Gwanghyun Kim Taesung Kwon Jong Chul Ye DiffM 184 650 0 06 Oct 2021
SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations Chenlin Meng Yutong He Yang Song Jiaming Song Jiajun Wu Jun-Yan Zhu Stefano Ermon DiffM 144 1,489 0 02 Aug 2021
Variational Diffusion Models Diederik P. Kingma Tim Salimans Ben Poole Jonathan Ho DiffM 178 1,121 0 01 Jul 2021
Diffusion Models Beat GANs on Image Synthesis Prafulla Dhariwal Alex Nichol 224 7,857 0 11 May 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 397 4,953 0 24 Feb 2021
Learning Graph Embeddings for Compositional Zero-shot Learning Muhammad Ferjad Naeem Yongqin Xian Federico Tombari Zeynep Akata CoGe 57 140 0 03 Feb 2021
Pre-Trained Image Processing Transformer Hanting Chen Yunhe Wang Tianyu Guo Chang Xu Yiping Deng Zhenhua Liu Siwei Ma Chunjing Xu Chao Xu Wen Gao VLM ViT 136 1,676 0 01 Dec 2020
Denoising Diffusion Implicit Models Jiaming Song Chenlin Meng Stefano Ermon VLM DiffM 275 7,384 0 06 Oct 2020
Cross-domain Correspondence Learning for Exemplar-based Image Translation Peiying Zhang Bo Zhang Dong Chen Lu Yuan Fang Wen 77 239 0 12 Apr 2020
Action Modifiers: Learning from Adverbs in Instructional Videos Hazel Doughty Ivan Laptev W. Mayol-Cuevas Dima Damen 61 30 0 13 Dec 2019
Contextual Imagined Goals for Self-Supervised Robotic Learning Ashvin Nair Shikhar Bahl Alexander Khazatsky Vitchyr H. Pong Glen Berseth Sergey Levine SSL 72 68 0 23 Oct 2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Antoine Miech Dimitri Zhukov Jean-Baptiste Alayrac Makarand Tapaswi Ivan Laptev Josef Sivic VGen 110 1,200 0 07 Jun 2019
Semantic Image Synthesis with Spatially-Adaptive Normalization Taesung Park Ming-Yuan Liu Ting-Chun Wang Jun-Yan Zhu 156 2,688 0 18 Mar 2019
Toward Multimodal Image-to-Image Translation Jun-Yan Zhu Richard Y. Zhang Deepak Pathak Trevor Darrell Alexei A. Efros Oliver Wang Eli Shechtman 123 1,351 0 30 Nov 2017
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs Ting-Chun Wang Ming-Yuan Liu Jun-Yan Zhu Andrew Tao Jan Kautz Bryan Catanzaro SSeg 174 3,940 0 30 Nov 2017
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks Tao Xu Pengchuan Zhang Qiuyuan Huang Han Zhang Zhe Gan Xiaolei Huang Xiaodong He GAN ViT 108 1,716 0 28 Nov 2017
StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation Yunjey Choi Min-Je Choi M. Kim Jung-Woo Ha Sunghun Kim Jaegul Choo GAN 132 3,553 0 24 Nov 2017
Conditional Image Synthesis With Auxiliary Classifier GANs Augustus Odena C. Olah Jonathon Shlens GAN 424 3,213 0 30 Oct 2016
Deep multi-scale video prediction beyond mean square error Michaël Mathieu Camille Couprie Yann LeCun GAN 124 1,882 0 17 Nov 2015
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting Xingjian Shi Zhourong Chen Hao Wang Dit-Yan Yeung W. Wong W. Woo 563 7,992 0 13 Jun 2015
Unsupervised Learning of Video Representations using LSTMs Nitish Srivastava Elman Mansimov Ruslan Salakhutdinov SSL 132 2,591 0 16 Feb 2015