v1v2v3 (latest)

Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach

24 June 2024

Papers citing "Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach"

34 / 34 papers shown

Title
PixelWeb: The First Web GUI Dataset with Pixel-Wise Labels Qi Yang Weichen Bi Haiyang Shen Y. Guo Yudong Han 83 0 0 23 Apr 2025
Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping Jingyu Xiao Yuxuan Wan Yintong Huo Zihan Wang Xinyi Xu Wenxuan Wang Zhiyao Xu Yansen Wang Michael R. Lyu 161 1 0 21 Feb 2025
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs Wenxuan Wang Xiaoyuan Liu Kuiyi Gao Jen-tse Huang Youliang Yuan Pinjia He Shuai Wang Zhaopeng Tu 96 2 0 16 Feb 2025
DynEx: Dynamic Code Synthesis with Structured Design Exploration for Accelerated Exploratory Programming Jenny Ma Karthik Sreedhar Vivian Liu Sitong Wang Pedro Alejandro Perez Riya Sahni Lydia B. Chilton 78 2 0 01 Oct 2024
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output Pan Zhang Xiaoyi Dong Yuhang Zang Yuhang Cao Rui Qian ... Kai Chen Jifeng Dai Yu Qiao Dahua Lin Jiaqi Wang 108 117 0 03 Jul 2024
Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models Hyungjoo Chae Yeonghyeon Kim Seungone Kim Kai Tzu-iunn Ong Beong-woo Kwak ... Seonghwan Kim Taeyoon Kwon Jiwan Chung Youngjae Yu Jinyoung Yeo LRM ReLM 71 16 0 03 Apr 2024
Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset Hugo Laurençon Léo Tronchon Victor Sanh VLM 81 43 0 14 Mar 2024
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs Shengbang Tong Zhuang Liu Yuexiang Zhai Yi-An Ma Yann LeCun Saining Xie VLM MLLM 105 347 0 11 Jan 2024
The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision) Zhengyuan Yang Linjie Li Kevin Qinghong Lin Jianfeng Wang Chung-Ching Lin Nasim Shakouri Mahmoudabadi Lijuan Wang LM&MA 75 642 0 29 Sep 2023
ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation Xueying Du Wentai Deng Kaixin Wang Hanlin Wang Junwei Liu Yixuan Chen Jiayi Feng Chaofeng Sha Xin Peng Xin Peng ELM ALM 65 149 0 03 Aug 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Xu Enhong Chen MLLM LRM 133 604 0 23 Jun 2023
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback Yann Dubois Xuechen Li Rohan Taori Tianyi Zhang Ishaan Gulrajani Jimmy Ba Carlos Guestrin Percy Liang Tatsunori B. Hashimoto ALM 137 605 0 22 May 2023
LIMA: Less Is More for Alignment Chunting Zhou Pengfei Liu Puxin Xu Srini Iyer Jiao Sun ... Susan Zhang Gargi Ghosh M. Lewis Luke Zettlemoyer Omer Levy ALM 106 847 0 18 May 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 139 2,095 0 11 May 2023
Semantic Compression With Large Language Models Henry Gilbert Michael Sandborn Douglas C. Schmidt Jesse Spencer-Smith Jules White 49 23 0 25 Apr 2023
Teaching Large Language Models to Self-Debug Xinyun Chen Maxwell Lin Nathanael Scharli Denny Zhou LRM 121 702 0 11 Apr 2023
Exploring Distributional Shifts in Large Language Models for Code Analysis Shushan Arakelyan Rocktim Jyoti Das Yi Mao Xiang Ren ALM 93 19 0 16 Mar 2023
PaLM-E: An Embodied Multimodal Language Model Danny Driess F. Xia Mehdi S. M. Sajjadi Corey Lynch Aakanksha Chowdhery ... Marc Toussaint Klaus Greff Andy Zeng Igor Mordatch Peter R. Florence LM&Ro 117 1,667 0 06 Mar 2023
Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks Wenhu Chen Xueguang Ma Xinyi Wang William W. Cohen ReLM ReCod LRM 165 824 0 22 Nov 2022
On the Transferability of Pre-trained Language Models for Low-Resource Programming Languages Fuxiang Chen F. Fard David Lo T. Bryksin 62 48 0 05 Apr 2022
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis Erik Nijkamp Bo Pang Hiroaki Hayashi Lifu Tu Haiquan Wang Yingbo Zhou Silvio Savarese Caiming Xiong ELM 150 1,037 0 25 Mar 2022
Assemble Foundation Models for Automatic Code Summarization Jian Gu P. Salza H. Gall 77 35 0 13 Jan 2022
Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots Jason Wu Xiaoyi Zhang Jeffrey Nichols Jeffrey P. Bigham 3DV 183 72 0 17 Sep 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 236 5,647 0 07 Jul 2021
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 183 788 0 25 Jun 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 972 29,810 0 26 Feb 2021
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning Jun Chen Han Guo Kai Yi Boyang Albert Li Mohamed Elhoseiny VLM 138 227 0 20 Feb 2021
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis Shuo Ren Daya Guo Shuai Lu Long Zhou Shujie Liu Duyu Tang Neel Sundaresan M. Zhou Ambrosio Blanco Shuai Ma ELM 115 542 0 22 Sep 2020
Code Generation as a Dual Task of Code Summarization Bolin Wei Ge Li Xin Xia Zhiyi Fu Zhi Jin 76 226 0 14 Oct 2019
Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation Victor Le Pochat Tom van Goethem Samaneh Tajalizadehkhoob Maciej Korczyński Wouter Joosen 62 654 0 04 Jun 2018
Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow Pengcheng Yin Bowen Deng Edgar Chen Bogdan Vasilescu Graham Neubig 63 304 0 23 May 2018
Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps Kevin Moran Carlos Bernal-Cárdenas Michael Curcio R. Bonett Denys Poshyvanyk 77 206 0 07 Feb 2018
pix2code: Generating Code from a Graphical User Interface Screenshot Tony Beltramelli 61 274 0 22 May 2017
Sequence to Sequence Learning with Neural Networks Ilya Sutskever Oriol Vinyals Quoc V. Le AIMat 446 20,590 0 10 Sep 2014