Do BERTs Learn to Use Browser User Interface? Exploring Multi-Step Tasks with Unified Vision-and-Language BERTs

15 March 2022

Papers citing "Do BERTs Learn to Use Browser User Interface? Exploring Multi-Step Tasks with Unified Vision-and-Language BERTs"

30 / 30 papers shown

Title
VUT: Versatile UI Transformer for Multi-Modal Multi-Task User Interface Modeling Yang Li Gang Li Xin Zhou Mostafa Dehghani A. Gritsenko MLLM 79 36 0 10 Dec 2021
Document AI: Benchmarks, Models and Applications Lei Cui Yiheng Xu Tengchao Lv Furu Wei VLM 66 73 0 16 Nov 2021
MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding Junlong Li Yiheng Xu Lei Cui Furu Wei VLM 3DGS 72 63 0 16 Oct 2021
UIBert: Learning Generic Multimodal Representations for UI Understanding Chongyang Bai Xiaoxue Zang Ying Xu Srinivas Sunkara Abhinav Rastogi Jindong Chen Blaise Agüera y Arcas 63 94 0 29 Jul 2021
Grounding 'Grounding' in NLP Khyathi Chandu Yonatan Bisk A. Black 80 54 0 04 Jun 2021
LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding Te-Lin Wu Cheng-rong Li Mingyang Zhang Tao Chen Spurthi Amba Hombaiah Michael Bendersky 60 14 0 16 Apr 2021
The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation Yuankai Qi Zizheng Pan Yicong Hong Ming-Hsuan Yang Anton Van Den Hengel Qi Wu LM&Ro 67 69 0 09 Apr 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 204 1,015 0 04 Mar 2021
UniT: Multimodal Multitask Learning with a Unified Transformer Ronghang Hu Amanpreet Singh ViT 89 300 0 22 Feb 2021
VisualMRC: Machine Reading Comprehension on Document Images Ryota Tanaka Kyosuke Nishida Sen Yoshida 91 145 0 27 Jan 2021
WebSRC: A Dataset for Web-Based Structural Reading Comprehension Xingyu Chen Zihan Zhao Lu Chen Danyang Zhang Jiabao Ji Ao Luo Yuxuan Xiong Kai Yu RALM 59 98 0 23 Jan 2021
ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces Zecheng He Srinivas Sunkara Xiaoxue Zang Ying Xu Lijuan Liu Nevan Wichers Gabriel Schubiner Ruby B. Lee Jindong Chen Blaise Agüera y Arcas 78 79 0 22 Dec 2020
A Recurrent Vision-and-Language BERT for Navigation Yicong Hong Qi Wu Yuankai Qi Cristian Rodriguez-Opazo Stephen Gould LM&Ro 104 301 0 26 Nov 2020
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 159 1,124 0 14 Sep 2020
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web Arjun Majumdar Ayush Shrivastava Stefan Lee Peter Anderson Devi Parikh Dhruv Batra LM&Ro 152 233 0 30 Apr 2020
Experience Grounds Language Yonatan Bisk Ari Holtzman Jesse Thomason Jacob Andreas Yoshua Bengio ... Angeliki Lazaridou Jonathan May Aleksandr Nisnevich Nicolas Pinto Joseph P. Turian 91 357 0 21 Apr 2020
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks Mohit Shridhar Jesse Thomason Daniel Gordon Yonatan Bisk Winson Han Roozbeh Mottaghi Luke Zettlemoyer Dieter Fox LM&Ro 114 770 0 03 Dec 2019
UNITER: UNiversal Image-TExt Representation Learning Yen-Chun Chen Linjie Li Licheng Yu Ahmed El Kholy Faisal Ahmed Zhe Gan Yu Cheng Jingjing Liu VLM OT 112 447 0 25 Sep 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 169 1,666 0 22 Aug 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 247 2,488 0 20 Aug 2019
VisualBERT: A Simple and Performant Baseline for Vision and Language Liunian Harold Li Mark Yatskar Da Yin Cho-Jui Hsieh Kai-Wei Chang VLM 148 1,962 0 09 Aug 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 234 3,693 0 06 Aug 2019
Know What You Don't Know: Unanswerable Questions for SQuAD Pranav Rajpurkar Robin Jia Percy Liang RALM ELM 290 2,845 0 11 Jun 2018
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 237 1,411 0 31 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,182 0 20 Apr 2018
An Overview of Multi-Task Learning in Deep Neural Networks Sebastian Ruder CVBM 153 2,830 0 15 Jun 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 524 4,492 0 18 Apr 2017
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 334 2,895 0 26 Sep 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 296 8,160 0 16 Jun 2016
VQA: Visual Question Answering Aishwarya Agrawal Jiasen Lu Stanislaw Antol Margaret Mitchell C. L. Zitnick Dhruv Batra Devi Parikh CoGe 214 5,497 0 03 May 2015