RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,742 papers shown

Title
Neural models for Factual Inconsistency Classification with Explanations Tathagata Raha Mukund Choudhary Abhinav Menon Harshit Gupta KV Aditya Srivatsa Manish Gupta Vasudeva Varma 29 3 0 15 Jun 2023
Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understanding Le Zhang Rabiul Awal Aishwarya Agrawal CoGe VLM 61 13 0 15 Jun 2023
Description-Enhanced Label Embedding Contrastive Learning for Text Classification Kun Zhang Le Wu Guangyi Lv Enhong Chen Shulan Ruan Jing Liu Qing Cui Jun Zhou Meng Wang VLM 52 10 0 15 Jun 2023
Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models Saleh Soltan Andrew Rosenbaum Tobias Falke Qin Lu Anna Rumshisky Wael Hamza 71 1 0 14 Jun 2023
LoSh: Long-Short Text Joint Prediction Network for Referring Video Object Segmentation Linfeng Yuan Miaojing Shi Zijie Yue Qijun Chen VOS 72 10 0 14 Jun 2023
Iterative self-transfer learning: A general methodology for response time-history prediction based on small dataset Yongjia Xu Xinzheng Lu Yifan Fei Yuli Huang AI4TS AI4CE 38 15 0 14 Jun 2023
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models Ziqiao Ma Jiayi Pan J. Chai ObjD VLM 72 9 0 14 Jun 2023
Radiology-GPT: A Large Language Model for Radiology Zheng Liu Aoxiao Zhong Yiwei Li Longtao Yang Chao Ju ... Wen Liu Dinggang Shen Xiang Li Quanzheng Li Tianming Liu LM&MA MedIm AI4CE 143 60 0 14 Jun 2023
MiniLLM: Knowledge Distillation of Large Language Models Yuxian Gu Li Dong Furu Wei Minlie Huang ALM 149 78 0 14 Jun 2023
MUBen: Benchmarking the Uncertainty of Molecular Representation Models Yinghao Li Lingkai Kong Yuanqi Du Yue Yu Yuchen Zhuang Wenhao Mu Chao Zhang 101 11 0 14 Jun 2023
Unifying Large Language Models and Knowledge Graphs: A Roadmap Shirui Pan Linhao Luo Yufei Wang Chen Chen Jiapu Wang Xindong Wu KELM 160 787 0 14 Jun 2023
Operationalising Representation in Natural Language Processing J. Harding 121 13 0 14 Jun 2023
Language models are not naysayers: An analysis of language models on negation benchmarks Thinh Hung Truong Timothy Baldwin Karin Verspoor Trevor Cohn 122 60 0 14 Jun 2023
Neural Mixed Effects for Nonlinear Personalized Predictions T. Wörtwein Nicholas B. Allen Lisa B. Sheeber Randy P. Auerbach J. Cohn Louis-Philippe Morency 86 7 0 13 Jun 2023
AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks Alexander Tornede Difan Deng Theresa Eimer Joseph Giovanelli Aditya Mohan ... Sarah Segel Daphne Theodorakopoulos Tanja Tornede Henning Wachsmuth Marius Lindauer 119 24 0 13 Jun 2023
FLamE: Few-shot Learning from Natural Language Explanations Yangqiaoyu Zhou Yiming Zhang Chenhao Tan LRM FAtt 95 11 0 13 Jun 2023
GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition Yu Pan Yanni Hu Yuguang Yang Wen Fei Jixun Yao Heng Lu Lei Ma Jianjun Zhao VLM 122 12 0 13 Jun 2023
Urania: Visualizing Data Analysis Pipelines for Natural Language-Based Data Exploration Yi Guo Nana Cao Xiaoyu Qi Haoyang Li Danqing Shi Jing Zhang Qing Chen Daniel Weiskopf 73 5 0 13 Jun 2023
Rethink the Effectiveness of Text Data Augmentation: An Empirical Analysis Zhengxiang Shi Aldo Lipani 80 2 0 13 Jun 2023
Is Anisotropy Inherent to Transformers? Nathan Godey Eric Villemonte de la Clergerie Benoît Sagot 72 4 0 13 Jun 2023
Rank-Aware Negative Training for Semi-Supervised Text Classification Ahmed Murtadha Shengfeng Pan Wen Bo Jianlin Su Xinxin Cao Wenze Zhang Yunfeng Liu 75 9 0 13 Jun 2023
Detect Depression from Social Networks with Sentiment Knowledge Sharing Yan Shi Yao Tian Chengwei Tong Chunyan Zhu Qian-qian Li Mengzhu Zhang Wei Zhao Yong Liao Pengyuan Zhou 19 2 0 13 Jun 2023
Improving Opinion-based Question Answering Systems Through Label Error Detection and Overwrite Xiao Yang A. Mohamed Shashank Jain Stanislav Peshterliev Debojeet Chatterjee Hanwen Zha Nikita Bhalla Gagan Aneja Pranab Mohanty 21 0 0 13 Jun 2023
PauseSpeech: Natural Speech Synthesis via Pre-trained Language Model and Pause-based Prosody Modeling Ji-Sang Hwang Sang-Hoon Lee Seong-Whan Lee 66 4 0 13 Jun 2023
Gender-Inclusive Grammatical Error Correction through Augmentation Gunnar Lund Kostiantyn Omelianchuk Igor Samokhin 69 8 0 12 Jun 2023
Textual Augmentation Techniques Applied to Low Resource Machine Translation: Case of Swahili Catherine Gitau VUkosi Marivate 59 3 0 12 Jun 2023
EriBERTa: A Bilingual Pre-Trained Language Model for Clinical Natural Language Processing Iker de la Iglesia Aitziber Atutxa Koldo Gojenola Ander Barrena 58 2 0 12 Jun 2023
Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training Lorenzo Baraldi Roberto Amoroso Marcella Cornia Lorenzo Baraldi Andrea Pilzer Rita Cucchiara 153 2 0 12 Jun 2023
RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation Shuai Liu Hyundong Justin Cho Marjorie Freedman Xuezhe Ma Jonathan May 62 26 0 12 Jun 2023
LTCR: Long-Text Chinese Rumor Detection Dataset Ziyang Ma Mengsha Liu Guian Fang Yingxiao Shen 49 1 0 12 Jun 2023
Augmenting Language Models with Long-Term Memory Weizhi Wang Li Dong Hao Cheng Xiaodong Liu Xifeng Yan Jianfeng Gao Furu Wei KELM RALM 104 96 0 12 Jun 2023
Measuring Sentiment Bias in Machine Translation Kai Hartung Aaricia Herygers Shubham Kurlekar Khabbab Zakaria Taylan Volkan Sören Gröttrup Munir Georges AI4CE 70 5 0 12 Jun 2023
Global and Local Semantic Completion Learning for Vision-Language Pre-training Rong-Cheng Tu Yatai Ji Jie Jiang Weijie Kong Chengfei Cai Wenzhe Zhao Hongfa Wang Yujiu Yang Wei Liu VLM 96 4 0 12 Jun 2023
Deep Model Compression Also Helps Models Capture Ambiguity Hancheol Park Jong C. Park 65 2 0 12 Jun 2023
Revisiting Token Pruning for Object Detection and Instance Segmentation Yifei Liu Mathias Gehrig Nico Messikommer Marco Cannici Davide Scaramuzza ViT VLM 112 27 0 12 Jun 2023
Transformers learn through gradual rank increase Enric Boix-Adserà Etai Littwin Emmanuel Abbe Samy Bengio J. Susskind 102 37 0 12 Jun 2023
Recurrent Attention Networks for Long-text Modeling Xianming Li Zongxi Li Xiaotian Luo Haoran Xie Xing Lee Yingbin Zhao Fu Lee Wang Qing Li RALM 92 15 0 12 Jun 2023
Multimodal Audio-textual Architecture for Robust Spoken Language Understanding Anderson R. Avila Mehdi Rezagholizadeh Chao Xing 58 1 0 12 Jun 2023
A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks Saidul Islam Hanae Elmekki Ahmed Elsebai Jamal Bentahar Najat Drawel Gaith Rjoub Witold Pedrycz ViT MedIm 89 210 0 11 Jun 2023
AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing Asaad Alghamdi Xinyu Duan Wei Jiang Zhenhai Wang Yimeng Wu ... Yifei Zheng Mehdi Rezagholizadeh Baoxing Huai Peilun Cheng Abbas Ghaddar VLM 52 8 0 11 Jun 2023
Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression Wen Wu Chuxu Zhang P. Woodland UQCV UD EDL 66 12 0 11 Jun 2023
QUERT: Continual Pre-training of Language Model for Query Understanding in Travel Domain Search Jian Xie Yidan Liang Jingping Liu Yanghua Xiao Baohua Wu Shenghua Ni VLM LRM 90 9 0 11 Jun 2023
EaSyGuide : ESG Issue Identification Framework leveraging Abilities of Generative Large Language Models Hanwool Albert Lee Jonghyun Choi Sohyeon Kwon Sungbum Jung 29 3 0 11 Jun 2023
GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model Shicheng Tan Weng Lam Tam Yuanchun Wang Wenwen Gong Yang Yang ... Jiahao Liu Jingang Wang Shuo Zhao Peng Zhang Jie Tang ALM MoE 80 13 0 11 Jun 2023
Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method Shicheng Tan Weng Lam Tam Yuanchun Wang Wenwen Gong Shuo Zhao Peng Zhang Jie Tang VLM 49 1 0 11 Jun 2023
Towards Diverse and Effective Question-Answer Pair Generation from Children Storybooks Sugyeong Eo Hyeonseok Moon Jinsung Kim Yuna Hur Jeongwook Kim Song-Eun Lee Changwoo Chun Sungsoo Park Heu-Jeoung Lim AI4Ed 107 7 0 11 Jun 2023
RoBERTweet: A BERT Language Model for Romanian Tweets Iulian-Marius Tuaiatu Andrei-Marius Avram Dumitru-Clementin Cercel Florin-Catalin Pop 47 1 0 11 Jun 2023
Multi-modal Pre-training for Medical Vision-language Understanding and Generation: An Empirical Study with A New Benchmark Li Xu Bo Liu Ameer Hamza Khan Lu Fan Xiao-Ming Wu LM&MA 65 9 0 10 Jun 2023
Annotation-Inspired Implicit Discourse Relation Classification with Auxiliary Discourse Connective Generation Wei Liu Michael Strube 59 16 0 10 Jun 2023
Enhancing Low Resource NER Using Assisting Language And Transfer Learning Maithili Sabane Aparna Ranade Onkar Litake Parth Patil Raviraj Joshi Dipali M. Kadam 64 5 0 10 Jun 2023