The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only

1 June 2023

Guilherme Penedo

Quentin Malartic

Daniel Hesslow

Ruxandra-Aimée Cojocaru

Papers citing "The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only"

50 / 587 papers shown

Title
MathDivide: Improved mathematical reasoning by large language models S. Srivastava Ashutosh Gandhi LRM ReLM 38 0 0 12 May 2024
InsightNet: Structured Insight Mining from Customer Feedback Sandeep Sricharan Mukku Manan Soni Jitenkumar Rana Chetan Aggarwal Promod Yenigalla Rashmi Patange Shyam Mohan 27 1 0 12 May 2024
Linearizing Large Language Models Jean-Pierre Mercat Igor Vasiljevic Sedrick Scott Keh Kushal Arora Achal Dave Adrien Gaidon Thomas Kollar 46 19 0 10 May 2024
OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruning Dan Qiao Yi Su Pinzheng Wang Jing Ye Wen Xie ... Wenliang Chen Guohong Fu Guodong Zhou Qiaoming Zhu Min Zhang MQ 35 0 0 09 May 2024
Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models Luke Merrick Danmei Xu Gaurav Nuti Daniel Campos 17 24 0 08 May 2024
ChuXin: 1.6B Technical Report Xiaomin Zhuang Yufan Jiang Qiaozhi He Zhihua Wu ALM 43 0 0 08 May 2024
Zero-shot LLM-guided Counterfactual Generation for Text Amrita Bhattacharjee Raha Moraffah Joshua Garland Huan Liu 46 4 0 08 May 2024
Large Language Models for Cyber Security: A Systematic Literature Review HanXiang Xu Shenao Wang Ningke Li Kaidi Wang Yanjie Zhao Kai Chen Ting Yu Yang Liu Haoyu Wang 37 23 0 08 May 2024
Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense Siqi Shen Lajanugen Logeswaran Moontae Lee Honglak Lee Soujanya Poria Rada Mihalcea AI4MH LRM ELM 32 24 0 07 May 2024
ERAGent: Enhancing Retrieval-Augmented Language Models with Improved Accuracy, Efficiency, and Personalization Yunxiao Shi Xing Zi Zijing Shi Haimin Zhang Qiang Wu Min Xu RALM 38 16 0 06 May 2024
Beyond Performance: Quantifying and Mitigating Label Bias in LLMs Philipp Benz Maitreya Patel 129 10 0 04 May 2024
101 Billion Arabic Words Dataset Manel Aloui Hasna Chouikhi Ghaith Chaabane Haithem Kchaou Chehir Dhaouadi 44 1 0 29 Apr 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 95 139 0 29 Apr 2024
Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities Kazuki Fujii Taishi Nakamura Mengsay Loem Hiroki Iida Masanari Ohi Kakeru Hattori Hirai Shota Sakae Mizuki Rio Yokota Naoaki Okazaki CLL 41 54 0 27 Apr 2024
Building a Large Japanese Web Corpus for Large Language Models Naoaki Okazaki Kakeru Hattori Hirai Shota Hiroki Iida Masanari Ohi Kazuki Fujii Taishi Nakamura Mengsay Loem Rio Yokota Sakae Mizuki 55 7 0 27 Apr 2024
CoMM: Collaborative Multi-Agent, Multi-Reasoning-Path Prompting for Complex Problem Solving Pei Chen Boran Han Shuai Zhang LRM LLMAG 40 4 0 26 Apr 2024
Text Quality-Based Pruning for Efficient Training of Language Models Vasu Sharma Karthik Padthe Newsha Ardalani Kushal Tirumala Russell Howes ... Po-Yao Huang Shang-Wen Li Armen Aghajanyan Gargi Ghosh Luke Zettlemoyer 54 6 0 26 Apr 2024
Tele-FLM Technical Report Xiang Li Yiqun Yao Xin Jiang Xuezhi Fang Chao Wang ... Yequan Wang Zhongjiang He Zhongyuan Wang Xuelong Li Tiejun Huang 38 3 0 25 Apr 2024
Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities Xiaomin Yu Yezhaohui Wang Yanfang Chen Zhen Tao Dinghao Xi Shichao Song Simin Niu Zhiyu Li 69 8 0 25 Apr 2024
Classifying Human-Generated and AI-Generated Election Claims in Social Media A. Dmonte Marcos Zampieri Kevin Lybarger Massimiliano Albanese Genya Coulter DeLMO 47 2 0 24 Apr 2024
CORM: Cache Optimization with Recent Message for Large Language Model Inference Jincheng Dai Zhuowei Huang Haiyun Jiang Chen Chen Deng Cai Wei Bi Shuming Shi 38 3 0 24 Apr 2024
Automated Multi-Language to English Machine Translation Using Generative Pre-Trained Transformers Elijah Pelofske Vincent Urias L. Liebrock 32 0 0 23 Apr 2024
OpenELM: An Efficient Language Model Family with Open Training and Inference Framework Sachin Mehta Mohammad Hossein Sekhavat Qingqing Cao Maxwell Horton Yanzi Jin ... Iman Mirzadeh Mahyar Najibi Dmitry Belenko Peter Zatloukal Mohammad Rastegari OSLM AIFin 38 50 0 22 Apr 2024
AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs Anselm Paulus Arman Zharmagambetov Chuan Guo Brandon Amos Yuandong Tian AAML 58 56 0 21 Apr 2024
Large Language Models in Targeted Sentiment Analysis Nicolay Rusnachenko A. Golubev Natalia Loukachevitch LRM 32 3 0 18 Apr 2024
OpenBezoar: Small, Cost-Effective and Open Models Trained on Mixes of Instruction Data Chandeepa Dissanayake Lahiru Lowe Sachith Gunasekara Yasiru Ratnayake MoE ALM 40 1 0 18 Apr 2024
Context-Aware Siamese Networks for Efficient Emotion Recognition in Conversation Barbara Gendron Gaël Guibon 25 0 0 17 Apr 2024
Small Language Models are Good Too: An Empirical Study of Zero-Shot Classification Pierre Lepagnol Thomas Gerald Sahar Ghannay Christophe Servan Sophie Rosset 49 7 0 17 Apr 2024
Fewer Truncations Improve Language Modeling Hantian Ding Zijian Wang Giovanni Paolini Varun Kumar Anoop Deoras Dan Roth Stefano Soatto 61 13 0 16 Apr 2024
HLAT: High-quality Large Language Model Pre-trained on AWS Trainium Haozheng Fan Hao Zhou Guangtai Huang Parameswaran Raman Xinwei Fu Gaurav Gupta Dhananjay Ram Yida Wang Jun Huan 48 5 0 16 Apr 2024
I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey Noah Lewis J. L. Bez Suren Byna 57 0 0 16 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 46 8 0 13 Apr 2024
Experimental Design for Active Transductive Inference in Large Language Models Subhojyoti Mukherjee Anusha Lalitha Aniket Deshmukh Ge Liu Yifei Ma B. Kveton LRM 43 1 0 12 Apr 2024
CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models Je-Yong Lee Donghyun Lee Genghan Zhang Mo Tiwari Azalia Mirhoseini 44 15 0 12 Apr 2024
JetMoE: Reaching Llama2 Performance with 0.1M Dollars Yikang Shen Zhen Guo Tianle Cai Zengyi Qin MoE ALM 46 28 0 11 Apr 2024
Groundedness in Retrieval-augmented Long-form Generation: An Empirical Study Alessandro Stolfo RALM HILM 34 6 0 10 Apr 2024
Low-Cost Generation and Evaluation of Dictionary Example Sentences Bill Cai Clarence Boon Liang Ng Daniel Tan Shelvia Hotama 17 3 0 09 Apr 2024
Order-Based Pre-training Strategies for Procedural Text Understanding Abhilash Nandy Yash Kulkarni Pawan Goyal Niloy Ganguly 26 2 0 06 Apr 2024
Sailor: Open Language Models for South-East Asia Longxu Dou Qian Liu Guangtao Zeng Jia Guo Jiahui Zhou Wei Lu Min-Bin Lin LRM 40 8 0 04 Apr 2024
MuLan: A Study of Fact Mutability in Language Models Constanza Fierro Nicolas Garneau Emanuele Bugliarello Yova Kementchedjhieva Anders Søgaard KELM HILM 35 7 0 03 Apr 2024
CSEPrompts: A Benchmark of Introductory Computer Science Prompts Md. Nishat Raihan Dhiman Goswami Sadiya Sayara Chowdhury Puspo Christian D. Newman Tharindu Ranasinghe Marcos Zampieri ELM 44 2 0 03 Apr 2024
Towards Large Language Model driven Reference-less Translation Evaluation for English and Indian Languages Vandan Mujadia Pruthwik Mishra Arafat Ahsan D. Sharma ELM 42 2 0 03 Apr 2024
Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data Parth Patwa Simone Filice Zhiyu Zoey Chen Giuseppe Castellucci Oleg Rokhlenko S. Malmasi 14 7 0 03 Apr 2024
Comparative Study of Domain Driven Terms Extraction Using Large Language Models Sandeep Chataut Tuyen Do Bichar Dip Shrestha Gurung Shiva Aryal Anup Khanal Carol Lushbough Etienne Z. Gnimpieba 28 10 0 02 Apr 2024
Using Large Language Models to Understand Telecom Standards Athanasios Karapantelakis Mukesh Shakur Alexandros Nikou Farnaz Moradi Christian Orlog Fitsum Gaim Henrik Holm Doumitrou Daniil Nimara Vincent Huang 33 13 0 02 Apr 2024
GUARD-D-LLM: An LLM-Based Risk Assessment Engine for the Downstream uses of LLMs Sundaraparipurnan Narayanan Sandeep Vishwakarma 42 3 0 02 Apr 2024
A Study on Scaling Up Multilingual News Framing Analysis Syeda Sabrina Akter Antonios Anastasopoulos 29 0 0 01 Apr 2024
A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias Yuemei Xu Ling Hu Jiayi Zhao Zihan Qiu Yuqi Ye Hanwen Gu LRM 27 36 0 01 Apr 2024
Bailong: Bilingual Transfer Learning based on QLoRA and Zip-tie Embedding Lung-Chuan Chen Zong-Ru Li ALM 34 0 0 01 Apr 2024
Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization Hritik Bansal Ashima Suvarna Gantavya Bhatt Nanyun Peng Kai-Wei Chang Aditya Grover ALM 64 9 0 31 Mar 2024