Cleaner Pretraining Corpus Curation with Neural Web Scraping

Cleaner Pretraining Corpus Curation with Neural Web Scraping

22 February 2024

Zhiyuan Liu

Papers citing "Cleaner Pretraining Corpus Curation with Neural Web Scraping"

6 / 6 papers shown

Title
Inner-Probe: Discovering Copyright-related Data Generation in LLM Architecture Qichao Ma Rui-Jie Zhu Peiye Liu Renye Yan Fahong Zhang ... Meng Li Zhaofei Yu Zongwei Wang Yimao Cai Tiejun Huang 50 0 0 06 Oct 2024
Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling David Grangier Simin Fan Skyler Seto Pierre Ablin 44 3 0 30 Sep 2024
VersiCode: Towards Version-controllable Code Generation Tongtong Wu Weigang Wu Xingyu Wang Kang Xu Suyu Ma Bo Jiang Ping Yang Zhenchang Xing Yuan-Fang Li Gholamreza Haffari 45 4 0 11 Jun 2024
Just Say the Name: Online Continual Learning with Category Names Only via Data Generation Minhyuk Seo Diganta Misra Seongwon Cho Minjae Lee Jonghyun Choi CLL 37 7 0 16 Mar 2024
WebFormer: The Web-page Transformer for Structure Information Extraction Qifan Wang Yi Fang Anirudh Ravula Fuli Feng Xiaojun Quan Dongfang Liu ViT 143 65 0 01 Feb 2022
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 261 1,996 0 31 Dec 2020