v1v2 (latest)

Rediscovering Hashed Random Projections for Efficient Quantization of Contextualized Sentence Embeddings

13 March 2023

Papers citing "Rediscovering Hashed Random Projections for Efficient Quantization of Contextualized Sentence Embeddings"

30 / 30 papers shown

Title
SparCL: Sparse Continual Learning on the Edge Zifeng Wang Zheng Zhan Yifan Gong Geng Yuan Wei Niu T. Jian Bin Ren Stratis Ioannidis Yanzhi Wang Jennifer Dy CLL 111 63 0 20 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 120 114 0 31 Aug 2022
FP8 Quantization: The Power of the Exponent Andrey Kuzmin M. V. Baalen Yuwei Ren Markus Nagel Jorn W. T. Peters Tijmen Blankevoort MQ 71 86 0 19 Aug 2022
EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generation Tao Ge Si-Qing Chen Furu Wei MoE 79 23 0 16 Feb 2022
Text and Code Embeddings by Contrastive Pre-Training Arvind Neelakantan Tao Xu Raul Puri Alec Radford Jesse Michael Han ... Tabarak Khan Toki Sherbakov Joanne Jang Peter Welinder Lilian Weng SSL AI4TS 379 444 0 24 Jan 2022
Towards Efficient Post-training Quantization of Pre-trained Language Models Haoli Bai Lu Hou Lifeng Shang Xin Jiang Irwin King Michael R. Lyu MQ 137 48 0 30 Sep 2021
Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing Haoyu He Xingjian Shi Jonas W. Mueller Zha Sheng Mu Li George Karypis 69 9 0 23 Sep 2021
SimCSE: Simple Contrastive Learning of Sentence Embeddings Tianyu Gao Xingcheng Yao Danqi Chen AILaw SSL 280 3,432 0 18 Apr 2021
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 425 1,057 0 17 Apr 2021
Language-agnostic BERT Sentence Embedding Fangxiaoyu Feng Yinfei Yang Daniel Cer N. Arivazhagan Wei Wang 176 915 0 03 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 904 42,463 0 28 May 2020
GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference Ali Hadi Zadeh Isak Edo Omar Mohamed Awad Andreas Moshovos MQ 67 188 0 08 May 2020
ProFormer: Towards On-Device LSH Projection Based Transformers Chinnadhurai Sankar Sujith Ravi Zornitsa Kozareva 55 9 0 13 Apr 2020
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices Zhiqing Sun Hongkun Yu Xiaodan Song Renjie Liu Yiming Yang Denny Zhou MQ 118 819 0 06 Apr 2020
Embedding Compression with Isotropic Iterative Quantization Siyu Liao Jie Chen Yanzhi Wang Qinru Qiu Bo Yuan MQ 49 13 0 11 Jan 2020
Q8BERT: Quantized 8Bit BERT Ofir Zafrir Guy Boudoukh Peter Izsak Moshe Wasserblat MQ 93 506 0 14 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 267 7,554 0 02 Oct 2019
Multilingual Universal Sentence Encoder for Semantic Retrieval Yinfei Yang Daniel Cer Amin Ahmad Mandy Guo Jax Law ... Steve Yuan Chris Tar Yun-hsuan Sung B. Strope R. Kurzweil 3DV 88 481 0 09 Jul 2019
On the Convergence of Adam and Beyond Sashank J. Reddi Satyen Kale Surinder Kumar 109 2,506 0 19 Apr 2019
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond Mikel Artetxe Holger Schwenk 3DV 158 1,018 0 26 Dec 2018
Random Projection in Deep Neural Networks P. Wójcik 39 5 0 22 Dec 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,324 0 11 Oct 2018
SentEval: An Evaluation Toolkit for Universal Sentence Representations Alexis Conneau Douwe Kiela 103 642 0 14 Mar 2018
Compressing Word Embeddings via Deep Compositional Code Learning Raphael Shu Hideki Nakayama 81 129 0 03 Nov 2017
Mixed Precision Training Paulius Micikevicius Sharan Narang Jonah Alben G. Diamos Erich Elsen ... Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh Hao Wu 178 1,806 0 10 Oct 2017
All-but-the-Top: Simple and Effective Postprocessing for Word Representations Jiaqi Mu S. Bhat Pramod Viswanath 95 311 0 05 Feb 2017
Binary Paragraph Vectors Karol Grzegorczyk Marcin Kurdziel 53 8 0 03 Nov 2016
Binary embeddings with structured hashed projections A. Choromańska K. Choromanski Mariusz Bojarski Tony Jebara Sanjiv Kumar Yann LeCun 67 33 0 16 Nov 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 367 19,745 0 09 Mar 2015
Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov Ilya Sutskever Kai Chen G. Corrado J. Dean NAI OCL 406 33,573 0 16 Oct 2013