Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval

27 May 2022

Pascal Notin

M. Dias

J. Frazer

Javier Marchena-Hurtado

Papers citing "Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval"

21 / 21 papers shown

Title
MassSpecGym: A benchmark for the discovery and identification of molecules Roman Bushuiev Anton Bushuiev Niek F. de Jonge A. Young Fleming Kretschmer ... Justin J. J. van der Hooft Michael A. Stravs Sebastian Böcker Josef Sivic Tomáš Pluskal 138 5 0 17 Feb 2025
Metalic: Meta-Learning In-Context with Protein Language Models Jacob Beck Shikha Surana Manus McAuliffe Oliver Bent Thomas D. Barrett Juan Jose Garau Luis Paul Duckworth AI4CE 68 1 0 10 Oct 2024
ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding Yijia Xiao Edward Sun Yiqiao Jin Qifan Wang Wei Wang 83 15 0 21 Aug 2024
ProtChatGPT: Towards Understanding Proteins with Large Language Models Chao Wang Hehe Fan Ruijie Quan Yi Yang 79 15 0 15 Feb 2024
A Text-guided Protein Design Framework Shengchao Liu Yanjing Li Zhuoxinran Li A. Gitter Yutao Zhu ... Arvind Ramanathan Chaowei Xiao Jian Tang Hongyu Guo Anima Anandkumar 108 70 0 09 Feb 2023
RITA: a Study on Scaling Up Generative Protein Sequence Models Daniel Hesslow Niccoló Zanichelli Pascal Notin Iacopo Poli D. Marks 45 99 0 11 May 2022
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 244 1,086 0 08 Dec 2021
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 251 156 0 17 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 328 774 0 27 Aug 2021
ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing Ahmed Elnaggar M. Heinzinger Christian Dallago Ghalia Rehawi Yu Wang ... Tamas B. Fehér Christoph Angerer Martin Steinegger D. Bhowmik B. Rost DRL 64 947 0 13 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 829 42,332 0 28 May 2020
Dense Passage Retrieval for Open-Domain Question Answering Vladimir Karpukhin Barlas Oğuz Sewon Min Patrick Lewis Ledell Yu Wu Sergey Edunov Danqi Chen Wen-tau Yih RALM 198 3,778 0 10 Apr 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 140 2,114 0 10 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 608 4,893 0 23 Jan 2020
Generalization through Memorization: Nearest Neighbor Language Models Urvashi Khandelwal Omer Levy Dan Jurafsky Luke Zettlemoyer M. Lewis RALM 166 837 0 01 Nov 2019
Evaluating Protein Transfer Learning with TAPE Roshan Rao Nicholas Bhattacharya Neil Thomas Yan Duan Xi Chen John F. Canny Pieter Abbeel Yun S. Song SSL 94 803 0 19 Jun 2019
Latent Retrieval for Weakly Supervised Open Domain Question Answering Kenton Lee Ming-Wei Chang Kristina Toutanova RALM 112 1,017 0 01 Jun 2019
Learning protein sequence embeddings using information from structure Tristan Bepler Bonnie Berger 73 290 0 22 Feb 2019
Unbounded cache model for online language modeling with open vocabulary Edouard Grave Moustapha Cissé Armand Joulin KELM CLL 46 63 0 07 Nov 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 725 132,199 0 12 Jun 2017
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 480 43,685 0 17 Sep 2014