On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning

17 November 2022

Papers citing "On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning"

23 / 23 papers shown

Title
Pre-Trained Language Models for Interactive Decision-Making Shuang Li Xavier Puig Chris Paxton Yilun Du Clinton Jia Wang ... Anima Anandkumar Jacob Andreas Igor Mordatch Antonio Torralba Yuke Zhu LM&Ro 93 257 0 03 Feb 2022
Can Wikipedia Help Offline Reinforcement Learning? Machel Reid Yutaro Yamada S. Gu 3DV RALM OffRL 184 96 0 28 Jan 2022
Offline Reinforcement Learning as One Big Sequence Modeling Problem Michael Janner Qiyang Li Sergey Levine OffRL 116 673 0 03 Jun 2021
Pretrained Transformers as Universal Computation Engines Kevin Lu Aditya Grover Pieter Abbeel Igor Mordatch 48 221 0 09 Mar 2021
Decoupling Representation Learning from Reinforcement Learning Adam Stooke Kimin Lee Pieter Abbeel Michael Laskin SSL DRL 345 345 0 14 Sep 2020
What is being transferred in transfer learning? Behnam Neyshabur Hanie Sedghi Chiyuan Zhang 81 519 0 26 Aug 2020
Finding Universal Grammatical Relations in Multilingual BERT Ethan A. Chi John Hewitt Christopher D. Manning 38 151 0 09 May 2020
Similarity Analysis of Contextual Word Representation Models John M. Wu Yonatan Belinkov Hassan Sajjad Nadir Durrani Fahim Dalvi James R. Glass 84 75 0 03 May 2020
The Information Bottleneck Problem and Its Applications in Machine Learning Ziv Goldfeld Yury Polyanskiy 51 133 0 30 Apr 2020
What Happens To BERT Embeddings During Fine-tuning? Amil Merchant Elahe Rahimtoroghi Ellie Pavlick Ian Tenney 65 187 0 29 Apr 2020
D4RL: Datasets for Deep Data-Driven Reinforcement Learning Justin Fu Aviral Kumar Ofir Nachum George Tucker Sergey Levine GP OffRL 210 1,359 0 15 Apr 2020
On the Cross-lingual Transferability of Monolingual Representations Mikel Artetxe Sebastian Ruder Dani Yogatama 165 793 0 25 Oct 2019
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML Aniruddh Raghu M. Raghu Samy Bengio Oriol Vinyals 300 644 0 19 Sep 2019
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives Elena Voita Rico Sennrich Ivan Titov 266 186 0 03 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 558 2,660 0 03 Sep 2019
What do you learn from context? Probing for sentence structure in contextualized word representations Ian Tenney Patrick Xia Berlin Chen Alex Jinpeng Wang Adam Poliak ... Najoung Kim Benjamin Van Durme Samuel R. Bowman Dipanjan Das Ellie Pavlick 170 858 0 15 May 2019
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent Karthik A. Sankararaman Soham De Zheng Xu Wenjie Huang Tom Goldstein ODL 59 104 0 15 Apr 2019
Linguistic Knowledge and Transferability of Contextual Representations Nelson F. Liu Matt Gardner Yonatan Belinkov Matthew E. Peters Noah A. Smith 113 730 0 21 Mar 2019
Transfusion: Understanding Transfer Learning for Medical Imaging M. Raghu Chiyuan Zhang Jon M. Kleinberg Samy Bengio MedIm 75 982 0 14 Feb 2019
On the importance of single directions for generalization Ari S. Morcos David Barrett Neil C. Rabinowitz M. Botvinick 64 333 0 19 Mar 2018
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 338 10,467 0 21 Jul 2016
Deep Learning and the Information Bottleneck Principle Naftali Tishby Noga Zaslavsky DRL 165 1,580 0 09 Mar 2015
How transferable are features in deep neural networks? J. Yosinski Jeff Clune Yoshua Bengio Hod Lipson OOD 196 8,321 0 06 Nov 2014