Improving Length-Generalization in Transformers via Task Hinting

1 October 2023

Papers citing "Improving Length-Generalization in Transformers via Task Hinting"

8 / 8 papers shown

Title
Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzles Kulin Shah Nishanth Dikkala Xin Wang Rina Panigrahy ELM ReLM LRM 34 9 0 16 Sep 2024
Block-level Text Spotting with LLMs Ganesh Bannur Bharadwaj Amrutur 33 0 0 19 Jun 2024
Transformers Can Achieve Length Generalization But Not Robustly Yongchao Zhou Uri Alon Xinyun Chen Xuezhi Wang Rishabh Agarwal Denny Zhou 52 36 0 14 Feb 2024
The Expressibility of Polynomial based Attention Scheme Zhao Song Guangyi Xu Junze Yin 32 5 0 30 Oct 2023
What Algorithms can Transformers Learn? A Study in Length Generalization Hattie Zhou Arwen Bradley Etai Littwin Noam Razin Omid Saremi Josh Susskind Samy Bengio Preetum Nakkiran 34 110 0 24 Oct 2023
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu (Allen) Zhang Ying Sheng Dinesh Manocha Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 52 254 0 24 Jun 2023
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 395 8,495 0 28 Jan 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 253 695 0 27 Aug 2021