The geometry of hidden representations of large transformer models

1 February 2023

Papers citing "The geometry of hidden representations of large transformer models"

35 / 35 papers shown

Title
ProtFlow: Fast Protein Sequence Design via Flow Matching on Compressed Protein Language Model Embeddings Zitai Kong Yiheng Zhu Yinlong Xu Hanjing Zhou Mingzhe Yin Jialu Wu Hongxia Xu Chang-Yu Hsieh Tingjun Hou Jian Wu 31 0 0 15 Apr 2025
Bridging the Dimensional Chasm: Uncover Layer-wise Dimensional Reduction in Transformers through Token Correlation Zhuo-Yang Song Zeyu Li Qing-Hong Cao Ming-xing Luo Hua Xing Zhu 40 0 0 28 Mar 2025
Inspecting the Representation Manifold of Differentially-Private Text Stefan Arnold 42 0 0 19 Mar 2025
Text-Speech Language Models with Improved Cross-Modal Transfer by Aligning Abstraction Levels Santiago Cuervo Adel Moumen Yanis Labrak Sameer Khurana Antoine Laurent Mickael Rouvier R. Marxer 77 1 0 08 Mar 2025
EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling Theodoros Kouzelis Ioannis Kakogeorgiou Spyros Gidaris N. Komodakis DRL 80 5 0 17 Feb 2025
Lines of Thought in Large Language Models Raphael Sarfati Toni J. B. Liu Nicolas Boullé Christopher Earls LRM VLM LM&Ro 66 1 0 17 Feb 2025
The Geometry of Tokens in Internal Representations of Large Language Models Karthik Viswanathan Yuri Gardinazzi Giada Panerai Alberto Cazzaniga Matteo Biagetti AIFin 94 4 0 17 Jan 2025
Lightweight Safety Classification Using Pruned Language Models Mason Sawtell Tula Masterman Sandi Besen Jim Brown 94 2 0 18 Dec 2024
Understanding Variational Autoencoders with Intrinsic Dimension and Information Imbalance Charles Camboulin Diego Doimo Aldo Glielmo DRL 72 0 0 04 Nov 2024
Unsupervised detection of semantic correlations in big data Santiago Acevedo Alex Rodriguez A. Laio 70 2 0 04 Nov 2024
ResiDual Transformer Alignment with Spectral Decomposition Lorenzo Basile Valentino Maiorca Luca Bortolussi Emanuele Rodolà Francesco Locatello 48 1 0 31 Oct 2024
Analyzing Deep Transformer Models for Time Series Forecasting via Manifold Learning Ilya Kaufman Omri Azencot AI4TS 31 2 0 17 Oct 2024
Persistent Topological Features in Large Language Models Yuri Gardinazzi Giada Panerai Karthik Viswanathan A. Ansuini Alberto Cazzaniga Matteo Biagetti 45 2 0 14 Oct 2024
Detecting and Approximating Redundant Computational Blocks in Neural Networks Irene Cannistraci Emanuele Rodolà Bastian Rieck 36 0 0 07 Oct 2024
Geometric Signatures of Compositionality Across a Language Model's Lifetime Jin Hwa Lee Thomas Jiralerspong Lei Yu Yoshua Bengio Emily Cheng CoGe 84 0 0 02 Oct 2024
Evidence from fMRI Supports a Two-Phase Abstraction Process in Language Models Emily Cheng Richard Antonello 77 4 0 09 Sep 2024
Residual Stream Analysis with Multi-Layer SAEs Tim Lawson Lucy Farnik Conor Houghton Laurence Aitchison 31 3 0 06 Sep 2024
The representation landscape of few-shot learning and fine-tuning in large language models Diego Doimo Alessandro Serra A. Ansuini Alberto Cazzaniga 96 4 0 05 Sep 2024
Pre-processing and Compression: Understanding Hidden Representation Refinement Across Imaging Domains via Intrinsic Dimension N. Konz Maciej Mazurowski MedIm 22 0 0 15 Aug 2024
Local Topology Measures of Contextual Language Model Latent Spaces With Applications to Dialogue Term Extraction Benjamin Matthias Ruppik Michael Heck Carel van Niekerk Renato Vukovic Hsien-chin Lin Shutong Feng Marcus Zibrowius Milica Gašić 45 2 0 07 Aug 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 67 6 0 10 Jul 2024
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning Arijit Sehanobish Avinava Dubey Krzysztof Choromanski Somnath Basu Roy Chowdhury Deepali Jain Vikas Sindhwani Snigdha Chaturvedi ALM 43 1 0 25 Jun 2024
Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations Lorenzo Basile Santiago Acevedo Luca Bortolussi Fabio Anselmi Alex Rodriguez 44 4 0 22 Jun 2024
MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression Tianyu Fu Haofeng Huang Xuefei Ning Genghan Zhang Boju Chen ... Shiyao Li Shengen Yan Guohao Dai Huazhong Yang Yu Wang MQ 52 17 0 21 Jun 2024
Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance Anna C. Marbut John W. Chandler Travis J. Wheeler 37 0 0 18 Jun 2024
A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models Hamidreza Kamkari Brendan Leigh Ross Rasa Hosseinzadeh Jesse C. Cresswell G. Loaiza-Ganem DiffM 42 11 0 05 Jun 2024
The Geometry of Categorical and Hierarchical Concepts in Large Language Models Kiho Park Yo Joong Choe Yibo Jiang Victor Veitch 50 27 0 03 Jun 2024
Beyond the noise: intrinsic dimension estimation with optimal neighbourhood identification A. Di Noia Iuri Macocco Aldo Glielmo A. Laio Antonietta Mira 48 3 0 24 May 2024
Emergence of a High-Dimensional Abstraction Phase in Language Transformers Emily Cheng Diego Doimo Corentin Kervadec Iuri Macocco Jade Yu A. Laio Marco Baroni 112 11 0 24 May 2024
Analyzing Narrative Processing in Large Language Models (LLMs): Using GPT4 to test BERT Patrick Krauss Jannik Hösch C. Metzner Andreas K. Maier Peter Uhrig Achim Schilling 39 1 0 03 May 2024
Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance Chiraag Kaushik Ran Liu Chi-Heng Lin Amrit Khera Matthew Y Jin Wenrui Ma Vidya Muthukumar Eva L. Dyer 46 3 0 18 Feb 2024
A Manifold Representation of the Key in Vision Transformers Li Meng Morten Goodwin Anis Yazidi P. Engelstad 29 0 0 01 Feb 2024
Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural language Eghbal A. Hosseini Evelina Fedorenko LLMSV 28 4 0 05 Nov 2023
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts Eduard Tulchinskii Kristian Kuznetsov Laida Kushnareva D. Cherniavskii S. Barannikov Irina Piontkovskaya Sergey I. Nikolenko Evgeny Burnaev DeLMO 28 82 0 07 Jun 2023
On convex decision regions in deep network representations Lenka Tvetková Thea Brusch Teresa Scheidt Fabian Martin Mager R. Aagaard Jonathan Foldager T. S. Alstrøm Lars Kai Hansen 39 2 0 26 May 2023