Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds

{ “meta”: “This paper analyzes the topological structure of LLMs, revealing their latent spaces organize into Small-World networks and enabling new insights into reasoning.”, “body”: “Problem — This work addresses the gap in understanding the internal mechanisms of Large Language Models (LLMs) that facilitate multi-hop reasoning over long contexts. Traditional interpretability methods, particularly those based on attention weights, fail to accurately represent semantic proximity due to routing artifacts. The paper is a preprint and has not undergone peer review.\n\nMethod — The authors propose a novel approach that bypasses attention weights to analyze the dynamic geometry of the hidden state manifold of LLMs. They sparsify continuous similarity matrices of long-context representations into unweighted graphs, allowing for the examination of connectivity between disjoint semantic anchors across two distinct architectures. The study identifies a topological phase transition where early syntactic layers are fractured, while deeper reasoning layers compress semantic distances into pathways limited by the “