Unifying Dynamical Systems and Graph Theory to Mechanistically Understand Computation in Neural Networks

Problem
This preprint addresses the gap in understanding how biological and artificial neural networks implement computation through their connectivity. Traditional approaches often focus on direct connections, neglecting the divergence between structural and functional connectivity in neural systems. The authors propose a framework that integrates dynamical systems and graph theory to analyze recurrent neural networks (RNNs) trained on hierarchically modular tasks, emphasizing the importance of multi-hop pathways in information routing.

Method
The core technical contribution is the introduction of resolvent-RNNs (R-RNNs), which are designed to constrain multi-hop communication pathways within RNNs. The authors model the RNN as a graph, allowing for the analysis of multi-hop pathways between input and output units. They decompose these pathways by hop length to understand how information is temporally routed. The R-RNNs employ a novel regularization approach that promotes temporal sparsity across multi-hop pathways, contrasting with standard L1 regularization that focuses on individual weights. The training compute specifics are not disclosed, but the methodology emphasizes the functional alignment of sparsity with task structure.

Results
R-RNNs demonstrate superior performance compared to standard L1-regularized RNNs on hierarchically modular tasks. The authors report that R-RNNs achieve improved task performance, particularly in scenarios where the task signal is sparse. They quantify this improvement through metrics that reflect sparsity-function alignment, showing that R-RNNs maintain robustness under strong regularization conditions. While specific numerical results are not provided in the abstract, the qualitative improvements suggest a significant effect size in terms of task performance and robustness.

Limitations
The authors acknowledge that their approach may not generalize to all types of neural network architectures or tasks beyond those tested. They do not address potential computational overhead introduced by the additional complexity of R-RNNs compared to standard RNNs. Furthermore, the reliance on graph-theoretic interpretations may limit applicability in scenarios where the underlying assumptions about connectivity do not hold.

Why it matters
This work has significant implications for both neuroscience and machine learning, as it reframes the understanding of how neural networks can be structured to enhance computational efficiency. By emphasizing multi-hop communication as a critical mechanism linking structure to function, the findings suggest that future research should focus on developing architectures that prioritize functional pathways over individual parameters. This perspective could lead to more robust and interpretable neural network designs, particularly in applications where task signals are sparse or complex.

Authors: Jatin Sharma, Danyal Akarca, Dan F. M Goodman
Source: arXiv:2605.03598
URL: https://arxiv.org/abs/2605.03598v1