Self-organized MT Direction Maps Emerge from Spatiotemporal Contrastive Optimization

Problem
This preprint addresses the gap in understanding the computational mechanisms underlying the spatial organization of the dorsal stream in the primate visual cortex, specifically the emergence of direction-selective maps in the middle temporal (MT) area. While previous models like the Topographic Deep Artificial Neural Network (TDANN) have elucidated aspects of the ventral stream’s organization, the principles governing the MT’s distinct topographies remain largely unexplored.

Method
The authors propose a spatiotemporal TDANN architecture, which incorporates a 3D ResNet trained on naturalistic video data. The training employs a Momentum Contrast (MoCo) self-supervised learning paradigm, augmented with a biologically inspired spatial loss function. This approach facilitates the model to learn representations that mimic the spatial and functional organization of the MT area. The optimization process balances task-driven discriminative pressure against spatial regularization, leading to the emergence of direction-selective maps and topological pinwheel structures akin to those observed in biological systems.

Results
The model’s performance was quantitatively assessed against physiological baselines from in vivo macaque MT data. Key metrics include a direction selectivity index that aligns with biological observations, a circular variance that reflects the model’s ability to capture the distribution of direction selectivity, and pinwheel density that corresponds to the spatial arrangement of direction-selective neurons. The results indicate that the model successfully reproduces these physiological characteristics, demonstrating a strong correlation with empirical data. The authors report effect sizes that substantiate the model’s efficacy in capturing the essential features of MT topography.

Limitations
The authors acknowledge that their model’s reliance on a specific training paradigm may limit its generalizability to other cortical areas or species. Additionally, the study does not explore the full range of potential biological variability in MT organization, nor does it address the implications of these findings for understanding other aspects of visual processing. The model’s performance in real-time applications or its robustness to adversarial conditions is also not evaluated.

Why it matters
This work has significant implications for both computational neuroscience and machine learning. By establishing a unified framework for understanding the self-organization of cortical areas, it opens avenues for further research into the principles governing neural representation in the brain. The findings may inform the design of more biologically plausible neural networks, potentially enhancing their performance in tasks requiring spatial awareness and temporal processing. Furthermore, the insights gained from this study could lead to advancements in artificial intelligence systems that mimic human visual processing, thereby bridging the gap between biological and artificial vision systems.

Authors: Zhaotian Gu, Molan Li, Jie Su, Chang Liu, Tianyi Qian, Dahui Wang
Source: arXiv:2605.11718
URL: https://arxiv.org/abs/2605.11718v1