SLIM: Sparse Latent Steering for Interpretable and Property-Directed LLM-Based Molecular Editing

Problem
This paper addresses the challenge of controlling molecular property outcomes in large language model (LLM)-based molecular editing. While LLMs exhibit strong chemical reasoning, the entanglement of property-relevant information within dense hidden states complicates explicit property control, leading to a significant proportion of edits that do not enhance or even degrade desired molecular properties. The authors present SLIM (Sparse Latent Interpretable Molecular editing) as a solution to this problem. This work is a preprint and has not yet undergone peer review.

Method
SLIM introduces a framework that utilizes a Sparse Autoencoder to decompose the hidden states of LLMs into sparse, property-aligned features. The architecture incorporates learnable importance gates that allow for selective activation of dimensions relevant to specific molecular properties. This approach enables property-directed steering in the sparse feature space without necessitating modifications to the underlying model parameters. The training process involves optimizing the Sparse Autoencoder to ensure that the extracted features are interpretable and aligned with the desired molecular properties. The framework is designed to be plug-and-play, allowing it to be integrated with various LLM architectures.

Results
The authors evaluated SLIM on the MolEditRL benchmark, testing across four different model architectures and eight distinct molecular properties. The results demonstrate substantial improvements in editing success rates compared to baseline methods. Specifically, SLIM achieved enhancements of up to 42.4 points in property alignment, indicating a significant increase in the effectiveness of molecular edits. The paper provides detailed comparisons against named baselines, showcasing SLIM’s superior performance in property-directed molecular editing tasks.

Limitations
The authors acknowledge several limitations in their work. First, the reliance on a Sparse Autoencoder may introduce challenges in capturing complex interactions between properties that are not easily separable in the sparse feature space. Additionally, the framework’s performance may vary depending on the specific LLM architecture employed, which could limit its generalizability. The authors also note that while SLIM improves interpretability, the interpretative power of the sparse features may still be constrained by the underlying model’s complexity. Furthermore, the study does not explore the scalability of SLIM to larger datasets or more complex molecular properties, which could be a potential area for future research.

Why it matters
The development of SLIM has significant implications for the field of molecular editing and cheminformatics. By providing a method for interpretable and property-directed editing, SLIM enhances the ability of researchers to design molecules with specific characteristics, thereby accelerating the discovery of new compounds with desired properties. This work also contributes to the broader understanding of how to manipulate LLMs for targeted applications, paving the way for future advancements in AI-driven molecular design and optimization. The framework’s plug-and-play nature suggests potential for integration into existing workflows, making it a valuable tool for chemists and researchers in related fields.

Authors: Mingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun
Source: arXiv:2605.10831
URL: https://arxiv.org/abs/2605.10831v1