Multi-Level Narrative Evaluation Outperforms Lexical Features for Mental Health

Problem
This preprint addresses the gap in computational approaches to therapeutic writing, specifically the inadequacy of existing methods that rely heavily on lexical features and embeddings for mental health assessment. Current techniques fail to account for the hierarchical structure of narratives, which is crucial for understanding how individuals articulate their psychological states. The authors propose a comprehensive framework that integrates various levels of narrative analysis, aiming to enhance predictive accuracy in mental health contexts.

Method
The authors introduce a three-level framework for narrative evaluation:

1. Micro-level: Utilizes lexical features, focusing on word counts and basic linguistic metrics.
2. Meso-level: Employs semantic embeddings to capture deeper contextual meanings within the text.
3. Macro-level: Implements large language model (LLM) evaluations to assess narrative coherence and structure based on established discourse theories, such as Labov’s story grammar and Rhetorical Structure Theory (RST).

The study analyzes 830 Chinese therapeutic texts related to depression, anxiety, and trauma. The training process involved multi-level classification, where the macro-level evaluation was shown to significantly outperform the other two levels. The authors do not disclose specific training compute or hyperparameters, focusing instead on the qualitative aspects of their framework.

Results
The macro-level narrative evaluation achieved a substantial improvement in predictive performance compared to lexical and embedding features. While specific metrics are not detailed in the abstract, the authors emphasize that formal structural features provide a stronger predictive signal for mental health outcomes. The results indicate that the macro-level approach captures essential narrative organization, which is often overlooked in traditional methods. Incremental gains from semantic embeddings suggest they contribute minimally to the overall predictive capability.

Limitations
The authors acknowledge that their framework may not fully encapsulate all dimensions of narrative complexity and that the reliance on a single language (Chinese) may limit generalizability to other languages and cultural contexts. Additionally, the study does not explore the potential impact of different narrative styles or the role of individual differences in narrative construction. The absence of a comparative analysis with other advanced models or techniques in the field is also a notable limitation.

Why it matters
This work has significant implications for both clinical practice and future research in mental health. By establishing a framework that prioritizes narrative structure over mere lexical analysis, it challenges prevailing methodologies and encourages a shift towards more nuanced approaches in therapeutic writing analysis. The findings suggest that understanding the organization of narratives can lead to better predictive models for mental health outcomes, informing intervention design and longitudinal studies. This research opens avenues for further exploration of discourse processing theories in computational mental health applications.

Authors: Yuxi Ma, Jieming Cui, Muyang Li, Ye Zhao, Yu Li, Yixuan Wang, Chi Zhang, Yinyin Zang et al.
Source: arXiv:2604.27846
URL: https://arxiv.org/abs/2604.27846v1