Mitigating Scoring Errors and Compensating for Nonverbal Subtests in Speech-Based Dementia Assessment

Problem
The paper addresses the limitations in existing dementia assessment methodologies, particularly the reliance on neuropsychological tests that can be subjective and prone to scoring errors. It highlights the challenges posed by transcription inaccuracies and the exclusion of nonverbal subtests, such as motor skills, in speech-based evaluations. The authors note that while speech-based assessments can improve accessibility and diagnostic accuracy, current methods do not effectively mitigate these issues. This work is a preprint and has not undergone peer review.

Method
The authors propose a model that integrates transcript-derived scores with Whisper embeddings for each verbal subtest of the German “Syndrom-Kurz-Test,” a standardized dementia screening tool. The architecture employs a fusion strategy to combine these features, aiming to reduce scoring errors associated with traditional assessments. To address the absence of motor subtests, the model approximates expert overall ratings using the fused representations derived from the speech data. The training process involves a dataset of speech samples from dementia patients, although specific details regarding the training compute resources and dataset size are not disclosed.

Results
The proposed model demonstrates a strong correlation with expert ratings, achieving a Pearson correlation coefficient of 0.85 when predicting overall cognitive status despite the omission of motor subtests. In comparison to baseline models that do not utilize speech-derived features, the proposed approach shows a significant improvement in classification accuracy, with an F1 score increase of 12% on the dementia classification task. The results indicate that the model can effectively discriminate between cognitive status groups, outperforming conventional scoring methods.

Limitations
The authors acknowledge several limitations, including the potential for bias in the training data and the need for a larger, more diverse dataset to generalize findings across different populations. They also note that the model’s performance may vary based on the quality of the speech input and the specific characteristics of the dementia subtypes being assessed. Additionally, the study does not explore the interpretability of the model, which could be crucial for clinical adoption.

Why it matters
This research has significant implications for the field of cognitive assessment, particularly in enhancing the accuracy and accessibility of dementia diagnostics through speech analysis. By addressing the shortcomings of traditional neuropsychological tests, the proposed method could facilitate earlier detection of cognitive impairments, ultimately leading to better patient outcomes. The integration of speech-derived features into dementia assessment aligns with ongoing efforts to leverage AI in healthcare, as discussed in related works on speech processing and cognitive evaluation, such as those available on arXiv.