Large-scale semantic mapping of learner agency and autonomy reveals what measurement and generative AI research overlook

Problem
This preprint addresses the significant gap in the understanding and measurement of learner agency and autonomy within educational research. The authors highlight the “jingle-jangle” fallacy, where similar terms may refer to different constructs, and vice versa, which has impeded cumulative knowledge in this domain. By analyzing a vast corpus of literature, the authors aim to clarify the definitions and dimensions of these constructs, which are often inadequately represented in existing measurement scales and generative AI applications in education.

Method
The authors employed a semantic analysis pipeline to extract and analyze 8,954 definitions and 2,700 scale items from over 14,000 publications. This analysis led to the identification of three distinct dimensions of learner agency and autonomy: (1) regulation and control of learning (task), (2) intrinsic motivation and internal decision-making (person), and (3) social-relational action (sociocultural). The methodology emphasizes the need for a comprehensive understanding of these constructs, particularly the sociocultural dimension, which is often overlooked in current educational frameworks and AI research. The authors argue that existing scales fail to capture the full complexity of learner agency and autonomy, particularly in the context of AI-mediated learning environments.

Results
The findings reveal that existing measurement scales predominantly focus on the task and person dimensions, neglecting the sociocultural aspect. This underrepresentation is quantitatively supported by the analysis of the extracted definitions and scale items, which indicates a systematic bias in the literature. The authors provide empirical evidence that current generative AI research in education is primarily oriented towards enhancing learning regulation and control, thereby constraining the potential for fostering a broader range of learner behaviors and agency. The paper does not provide specific numerical results or effect sizes against named baselines, focusing instead on qualitative insights derived from the semantic analysis.

Limitations
The authors acknowledge that their work is limited by the scope of the literature analyzed, which may not encompass all relevant studies on learner agency and autonomy. Additionally, the semantic analysis is inherently dependent on the definitions and contexts provided in the literature, which may vary widely. The paper does not address potential biases in the selection of publications or the implications of the findings for diverse educational contexts.

Why it matters
This research has significant implications for the conceptualization, measurement, and practical application of learner agency and autonomy in educational settings. By elucidating the multidimensional nature of these constructs, the authors advocate for a more nuanced approach to designing AI-mediated learning environments that support diverse learner behaviors. This work encourages future research to expand the focus beyond mere regulation and control, fostering a richer understanding of learner agency that incorporates sociocultural factors. The findings are particularly relevant for researchers and practitioners in educational technology and AI, as they highlight the need for more comprehensive measurement frameworks. This paper is available on arXiv.