Which Sections of a Research Paper Best Reveal Its Research Methods? Evidence from Library and Information Science

Problem
This work addresses the gap in automatic classification of research methods within academic papers, specifically in the Library and Information Science domain. Existing approaches predominantly utilize titles and abstracts, which often lack comprehensive methodological details. The authors highlight the limitations of these methods, particularly the insufficient information provided by abstracts and the challenges posed by the excessive length and redundancy of full-text documents. This paper is a preprint and has not undergone peer review.

Method
The authors propose a segment combination strategy that partitions full-text articles based on their physical position within the document. They utilize an annotated corpus of 1,954 full-text articles sourced from three prominent journals: the Journal of the Association for Information Science and Technology (JASIST), Library and Information Science Research (LISR), and Journal of Documentation (JDoc). The study evaluates the classification performance of various segments (e.g., introduction, methods, results, discussion) and their combinations using multiple machine learning models. The integration of bibliographic metadata with cross-segment combinations is also explored to enhance classification accuracy.

Results
The experimental results reveal that methodological information is not uniformly distributed throughout the full-text content. The middle-to-late and final segments of the articles demonstrate significantly higher discriminative power for classification tasks. The authors report that their proposed segment combination strategy, when combined with bibliographic metadata, leads to improved classification performance. Specific performance metrics, such as accuracy and F1 scores, are not disclosed in the summary but are critical for evaluating the effectiveness of the proposed method against baseline models.

Limitations
The authors acknowledge that their findings are specific to the Library and Information Science field and may not generalize to other disciplines. They also note the potential for overfitting when using complex models on limited datasets. Additionally, the reliance on physical document structure may not account for variations in writing styles or formats across different journals. The study does not address the computational costs associated with processing full-text documents, which could be a barrier for widespread adoption.

Why it matters
This research has significant implications for enhancing knowledge services in academia, particularly in the areas of method retrieval, review generation, and research intelligence analysis. By improving the automatic classification of research methods, the proposed approach can facilitate better access to methodological information, thereby supporting researchers in identifying relevant techniques for their work. This advancement is crucial for the ongoing development of intelligent systems that assist in literature review and research synthesis, as discussed in related works on automated content analysis and information retrieval, as published in arXiv.