Detecting LLM-Generated Texts with “Classical” Machine Learning

The article discusses a recent exploration into the detection of AI-generated text using classical machine learning techniques, specifically focusing on the capabilities of models like Linear SVC and Naive Bayes. The author, motivated by the proliferation of low-quality AI-generated content, developed a classifier that achieves approximately 85% accuracy in distinguishing between human-written and LLM-generated texts. This was accomplished by generating a dataset of nearly 10,000 human-written samples and an equal number of LLM-generated samples, which were then used to train the classifiers.

The research highlights the effectiveness of traditional machine learning approaches over more complex methods, such as using LLMs for perplexity-based detection, which yielded unsatisfactory results due to high false positive and negative rates. The author utilized scikit-learn’s TF-IDF feature extraction combined with Linear SVC for classification, achieving a notable performance that surpassed expectations. The final model employed a majority voting mechanism across multiple binary classifiers to enhance detection reliability, demonstrating that classical methods remain robust in the face of evolving AI text generation technologies. For further details, the original article can be accessed here.