Abstract
Recent findings suggest that detection models for artificial intelligence (AI) cannot accurately identify AI-generated text and may exhibit bias against certain minority groups. This study empirically examines anecdotal claims that autistic writers more often have their work flagged as AI-generated. A corpus of approximately 60,000 Reddit posts split into 'likely-autistic' and 'general-Reddit' subcorpora is used to compare the distribution of probabilities output by the OpenAI GPT-2 detection model.
Methodology
Differences in textual features between subcorpora are observed and compared to reported features of AI-generated text. Results showed that while less than two percent of either subcorpus was flagged as AI-generated by the model, significantly more texts from the likely-autistic subcorpus were flagged.
The connections between features of text with likely-autistic authors and AI-generated text were not straightforward.
Ethical Considerations
The widespread use of AI-detection models with a potential bias against autistic writers prompts ethical scrutiny, and the authors recommend further critical examination of the models themselves as well as their use in academic contexts.
Blogger's Review: This study highlights the inherent biases that AI detection models may have when handling texts from minority groups, particularly autistic writers. This not only affects the freedom of expression for these writers but also raises significant ethical discussions about the technology, calling for deeper scrutiny and improvement of these models.