Writers Face New Hurdle From AI Detectors

The publishing industry has long relied on human judgment to evaluate manuscript quality, but that process is shifting. Editors are increasingly turning to software to sort through a growing volume of submissions, a change that has created tension between writers and the gatekeepers of their work.

Sally McGraw, a nonfiction writer with more than 20 years of experience, encountered this shift while pitching a book proposal to a client. She submitted a writing sample and received an email stating that multiple artificial intelligence detection tools had flagged her text as machine-generated. McGraw, who wrote the sample herself, said she was surprised by the result. She assured the client of her authorship but did not push the issue, viewing the project as a minor engagement. The client later confirmed to NPR that she had screened the work but cited experience and fit as the primary reasons for hiring a different writer. McGraw was paid for the sample but believes the detector results cost her the full proposal.

The increase in submissions is putting pressure on editorial staff. Neil Clarke, publisher of Clarkesworld, reported that the magazine received roughly 1,100 submissions per month in 2021. By February 2023, following the widespread availability of AI agents, that number had doubled. Clarke temporarily closed the submission portal and estimates his own workload has increased by about 30 percent. He described the situation as introducing an element of mistrust into the editing process.

Researchers have found that detection tools often disagree on the same piece of text. A study published in June in the International Journal for Educational Integrity tested four widely used detectors against 160 academic papers. The papers were divided into four categories: entirely human-written, entirely AI-written, a mix of both, and AI text rewritten by a human. Pangram, a detection tool, was the most consistently accurate, correctly identifying AI use in 37 of 40 papers in two specific categories. However, Pangram CEO Max Spero noted that the tool can make mistakes, particularly when evaluating short texts.

Chris Callison-Burch, a computer and information science professor at the University of Pennsylvania, explained that many detectors work by asking a large language model how predictable a passage is. He noted that these models often find text from their own training data unusually predictable. This issue affected science blogger Janelle Shane in late 2022 when a detector reported 98.7 percent confidence that a passage from her 2019 book was machine-generated. Shane called the result a clear false positive.

Callison-Burch said detectors look for patterns in word choice, punctuation, and sentence structure. Common human writing habits, such as using the word "delve" or grouping ideas in lists of three, can trigger an AI-positive result. He emphasized that detectors weigh patterns across an entire piece rather than relying on single elements. The Authors Guild stated in May that the use of these tools creates an environment reminiscent of witch hunts. The organization warned that a false positive can cost an author a contract and reputation, while a false negative can erode reader trust.

McGraw has changed her writing style to avoid triggering detectors. She now avoids certain words and sentence constructions and occasionally omits commas she previously used. She misses her em dash, a tool she learned in journalism school and used for decades. McGraw sees the irony in large language models learning from human prose while writers feel pressure to make their own text look less like AI. She is bracing for a future assignment where the outcome is critical and the stakes are higher.