There are no reliable ways to distinguish text written by a human from text written by an large language model. OpenAI writes:
Do AI detectors work?
In short, no. While some (including OpenAI) have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content.
Additionally, ChatGPT has no “knowledge” of what content could be AI-generated. It will sometimes make up responses to questions like “did you write this [essay]?” or “could this have been written by AI?” These responses are random and have no basis in fact.
To elaborate on our research into the shortcomings of detectors, one of our key findings was that these tools sometimes suggest that human-written content was generated by AI.
When we at OpenAI tried to train an AI-generated content detector, we found that it labeled human-written text like Shakespeare and the Declaration of Independence as AI-generated.
There were also indications that it could disproportionately impact students who had learned or were learning English as a second language and students whose writing was particularly formulaic or concise.
Even if these tools could accurately identify AI-generated content (which they cannot yet), students can make small edits to evade detection.
There is some good research in watermarking LLM-generated text, but the watermarks are not generally robust.
I don’t think the detectors are going to win this arms race.
More Stories
White House to Tackle AI-Generated Sexual Abuse Images
White House issues new voluntary commitments to combat image-based sexual abuse in AI Read More
Legacy Ivanti Cloud Service Appliance Being Exploited
CISA wants everyone—and government agencies in particular—to remove or upgrade an Ivanti Cloud Service Appliance (CSA) that is no longer...
Half of UK Firms Lack Basic Cybersecurity Skills
A new government report reveals that nearly half of UK businesses lack basic cybersecurity skills, while advanced skills like penetration...
Advanced Phishing Attacks Put X Accounts at Risk
SIM swapping and “adversary-in-the-middle” can bypass security for accounts on X (formerly Twitter) Read More
Apple to Drop Spyware Lawsuit Over Security Concerns
Apple filed a motion to drop its lawsuit against NSO Group, fears key elements of its cyber defensive measures could...
Tackling the Unique Cybersecurity Challenges of Online Learning Platforms
The content of this post is solely the responsibility of the author. LevelBlue does not adopt or endorse any of...