Type the words “college student” and “ChatGPT” into Google and you’ll probably find multiple horror stories of robot-written book reports and the inevitable downfall of academia.
However, one college student has built a tool specifically designed to sniff out text written by AI, giving teachers a small sense of relief.
Edward Tian, a 22-year-old senior at Princeton University, spent his winter break diligently working on GPTZero—an app that (perhaps ironically) incorporates the same technologies used in ChatGPT to “quickly and efficiently” detect AI in any block of text. The fact that Tian is studying computer science and journalism seems oh-so fitting.
The process behind GPTZero is fairly simple. Users can copy and paste a piece of text into the app and it will scan the text to provide a score based on two basic metrics—perplexity and burstiness.
Perplexity is the spontaneity or complexity of a single sentence. AI tends to churn out straightforward, though generic sentences, whereas humans don’t. Even the most succinct among us can meander a bit, lose the plot, ramble … you get it.
Similarly, AI wouldn’t really have a series of sentences that vary widely in length, with long run-on sentences followed by a very short one. Like this one. We call this overall randomness burstiness. And so far, burstiness isn’t really AI’s chosen writing style.
The lower the scores on perplexity and burstiness, the more likely the text was written by a bot, which Tian demonstrated in a video using a report from The New Yorker (human, John McPhee to be precise) and a LinkedIn post (ChatGPT).
here’s a quick demo with john mcphee’s “frame of reference” pic.twitter.com/WphxfxxFdr
— Edward Tian (@edward_the6) January 3, 2023
“There are beautiful qualities of human written prose that computers can and should never co-opt,” Tian explained in an interview with Daily Beast.
Tian released GPTZero on Jan. 2, expecting only “a few dozen people” to try it. He woke up to discover that it became an instant hit, primarily among teachers, with positive reviews. According to NPR, more than 30,000 people had tried out GPTZero within a week, causing the app to temporarily crash.
I spent New Years building GPTZero — an app that can quickly and efficiently detect whether an essay is ChatGPT or human written
— Edward Tian (@edward_the6) January 3, 2023
Since its launch in late November 2022, the AI chatbot tool ChatGPT has been used to write short poems, formal emails, film scripts and even high school and college papers—the latter of which has caused concern among educators who fear the program will make cheating and plagiarism easier than ever.
Other measures have been initiated to regulate AI usage. Chalkbeat reports that the New York Education Department blocked access to ChatGPT. And OpenAI, an artificial intelligence research lab, is working to implement watermarks that would verify whether or not something was generated by ChatGPT.
However, the virtual, uncharted terrain of artificial intelligence is essentially still a lawless no man’s land. So while Tian doesn’t find artificial intelligence as inherently a bad thing, he hopes that his app, while still not foolproof, might add some much-needed clarity and transparency.
“For so long, AI has been a black box where we really don’t know what’s going on inside,” he said, according to NPR. “And with GPTZero, I wanted to start pushing back and fighting against that. [GPTZero] is not meant to be a tool to stop these technologies from being used. But with any new technologies, we need to be able to adopt it responsibly and we need to have safeguards.”