‘Unacceptable’: condemning writers over AI detector scores

The Haitian-Canadian writer Thélyson Orélien has been accused of using artificial intelligence to write his novel "C’était ça ou mourir". Afp Or Licensors

When a detector flagged a bestselling novel as AI-generated, it sparked a controversy that had serious consequences for the author. Swiss experts and artists examine how much these tools can really prove and what their verdicts mean for creative work.

Haitian-Canadian writer Thélyson Orélien had a promising career ahead of him. His novel C’était ça ou mourir (It was either that or die), a critical and commercial success, was in the running for some of France’s most prestigious literary prizes, including the Goncourt. That was until an anonymous account on X accused him of writing almost the entire book using artificial intelligence (AI).

Orélien denied using AI to write the novel. But to his accusers, the evidence seemed unequivocal: Pangram, one of the most popular tools for detecting AI content, had classified more than 95% of the book as AI-generated.

From there, things took a turn for the worse for Orélien. His novel was removed from the Goncourt selection, also amid revelations by the press of plagiarism in earlier works, for which he apologised. His Canadian publisher cut ties with him after an external analysis uncovered further evidence of AI use. Orélien continued to deny having written the book with AI, while acknowledging that he had turned to it for historical research during the creative process.

From the beginning, the controversy divided the literary world between those who defended the author and questioned the reliability of AI detectors, and those who criticised him.

Manuel Flurin Hendry is a Swiss director, screenwriter and lecturer. Copyright 2017 Fabian Frost

Beyond Orélien’s case, Swiss filmmaker, screenwriter and lecturer Manuel Flurin Hendry believes it’s unfair to damage a writer’s reputation based on a software-generated score. “The idea that you feed a text into Pangram and then socially execute a writer feels unacceptable,” Hendry told Swissinfo in an interview at the AI+X Summit in early October. Hendry defends artists’ freedom to choose their own tools.

Other Swiss writers and artists have openly experimented with AI. Author Charles Lewinsky incorporated entire passages generated by ChatGPT into his novel Täuschend echt. Photographer Matthieu Gafsou has used image-generation tools such as DALL·E in his artistic work.

These are just a few transparent examples of how AI has entered creative work in Switzerland. But deciding how to respond to undisclosed and potentially extensive use of AI is far more difficult. As is determining how much AI-generated material is acceptable in a work of art, and whether tools like Pangram can really be trusted as a definitive proof of fraud. 

How reliable is Pangram?

Hendrik Strobelt is a senior researcher at IBM in the United States. Courtesy of Hendrik Strobelt

For Hendrik Strobelt, a senior researcher at IBM in the United States, the question of Pangram’s reliability is complex. In 2019, Strobelt and his colleagues developed GLTR, one of the first experimental tools designed to help identify text potentially generated by language models.

At the time, OpenAI had just released GPT-2, the model on which GLTR was based. AI models tended to produce rather dull texts that were very similar in style, because this type of content was the most common, Strobelt explains.

Identifying whether a text might have been produced with the help of AI was therefore relatively straightforward: it was enough to look for the most likely words.

Then language models became more sophisticated. Strobelt believes that, with today’s advanced AI systems, it has become extremely difficult to establish clear rules to distinguish AI-generated text from human writing.

“Now these models are significantly larger. They can imitate styles if you prompt them accordingly,” the IBM researcher says.

The tool developed by Strobelt highlighted the statistically most predictable words – potentially generated by AI – in green, the less common ones in yellow, and the most surprising ones – likely of human origin – in red or purple. Courtesy of Hendrik Strobelt

Software such as Pangram uses a method known as “supervised classification”. It compares and classifies texts written before generative AI became widespread – which is presumed to be human-written – with “mirror” versions on the same topic, generated by different commercially available language models. During training the software learns to identify the differences between them.

The logic is that if a system can distinguish between texts that are very similar in subject matter, its predictions will become more accurate.

“You want to train on things that are very close to each other, to be sure that you get the hard tasks trained first,” Strobelt explains.

Pangram claimsExternal link an accuracy rate of 99.98% and a false-positive rate of 0.01%. Its high performance has also been supported by independent researchExternal link, including a studyExternal link published this year.

According to Paolo Italiani, an AI and machine-learning researcher at the Università della Svizzera italiana (USI), the longer a text is, the more reliable a detector’s assessment tends to be.

“It’s quite probable that at least a portion of [the text] is AI-generated,” he says.

The limits of AI detectors

Yet it remains difficult to understand exactly which features of a text these tools rely on to make their classifications, says Lonneke van der Plas, professor of natural language processing at USI.

Interpretability studies would be needed to gain more understanding, she explains. But because commercial tools such as Pangram are largely black boxes whose internal workings are not accessible to independent researchers, it is challenging to study how they reach their conclusions.

AI detection tools lose accuracy when dealing with non-English texts and with content written in collaboration with AI, according to Lonneke van der Plas and Paolo Italiani of the Università della Svizzera italiana. Universita’ Della Svizzera Italiana

In theory, certain clues such as an overly linear argumentative structure could help determine whether a text was more likely written by AI or by a human, researchExternal link co-authored by Van der Plas highlights. But such features, she cautions, are not sufficient to establish a text’s origin with certainty.

Van der Plas also points out that detection tools tend to perform less reliably in languages other than English and in more specialised genres that may have been less well represented in their training data.

The problem becomes even more complicated when a text is the product of collaboration between humans and AI. Some tests showExternal link, detectors’ accuracy drops by around 20 percentage points in such cases. Pangram’s own technical reportExternal link also indicates that the software loses accuracy when faced with content that alternates between human-written and AI-generated sentences.

“This software has been mostly trained on either completely human-generated or machine-generated texts. As soon as they get something that it’s in between, they get into trouble,” van der Plas says.

A score is not sufficient proof

From a legal perspective, results produced by tools such as Pangram do not constitute proof that an author has committed fraud.

Jacques de Werra is professor of intellectual property law at the University of Geneva. Courtesy of Jacques de Werra

“A legal decision cannot simply rely on those technologies because they can be inaccurate” says Jacques de Werra, professor of intellectual property law at the University of Geneva.

If a case like Orélien’s were brought before a court, de Werra explains, other evidence alongside detector scores should be considered, including original manuscripts, successive drafts and even exchanges with AI systems. De Werra also points out that AI involvement in creative work does not automatically invalidate an author’s copyright claims.

“If a person can demonstrate that there is a sufficiently close creative link between the input, creative prompting, and the output, it could be possible to claim human creativity,” he says.

The key is whether the author’s creative contribution is sufficient. De Werra stresses, however, that there is no fixed numerical threshold that determines how much of a text must be written by a person for a work to be considered their own.

If a book moves you, does it matter if AI helped write it?

For Manuel Flurin Hendry, however, the issue goes beyond legal protection.

Hendry openly uses AI-based technologies, some of which he has developed himself, for various aspects of his work, from analysing screenplays to assisting his academic research.

Technology, he explains, opens new channels of inspiration, sometimes suggesting approaches he had not previously considered. It also frees up more time for the creative process.

“True human writing is thinking in action. It requires friction and to enter a state where one can just concentrate on their flow of thoughts,” he says.

And he poses a provocative question: why should it matter how much AI contributed to a work? “If it moves you. If you find it interesting. If it makes you think, what is the argument?” he asks.

What’s important, Hendry argues, is learning which tasks to delegate to AI and which are better done personally. Rather than rejecting technology, artists should be transparent about how they use it.

“If I pretend this is mine, but it’s totally AI generated, then it’s fake,” Hendry says.

What is at stake is not only an artist’s reputation, but above all the relationship of trust with their audience.

Join the debate:

Edited by Gabe Bullard/vdv 

Prefer Swissinfo on Google

See more Swissinfo articles in your Google results. Learn more

Close