C
ChaoBro

Wrote a Book on How AI Harms Truth, Only to Find AI-Fabricated Quotes Inside—Author Says "I Can't Stop"

Wrote a Book on How AI Harms Truth, Only to Find AI-Fabricated Quotes Inside—Author Says "I Can't Stop"

This story carries a sense of irony reminiscent of a Greek tragedy.

Steven Rosenbaum wrote a new book titled The Future of Truth. Its central theme is that AI is "bending, blurring, and synthesizing truth."

Then, The New York Times discovered that six quotes in the book were "misattributed or synthetic"—in other words, fabricated by AI.

Regarding one of the quotes, tech journalist Kara Swisher told The New York Times: "I never said that."

For another quote, Northeastern University professor Lisa Feldman Barrett stated: "That's not in my book, and it's also wrong."

That's the first layer of irony.

The second layer of irony is that after acknowledging the problem, Rosenbaum said he will not stop using AI.

What Happened

During the writing process, Rosenbaum used ChatGPT and Claude to "discover ideas, locate articles, summarize themes, and identify people or papers that might warrant deeper exploration." He tagged this information in his notes with a "this came from AI" label.

These notes were then handed over to the publisher's fact-checker and two copy editors.

Out of 285 external citations, six went wrong. Three of them were "synthetic quotes"—completely fabricated with no source whatsoever.

Rosenbaum said: "I think we did it very effectively, but not 100 percent. We're doing the work, we're doing our best. We looked at it, it looked right. We double-checked, and then we made a mistake."

Yet, most authors manage to have zero fabricated quotes when writing a book.

Why This Is More Serious Than It Appears

The issue isn't just that one author made a mistake. It's that the traditional fact-checking process has systemically broken down in the face of AI-assisted writing.

In the past, fact-checkers could reasonably assume that the sources an author cited were transcribed directly from the original texts. These citations certainly needed verification, but they were inherently less suspicious—because they were verifiable.

But when AI enters the workflow, that assumption collapses. AI can generate quotes that look entirely plausible—correct formatting, correct names, even correct book titles—but the content is fabricated.

Fact-checkers see a perfectly formatted, seemingly credible quote. They have no reason to doubt it—unless they specifically verify each one. And when a book contains 285 citations, the cost of verifying them one by one is enormous.

It's like the money-printing technology has been upgraded, but the counterfeit detectors are still the old models.

The Most Unsettling Part: The Author Can't Stop

A statement Rosenbaum made to Ars Technica concerns me more than any synthetic quote:

"Asking me to not use AI for a few years, wait for it to sort itself out, and then go back to using... Microsoft Word... that's not in my nature. AI is magical. Because it connects, it weaves ideas together, and gives you paths of thought you wouldn't come up with on your own."

He compared AI to the One Ring from The Lord of the Rings—"making many users believe they can properly control its power. But can they?"

The metaphor is apt. But what he fails to realize is: he himself is the proof of that metaphor.

He wrote a book about the dangers of AI. AI fabricated quotes inside his book. He acknowledged it, corrected it, but says he can't stop.

That is the curse of the Ring. It's not "I don't know it's harmful"—it's "I know it's harmful, but I'm going to use it anyway."

The Structural Dilemma of the Publishing Industry

There's a broader backdrop to this story: newsrooms and publishers are "laying off copy editors and fact-checkers."

Under financial pressure, the publishing industry has fewer and fewer people doing less and less. Then AI arrived, promising to boost efficiency—replacing the laid-off staff with a single tool.

But AI is not a "better person." It's an entirely different category of thing. It makes mistakes humans wouldn't make—and these mistakes don't look like mistakes at all.

When publishers lay off fact-checkers, then use AI to assist with writing, and finally stamp and publish content that merely "looks right"—every step in this process introduces risk.

My Take

Rosenbaum's attitude is more honest than I expected. He acknowledged the problem, made no excuses, and is conducting a "full citation audit." That's a responsible approach.

But his statement that he won't stop using AI worries me.

Not because "writing with AI" is inherently problematic. But because most authors using AI assistance likely won't be as honest as Rosenbaum, nor will they have The New York Times investigating them.

Imagine this: a lesser-known author uses AI to help write a book. AI fabricates a few quotes. The fact-checker (if there is one) misses them. The book gets published. No one investigates.

No one ever finds out.

That is the core of the issue. It's not that "AI makes mistakes"—it's that "when AI makes mistakes, we may never know."

Rosenbaum's book is about how AI synthesizes truth. The book itself has become an example of how AI synthesized truth within a book about AI synthesizing truth.

Ironic? Yes. But more importantly: how many more examples like this will follow this book?

Primary Sources: