Alpha Reader Versus the Robot

I was going to title this “A Comparison of Human and AI Beta Readers,” but there is no comparison. Humans win.

Last week, I ran an experiment with my upcoming novel. I was curious to see how AI beta feedback would stack up against human readers, so I fed my manuscript to the machine.

(If you’re unfamiliar with the term, a beta reader critiques an early version of a book. I dislike the term because it implies readers need upgrades rather than books. My readers are smart, generous, and Alpha all the way.)

After incorporating two rounds of feedback from Alpha readers over the course of six months, I purchased a structured AI critique called “Virtual Beta Reader” through ProWritingAid, a service I have long used for proofreading… though you wouldn’t know it from all the typos I slip past the goalie.

I have never been tempted to let AI influence my writing. It’s fraught with problems, beginning with ethics. I try to behave myself, and AI famously does not. For example, its owners have no qualms about committing mass-piracy to train their large language models. I’m willing to let AI catch typos, but I won’t pilfer other people’s writing to improve my own.

But if ever I was tempted, this experiment convinced me never to let AI touch my writing beyond rudimentary proofreading. I noticed several differences between human and AI readers. One difference outshines them all.

Humans Understand Subtext; AI Does Not

Before we go any further, a word to the wise: If a friend or family member asks you to critique their novel, you should probably decline. Tell them the chickens need tending, or you developed a fear of books during a tragic library accident.

Why? Because there’s a good chance their first attempt at a novel is weak and amateurish — or worse, it’s a polemic. Then you’re in the position of being honest about the book’s shortcomings or dishonest about its strengths.

You might also learn things you don’t want to know about your friend. Do you really want to read three hundred pages of tortured metaphor on the ills of capitalism or the Eastern Orthodox Church? Do you truly want a backstage pass to the dark preoccupations boiling beneath your friend’s calm exterior? Probably not. Run away.

Anyhoo, I asked ten friends and acquaintances to critique my debut novel.

In my defense, I only did so after hiring an editor to help me fix the first draft, and I gave my readers every reason and opportunity to decline. I practically encouraged them not to read it.

I also approached people who I was certain would understand the need for honesty, and who would not feel guilty about providing it. One of my readers is an accomplished novelist… and boy, was he honest. That’s important. I’ll come back to it.

My novel, Shanghai Lottery, is a basic hero’s journey, and it’s heavily character-driven. I’m a psychologist, after all, so I poured a lot of effort into the personality structure of my characters. In real life, people reveal themselves more through actions than words. The same is true of good fiction. That’s where subtext comes in. Subtext is the implied messages and meanings that exist at all levels of a story, from dialog to theme.

One of my favorite aspects of the storytelling in Breaking Bad and Better Call Saul is the unspoken handling of fatal flaws. For Walter White, it was pride. His pride was often on display, but it wasn’t openly discussed. Plenty of characters chewed him out for his behavior, but no one said, “Hey, Walter, you know what your problem is? Pride. Pride will be your downfall.”

Kim Wexler’s fatal flaw was vengeance. She luxuriated in it, until it went too far for her comfort. She almost admitted to her lust for revenge when she broke up with Jimmy.

“You asked if you were bad for me. That’s not it. We are bad for each other… I have had the time of my life with you, but we are bad for everyone around us. Other people suffer because of us. Apart, we’re OK. But together, we’re poison.”

That scene grabs the viewer by the emotions and whips us around the room. I think it’s effective largely because of what she didn’t say. It was something like this:

I am driven by vengeance, but I can’t tolerate it anymore. I can’t even acknowledge it. I would rather slink away and blame “us” than own up to my choices.

That’s amazing use of subtext.

Back to my Alpha readers. They shared helpful observations about plot and structure, but I’ll set those aside and focus on the fact that they were exquisitely attuned to subtext.

The unstated.

That which is not explicitly named.

There were sections of my novel where I provided the right amount of subtext to help my readers understand a character’s motivation. Their favorite scenes involved characters struggling with problems the reader could see, but the characters could not. In response to a scene where my protagonist lamented his history with women, one reader wrote:

“There is so much character development in lines like this. As a reader, at this point I wonder how is this guy so sharp about people one minute and so damn blind about himself in the next? BUT absolutely human.”

There were other sections where I provided too little subtext. For example, my readers were most critical of the love interest because her devotion to my protagonist almost added up, but not quite. I didn’t show my readers enough of that character to help them understand her motivation, and that undermined the story.

Finally, there were places where I deprived the reader of subtext altogether by spelling things out — for example, by having a character explain a motivation that’s better left unsaid. This is insulting to the reader, and my novelist friend yelled at me twice about that mistake.

My Alpha readers were also sensitive to consistency within characters. They told me when words didn’t fit the characters speaking them, and when behaviors seemed a bit off. Some inconsistencies they noticed were pretty danged subtle, and my readers were correct in every case.

I could go on, but you get the point. My Alpha readers knew when subtext was insufficient, and they knew when it was excessive. Detecting subtext was effortless for them, and their feedback was crucial to making my characters feel human.

AI was less insightful.

What AI Said About My Manuscript

I’ll start with a positive. AI gets an A+ for organization. Alpha readers generally presented their thoughts in order of appearance, usually in haphazard emails or free-flowing conversations, but AI produced a sixteen-page report with four major headings:

• General Impressions

• Reading Experience

• Reader Insights

• Marketing Ideas

The first paragraph of the report was this AI-generated story overview:

“A witty but aimless butcher’s assistant moonlights as a wildly popular, anonymous social media troll named Eddie Luzhin. When a viral feud with a self-help guru escalates into a high-stakes, televised boxing duel, he must confront his own deceptions and fight for his honor, his future, and the safety of those he loves. It’s a battle for redemption where the biggest enemy might just be the mask he refuses to take off.”

AI stumbled right out of the gate by misreading my protagonist’s motivation. At no point is he aimless. He may not know what he needs, but he knows exactly what he wants. That was clear to Alpha readers, but it eluded the robot.

The worst thing about that paragraph is how damned lifeless it is, devoid of emotion or imagery. Even I don’t want to read the book now. That paragraph feels like it was written by a low-level bureaucrat at the U.S. Department of Book Blurbs.

That blindness to motivation and emotion plagues the AI beta feedback throughout. The machine can talk about emotions, but only in the same shallow terms in which I can talk about my QuickBooks account. I know it exists, and I’m told it’s important, but I don’t really understand how it works.

Likewise, AI can discuss basic emotions and motivations, but it doesn’t seem to grasp how a character’s stated desires are undermined by ambivalence and conflict that you and I, as human beings, can easily recognize.

I’ll give you an example, but first I need to take a detour and discuss a sub-complaint.

Sub-Complaint: AI Is a Shameless Ass-Kisser

One of the biggest problems with my AI feedback was its effusive praise. Only toward the end of the report did AI offer a few tepid, apologetic suggestions. Mostly, the report was one big pat on the head, with phrases like these:

“Pacing was masterful… Dialog is exceptionally well crafted… The narrator’s voice is absolutely magnetic… The emotional journey of redemption feels earned and deeply satisfying.”

This is bullshit, and it is not helpful. If a person said these things, I would consider them patronizing and disingenuous. I would never trust them to answer another question. My Alpha readers offered plenty of compliments too, but I can trust they were honest because they also offered criticism.

AI’s effusive praise sets an author up for a self-imposed face-plant. It encourages us to be Kim Wexler, pretending our biggest flaws don’t exist… or in this case, the biggest flaws in our writing.

In recent years, the term “gaslighting” has had quite a run in the pop-psychology industry. Instagram therapists would have you believe that your boss, your spouse — basically, anyone who asks you to improve — is a gaslighting narcissist. When they criticize you (says the insta-therapist) they’re only trying to manipulate you into submission. Ignore them. You’re perfect just the way you are.

I can’t speak for you, but I know me. And friend, no one can gaslight me like I can. AI’s over-the-top praise creates no small temptation to convince myself my book is perfect just the way it is.

That is not the path to excellence.

Remember the novelist friend I mentioned earlier? Overall, he really liked Shanghai Lottery. But he straight-up chewed me out over a few mistakes he thought were beneath me. (He was wrong. No mistake is beneath me.)

Thanks to him, I won’t make those mistakes again. Now I can move on to more important mistakes, and that is a true kindness.

AI, on the other hand, is a shameless ass-kisser. It’s a yes-man that will drive your manuscript off a cliff if you let it. Heed its advice at your own peril.

This concludes my sub-complaint. (Go read my friend’s books. They’re big fun.)

Back to My Main Complaint

When AI finally stopped sucking up and offered a few apologetic suggestions, its inability to grasp subtext really stood out. Here is the most prominent example.

In a section titled “Editing Suggestions,” AI recommended I add a closing scene explicitly showing that my protagonist overcame his fatal flaw:

“What would happen if, years later, a situation arose where [protagonist] felt tempted or even compelled to [repeat his mistakes]? What would that internal conflict look like for the man he has become?”

The machine seemed to sense something was missing at the end of the book. It wasn’t exactly wrong. One of my early Alpha readers also observed that my protagonist’s arc was incomplete. He told me this:

“You wrote a hero’s journey, but there’s a piece missing at the end. When Frodo returned to the Shire, he was changed. He was a badass, and it showed.”

I don’t know anything about Frodo. I accept that Lord of the Hobbit is a great book, but I couldn’t even stomach the movie. Hearing this, my friend was kind enough to explain the problem in terms I understand: Skywalker, Rocky, and The Little Engine that Could. Their journeys changed them, and the reader deserves to see it. There’s a way to show that evolution, but not by clubbing the reader over the head with the type of ham-fisted, AI-inspired scene that says,

“Look! My protagonist is stronger and wiser now! He faced a similar dilemma, and he made a different choice! Can you see it, you moron?”

In fact, I fixed the problem before feeding the manuscript to AI. I adjusted the ending in a manner that subsequent Alpha readers understood and appreciated, but it was apparently too subtle for the robot to grasp. As a result, the machine would have me insult my readers by giving my protagonist a final exam rather than showing the calm dignity and the ongoing effort of redemption.

If I were to take AI’s advice, I would deny my readers the most gratifying bit of subtext in the entire story. That isn’t surprising, assuming AI’s suggestions were based on all the literature it has ingested, most of which hovers around average.

I’m not aiming for average.

My conclusion? The Alpha readers win. It seems the machine simply cannot reconcile the human equation. At least, not yet. That’s too bad for AI. We’re fascinating creatures, especially when we hide our motives from ourselves.

I’ll leave it here. In a recent survey of 1,200 authors, nearly half said they let AI outline or plot for them. More than a quarter said they let AI write for them.

Ethics aside, that’s like letting strangers piss in your swimming pool. Once the dead, soulless voice of the robot contaminates your writing, it remains until you drain the pool and start over.