You are Here
Editor Vs AI: CMOS Grammar
computers copy editing editing Editor vs AI MS Word proofreading software

Editor Vs AI: CMOS Grammar

For this investigation in the series, I scraped some grammar examples from the CMOS “Chicago Style Workout” quizzes on their Shop Talk blog. Some test sentences were adapted from questions as not all of them were in sentence form. ChatGPT-3 did the best of all, correctly fixing 33/43 examples when asked to “Fix the grammar in this.” Three of the “errors” tested might be considered a matter of style (a choice), and there’s no way to tell any of these AI (see note at end) that we are looking for them to apply CMOS style. So you might award three more points for those, across the board.

This time, I’ve grouped the test sentences by how well the tools dealt with them. As with other summaries in the series, I have paraphrased the answers/suggestions for the sake of brevity. The answers that matched the CMOS solutions are set in boldface. Links within the CMOS answer for each example will take you to the relevant quiz page. I note that Word now phrases its suggestions as questions, absolving itself of responsibility; sometimes I kept that wording.

The sentences and results are hidden for ease of reading. Click the “The test sentence corrections” to reveal them in each section.

The Score

MissedMisidentifiedScore
Word29103/43
Grammarly29310/43
ChatGPT-364+33/43

*One example contained no errors, so the numbers above don’t sum up neatly.
+ = Also got a few corrections half right (e.g., there were two errors).

What AI gets right

Sometimes our automated assistants are somewhat helpful! Word lags a bit behind the performance of the others on these examples, but these misspellings and comma errors were the most consistently corrected by them.

The test sentence corrections

This is the drivers seat.

  • Grammarly: driver’s
  • ChatGPT: driver’s
  • Word: driver’s
  • [See CMOS 17, paragraph 5.20] driver’s seat

The results from June 5, 2021 are now available.

“Are you sure?,” I asked.

We need to ask the Jones’.

I don’t know if I’m sick or sad, but I feel badly.

What AI can’t detect

Dangling modifiers slip right by these tools. Granted, a lot of readers don’t even blink at these. A dangler occurs because the right subject is missing, so the modifier refers to the wrong thing. There’s often no confusion caused by danglers, as a misreading would be absurd. (As in some examples here.) But we sure do see a lot of them in the wild! Here’s another explainer.

The test sentence corrections

Who was it who said, “To be, or not to be, that is the question”?

Valued for their keen sense of smell, trainers recommend beagles for airport security work.

Writing out the answers, it was clear that I would ace the exam.

  • Grammarly: [no errors flagged]
  • ChatGPT: became clear
  • Word: [no errors flagged]
  • [See CMOS 5.112] Who is writing?

Given how much Word likes to flag constructions as passive, I’m disappointed that it missed one that actually is passive. But the error here is the dangling modifier. We don’t know who wrote out the answers.

Peter and Harriet’s correspondence were both in the file.

Hilda and his vacation.

  • Grammarly: [no errors flagged]
  • ChatGPT: [no errors flagged]
  • Word: [no errors flagged]
  • [See CMOS 17, paragraph 5.22] Hilda’s

The audience showed their appreciation. The audience rushed back to its seats.

We never fully reconciled with the lack of Wi-Fi in our basement apartment.

I have been to both Harper’s Ferry and Pike’s Peak.

Twitter is a popular, if controversial, source of news.

In addition to missing the error, ChatGPT-3’s suggestion changes the tone in a way that an editor might not want.

The tortoise finished one minute, thirty seconds ahead of the hare.

Word’s troubled obsession with passive voice

Word is like that desperately in-over-their-head Jeopardy contestant who is constantly slamming their buzzer and saying “What is passive voice, Alex?” Remember, passive voice is a grammatical construction, not an error. Editors leave it alone when necessary or when it fits the flow. Some of these are passive, but not all of them!

The test sentence corrections

A class-action lawsuit was filed in 2017 three years after the app debuted.

The discussions centered around editing.

The author was vexed with all the changes involving prepositions.

Public opinion is divided sharply.

After subtracting his expenses from his estimated income, Omar’s budget had to be revised.

This one actually is written in the passive voice [by zombies]. Well done, Word!

The nonstandard use of apostrophes in plural’s has often been blamed on grocers.

  • Grammarly: plurals
  • ChatGPT: plurals
  • Word: passive voice
  • [CMOS] delete apostrophe

This was needed for the mices’ cage.

  • Grammarly: mice’s
  • ChatGPT: mice’s
  • Word: passive voice
  • [See CMOS 17, paragraph 5.20] mice’s

When ChatGPT has your back

In addition to its successes in other categories, here are some examples where only ChatGPT-3 knew what the problem was.

The test sentence corrections

Whatever will be will be.

She has two PhD’s and spells her name with two n’s.

In a conversation about songwriters, she pointed out that the poet Robert Burns wrote many songs.

She read her parents’-in-law message.

This was an idea of Hills; she was a friend of my grandfathers.

  • Grammarly: grandfather’s
  • ChatGPT: Hill’s, grandfather’s
  • Word: [no errors flagged]
  • [See CMOS 17, paragraph 5.21] Hill’s, grandfather’s

Peter’s and Harriet’s correspondence refers to the correspondence between Peter and Harriet.

The author’s riveting, three-page conclusion broke new ground.

Where they disagree

These challenges fell into the win some, lose some department for the AI.

The test sentence corrections

Before thinking up questions for the game, they divided up the categories and assigned them to teams.

  • Grammarly: [no errors flagged]
  • ChatGPT: delete up
  • Word: [no errors flagged]
  • [See CMOS 5.113] Not a dangler

This was the fox’ den.

  • Grammarly: fox’s or foxes
  • ChatGPT: fox’s
  • Word: [no errors flagged]
  • [See CMOS 17, paragraph 5.20] foxes’

Every college and university encourages their students to succeed.

  • Grammarly: its
  • ChatGPT: its
  • Word: Make encourages singular
  • [See CMOS 17, paragraph 5.33] the pronoun referring to the antecedents is singular: its students

Neither the orange nor the peach smells as sweet as they should.

  • Grammarly: [no errors flagged]
  • ChatGPT: it
  • Word: [no errors flagged]
  • [See CMOS 17, paragraph 5.33] singular pronoun it

We three—Bruce, Felipe, and me—traveled together. She asked us—Barbara, Sarah, and I—to move our cars.

  • Grammarly: [no errors flagged]
  • ChatGPT: I, me
  • Word: [no errors flagged]
  • [See CMOS 17, paragraph 5.36] swap I and me

The test would be simple for you or I. Read this and tell Laura and I what you think.

  • Grammarly: me [once]
  • ChatGPT: me [once]
  • Word: [no errors flagged]
  • [See CMOS 17, paragraph 5.37] me & me

The patient developed a rash on the hands.

Matters of (CMOS) style

The quiz items in this section did not necessarily contain outright errors but, rather, they are style choices particular to CMOS. To be fair, the AIs didn’t know we wanted these in CMOS style. There’s no reason they would have applied it.

The test sentence corrections

Mr. Blain’s background qualified him for the job.

  • Grammarly: [no errors flagged]
  • ChatGPT: [no errors flagged]
  • Word: [no errors flagged]
  • [See CMOS 17, paragraph 5.29] Mr. Blain had a background that qualified him for the job.

The original sounds colloquially correct. CMOS may be being a bit more formal than most editors would be, here.

Here we see the horse and rider.

CMOS might just be using a more formal register, as we say. Many people would not see an error here.

“Austin, Texas is one of the Gulf States,” I said to my friend.

They visited the University of Wisconsin in 1971 but somehow managed to miss the concert by the Grateful Dead.

We were assigned the Invisible Man by H. G. Wells.

We read about it in the Detroit Free Press.

“Yes we’ll be there,” I said.

Defining AI

The term “artificial intelligence” is used pretty loosely and liberally these days. Many are using it to mean any algorithm that can do what it used to take a human to do. But that is too broad for me, as even a simple calculator might qualify. To be “intelligent,” that algorithm must learn as it goes, improving itself without further human instruction. A user clicking “add to dictionary” isn’t enough to qualify Spellcheck as self-learning, for example. In this sense, Spellcheck and PerfectIt are not AI. Grammarly may be AI; I don’t know how it works. The translator app on an iPhone isn’t likely to fit this definition either, as it uses crowd-souced training to improve. The predictive text on phones, however, is more like AI than those other examples, as its predictions get better the more you use it. It “learns” your style (or at least your tendencies).

But sometimes we use terms that capture readers’ attention rather than only what is accurate, like when I answer the call to “look something over” instead of the precise “stylistic edit” or whatever they actually want. And as for why I keep saying “an” or “this” GPT, it’s because there are several iterations of Generative Pre-trained Transformer out there. ChatGPT-3 is just the (free) instance that I’ve been testing.

This slide set provides a great overview of the various queries and answers that ChatGPT can provide.

Be sure to check out the other tests in this series as well as comparing these results against the next iteration: ChatGPT-4.

The image for this post was created using the Dall-e 2 AI art generator and the author’s prompt of “a robot reading a big orange book”

Related Posts

Leave a Reply