Editor Vs AI: NYT Copy Edit

Editor Vs AI: NYT Copy Edit

Let’s try this again: I’m excited to have more tools available, but is AI ready to replace a human editor? Last time, I challenged various editing tools to fix “10 common grammar mistakes.” For this next test, I used the first New York Times Copy Edit This! challenge to see what the “editing software” has to say compared to a real-life, human NYT copy editor. To test this, I compiled the test sentences into a (really bad) paragraph and had three “editing tools” offer their suggestions. This time, I’ve organized the results by sample, with the results from each tool grouped below it:

  1. MS Word’s full Editor which purports to check spelling, grammar, and clarity
  2. Grammarly
  3. ChatGPT which is an interactive bot, not a “checker”
  4. NYT explanation from their quiz, rephrased

Making the ChatGPT AI work involves asking it to explain what is wrong. Different questions yield different results. This time, I asked it “What is the copyediting problem in [the sample text].” (Yes, I chose copyedit as a closed compound for the query to add clarity.) Ask it a different way — like even, “explain” instead of “what is” — and you will get a different answer. The ChatGPT seems to have a better grasp of what it means to copy edit something than most people do. See sample 8 for a case study in this or give it a try yourself!

To keep the length of this post under control, I had intended to only share the samples that Grammarly had an opinion on, but there weren’t enough to create a meaningful post. Some of ChatGPT’s replies were summarized, as it tended to give long, detailed explanations and repeat the sample with its suggested fixes inserted, and that was overly long for an already long blog post. For Word, I turned on every single check within the Editor and reported its findings verbatim.

  • 2—Word
  • 3—Grammarly
  • 4—ChatGPT
  • 10—NYT
Overall scoring against the NYT answers.

The NYT site offers an actual quiz with longer explanations, and here be spoilers. So you may want to go try the quiz yourself, first, then see how your assessments compare! You may decide there are other things wrong with the samples, but this post is only comparing the NYT’s own quiz answers to the AI.

Sample 1: The patchwork of laws governing background checks, assault-weapons limits and open-carry practices help explain why people continue to be wounded and killed.

Sample 2: After receiving his doctorate at Princeton, the university hired him as an assistant professor and promoted him to full professor in 1965.

Sample 3: He has been dogged by reports — many published in Newsday — that he had received free gifts and vacations from a longtime friend, Harendra Singh, a Long Island restaurateur with about 30 businesses in the area and several government contracts.

Sample 4: The names of the sergeant, whom Chief Nikunen said was an eight-year veteran of the New York Police Department, and the woman were not released.

Sample 5: The total amount available to disperse will be $150 million annually.

Sample 6: In Tutzing, the Prince’s three-story villa is defended from prying eyes by a fence and hedge more than six-feet tall.

Sample 7: HERNDON, Va. — Bugs Bunny on Monday called for strengthening the nation’s defenses against cyberattacks, saying that as president he would create a joint law enforcement task force to handle both cybersecurity as well as offensive cyberwarfare.

Sample 8: Though the terminology and process is (wildly, needlessly) complex, the advice is simple for anyone wanting to borrow $25,000: Take out federal student loans from the government, not private ones that come from a bank or similar institution.

Sample 9: The only criteria is that the player played for five seasons, made one All-Pro team or Pro Bowl and has been retired for five full seasons.

Sample 10: It occurred when American aircraft were called in to aide a police post that was under attack in Uruzgan, a province in the country’s south. At least seven police officers were killed, Afghan officials said.

Conclusion: The NYT must keep the human copy editors

Software is getting better at judging subject–verb agreement and wrong word choice, but it isn’t ready to take over, yet.

Up Next

In future posts, I’ll give ChatGPT a shot at editing for style, flow and concision, things that other tools don’t even pretend to do. You may also want to read the first post in the series in which tested the AI against “10 common grammar mistakes.” Also check out Australian editor Rhonda Bracey’s tests using GPT for rewriting memos. Spoiler: it’s not ready to take over.

Want to learn more about editing?

Start with this free sneak peek at proofreading in which a proofreader (me) explains their thinking as they live-proof a single page of a book. Then take one of our training courses or check out Editing in Word 365 for technical help. And be sure to check out our full list of free resources on how to become an editor as well as our Starter Kit collection.


Picture created by the art AI bot Photoleap based on the author’s query: a photo of a steampunk robot reading the New York Times newspaper in Times Square.

Leave a Reply