Media, Publishing & Streaming// diagnostic

Machine-drafted copy is reaching the live site unreviewed

In short

Query your published items for a provenance flag and the flag will usually turn out to be absent, or present and optional. AI generated text published without editor review is a missing-field problem rather than a lapse of discipline, and until the drafting tool writes provenance itself and the publish action refuses to run without it, the rule is only forbidden, never impossible.

Key takeaways

  • The confirming check is a join, not an interview: published items against assistance records.
  • A flag that defaults to empty measures conscientiousness, not usage, and the careless never fill it.
  • Provenance written by the tool at insert survives copy-paste; provenance written by a form at save does not.
  • Automation feeds are the biggest unflagged category in most newsrooms, and the exemption is never written down.
  • A publish-time block belongs on the server transition, because scheduled and API publishes skip every UI check.
  • For the archive, record not-recorded. Detectors misclassify, and a wrong accusation costs more than the gap.

Before you investigate anybody, try to write one query: every item published in the last 90 days, joined to whatever record says a model touched it. In most newsrooms the query cannot be written, because no such record exists. Where it does exist, it is optional, which means it is filled in by exactly the people who were never the risk. That is the finding, and it reframes the problem: this is a missing field with a discipline problem sitting on top of it, not the other way round.

The distinction matters because the 2 framings have different fixes and only 1 of them works. You cannot enforce a policy against something your platform cannot see. A paragraph added to the style guide makes the failure forbidden. A required field written by the tool, plus a publish action that refuses to run without it, makes the failure impossible.

The join that tells you which problem you actually have

  1. List every item published in the last 90 days with its section, its byline and whether any assistance record exists against it. If there is nothing to join to, stop — you have your answer, and everything below is about creating the record rather than policing it.
  2. Where records exist, split them by what wrote them. A flag ticked by a human and a flag written by the drafting tool at the instant text was inserted are different measurements. The first tells you who is conscientious; only the second tells you what happened.
  3. Cross the unflagged items against the desks that have tooling. A section with an approved drafting assistant and 0 flagged items in 90 days did not abstain — its route to the page runs around the field, and it will be 1 of the 5 below.

Two supporting counts narrow it further. Publish events with no preceding review transition expose the bypass route. Items whose body arrived as 1 large edit rather than accumulating over a CMS session are the signature of a paste. Neither is proof alone; together they cut the search from the whole newsroom to 2 or 3 routes.

Five ways generated text reaches readers with nothing attached

RouteWhat happensSignal in your dataThe gate
Generation happens outside the platformA staffer or contributor drafts in a separate tool and pastes the result into the CMSNo record anywhere; the body arrives as 1 large edit rather than a typing sessionUndetectable at the text layer. Make provenance a question the filing form asks
The flag exists and is optionalThe field defaults to empty and saving works without itFlagged share far below what the desk says it usesRequire it, with an explicit none value, so silence becomes a validation error
The assist sits in a side panelStructured provenance stays with the suggestion; only the characters move into the bodyProvenance on summaries and tags, absent on body textWrite the record on insert, from the editor component, not on save from a form
Automation feeds treated as exemptResults, listings, weather, market data, notices and transcripts publish via a service accountItems with no human byline and no assistance record, often at fixed timesThe exemption may be fair policy, but it must be an explicit value on every item
The review state can be skippedPublish permission carries a workflow bypass, usually justified by breaking newsPublish events with no preceding review transition, clustered on a few accountsRemove the bypass, or log it as a named exception with a reason and an owner
How unflagged machine output arrives, what it looks like in the data, and the gate that closes each route

Ranked by how often each turns out to be the real one: automation feeds first, because nobody ever decided they were exempt and everybody assumes somebody did; generation outside the platform second, because it is invisible and therefore never disproved; the optional flag third. The skippable review state worries desks most and accounts for the fewest items.

The record the drafting tool writes for itself

A boolean saying assisted true or false is not enough to act on, and a free-text box is not enough to query. The record has to answer what was done, to which part, by whom, when, and who accepted it. Five values written automatically beat a page of guidance.

  • The stage. Research, drafting, headline variants, translation, transcription, summarisation, illustration — 7 categories carrying very different weight, which a single assisted boolean flattens.
  • The target. Which field, or which range of the body, so a machine-written standfirst on hand-reported copy is distinguishable from the reverse.
  • The operator. The account whose session ran it, which is what makes the record answerable later rather than merely descriptive.
  • The instant, as ISO 8601 with an offset. Generation time is separate from publication time, and the gap between the 2 is the single most useful measure of whether review is happening at all.
  • The acceptance. Who reviewed the output and when, as its own event rather than inferred from the publish state. Generation and acceptance are 2 facts, and conflating them is how the audit stops being usable.

Where the block has to sit for it to be a block

A check living in the publishing screen stops the 1 route that was already least dangerous. The gate belongs on the server-side publish transition — the single place every route passes through — enforced identically on all 4 of them.

  • Editor publish. The obvious path, and usually the only one anybody tests.
  • Scheduled publish. Fires with no session and no screen, so a UI-level check is bypassed by design rather than by accident.
  • API and integration publish. Wire ingest, partner pushes, mobile tooling and internal scripts reach the same objects, usually through a service account with wide scopes.
  • Bulk republish and migration. The path that re-emits thousands of items during a replatform, and where a validation rule added 2 years ago is most likely to have been switched off to get the job finished.

Keep the block narrow. Refuse the transition when the assistance record is absent, and name the field that is missing. Do not make it a judgement about content: a gate that tries to decide whether text is good enough gets switched off within a month by whoever is on late shift. Deciding whether raw output is publishable is a separate editorial pass with its own criteria, described in the pass that turns raw output into publishable copy.

A rule that can only be broken by carelessness is a rule. A rule that cannot be broken at all is a gate. Newsrooms keep writing the first and calling it the second.

Four things a provenance record still leaves open

It does not settle what readers are told. Whether a translated quote and a drafted news-in-brief belong on the same side of the line is a policy argument with a real disagreement in it, argued out in where to draw the disclosure line. It does not stand in for verification either: knowing a paragraph was drafted says nothing about whether the figure in it was checked, which is what the claim-level evidence in a fact-check record that survives a complaint is for.

It does not cover pictures. Images and video carry manifests of their own, under standards such as C2PA and the IPTC vocabularies, and those manifests are lost in the resize and CDN transform steps rather than at the point of capture — a different engineering problem, set out in keeping provenance metadata alive from camera to CDN. And it does not reach work filed from outside the building unless the commissioning record asks the question at brief time and carries the answer through filing and payment, which is the trail described in commission, file, sub, pay.

What to do about everything already published

Date the gate and be honest about the boundary. From the switch-on instant, every item carries an explicit answer. Before it, the correct value is not-recorded, stored as its own state rather than collapsed into none. An archive where absence means 2 different things is an archive nobody can query.

Resist the urge to run a detector over the archive. Classifiers that claim to identify machine-written prose return both false positives and false negatives, and no threshold avoids both at once. The 2 error types cost wildly different amounts in a newsroom: a false positive is an unsubstantiated accusation against a named reporter, which does more damage than an honest gap. Where a specific item matters — a complaint, a legal query — reconstruct it from edit history, item by item, and record the conclusion as evidence rather than as a score.

The work is small and unglamorous: 1 field, a writer in the editor component, a server-side check on 4 publish paths, and a weekly report. What makes it awkward is that it has to be threaded through a CMS somebody else built — the shape of work we take on as MVP and product builds inside our media and publishing practice. Two neighbours in editorial workflow and newsroom AI are worth reading alongside it: who is allowed to press publish, because the bypass is a permissions question, and stories that sit between review states, because a gate with no owner becomes the slowest step of the day.

Frequently asked questions

Short answers to the follow-ups this page tends to raise.

How do we find out whether machine-drafted text has already been published on our site?

Join published items to assistance records for the last 90 days and see whether the join can be written at all. If there is nothing to join to, that is the finding, and asking around will not improve on it. If records exist, separate the ones a tool wrote automatically from the ones a person ticked, and treat only the first as a measurement. Then cross the unflagged items against the desks that have tooling.

Should the provenance field be required, or will that just annoy the desk?

Required, with an explicit none option and no default. A field that defaults to empty is filled in by the people who were never the risk, so it measures conscientiousness rather than usage. What must not be required is a justification box: asking for prose turns a gate into paperwork people route around.

Do automated results, listings and transcripts need the same flag?

Yes, as an explicit value rather than as an exemption. Sports results, property listings, market data and machine transcripts are usually the largest unflagged category in a newsroom, and nobody ever formally decided they were exempt. Whether readers see anything is a separate question. What matters technically is that the item carries a recorded answer, so no category exists where absence quietly means something different.

Can a detector tell us which old articles were machine-drafted?

No, not to a standard you would act on. Detection classifiers produce both false positives and false negatives, and the two errors cost very different amounts in a newsroom: an unsubstantiated accusation against a named reporter is far more damaging than an archive with a documented gap. Record everything published before the gate as not-recorded, kept distinct from a recorded none, and reconstruct individual items from edit history only when a complaint or a legal query makes it worth doing properly.

Where should the publish-time block live in the stack?

On the server-side publish transition, not in the publishing screen. A UI check covers the 1 path that was already safest and misses scheduled publishes, API and wire ingest, and bulk republish during a migration. Enforce the same rule on all 4, keep the failure message specific about which field is missing, and log every override with a named person and a reason.

  • editorial workflow
  • provenance
  • newsroom AI
  • publishing controls
// shipped work

The work behind this page

Builds from our portfolio that this page draws on.

Read next

Working on something in this space?

Tell us where you are in a sentence or two. We'll tell you honestly whether we're the right team, and what a sensible first slice of the work looks like.

Start the conversation