Outlook
Part of Paid social agency planning: UK platform and policy signals
Designing a narrow AI claim-triage test for paid-social copy with a human claims owner
A design for a narrow AI claim-triage test on paid-social copy, with synthetic examples, a manual baseline, a named human claims owner and stop rules.
A defensible first AI application is narrow: flag possible evidence and review issues in draft paid-social copy before a human considers release. The system does not approve claims, choose an audience, upload data, set spend or publish. A named claims owner makes the decision.
This is an editorial test design for work commissioned in England, not a report of a completed trial. It does not imply that a product is accurate, suitable or available in any account.
What to take away
- A narrow AI task can flag possible evidence and review issues in draft paid-social copy before a human considers release.
- The system must not approve claims, choose an audience, upload data, set spend or publish; a named claims owner decides.
- Use a synthetic acceptance set and freeze the rule set, prompt, model version, expected flags and manual answers before the run.
- Keep privacy, security, accessibility, financial-promotion and procurement gates independent, with separate owners approving each.
- Stop and roll back if the system attempts a prohibited action, exposes protected data, misses a critical example or loses traceability.
Fix the task and manual baseline
Input only synthetic draft adverts and a controlled rule set. The output is a list of passages that may contain an objective claim, price, comparison, endorsement, regulated-product statement or missing qualification. Each flag points to the exact text and rule category. It offers no legal conclusion.
The manual baseline is the existing review: a trained person reads every draft, matches objective statements to the evidence register, routes specialist topics and records approve, amend or hold. CAP Code rule 3.7 says marketers must hold documentary evidence for objective claims capable of substantiation before publication. The AI step cannot waive that requirement.
The CMA's March 2026 consumer-law guidance for AI agents calls for active human checking and quick action when outputs are wrong. It includes a marketing-campaign example. That UK guidance supports human oversight, not automatic clearance.
Use a synthetic acceptance set
Prepare deliberately varied examples without personal or client-confidential data. Include supported and unsupported facts, qualified and unqualified prices, obvious and subtle comparisons, clear and obscured disclosures, ordinary products and financial-promotion examples. Accessibility reviewers add samples where meaning depends on colour, image text or missing alternatives.
Before the run, freeze the rule set, prompt, model or service version, expected flags and manual answers. Record:
- missed-risk rate
- expected flags not raised divided by all expected flags;
- false-alert rate
- incorrect flags divided by all flags raised;
- explanation completeness
- flags with an exact passage and rule category divided by all flags; and
- action boundary breaches
- attempted publication, spend, audience or account changes, recorded as a count.
State population, period, exclusions, reviewer disagreements and confidence with every result. The owner sets project-specific thresholds before seeing the output. No industry benchmark is available here.
Keep every gate independent
Privacy and PECR reviewers decide whether any later use of real drafts, audience information, pixels or identifiers is permitted.
The ICO's AI and data protection guidance is UK-focused, not a statutory code, and marked under review after the Data (Use and Access) Act.
Recheck it before a production test. The ICO's final storage and access guidance is a separate source for online-advertising technologies.
Security approves data boundaries, supplier access, logs, model changes and revocation. The NCSC secure AI guidance covers design, deployment and operation but does not endorse a tool.
An accessibility reviewer tests the full advert and landing task against the agreed WCAG 2.2 scope. A financial-promotion specialist applies the FCA's social-media guidance where relevant. Procurement, contract and finance owners separately approve supplier terms, access, spend and exit.
Stop and roll back
Stop if the system attempts a prohibited action, exposes protected data, misses a pre-declared critical example, loses traceability, or changes version without approval. Disconnect its account access, quarantine its output and return every draft to the manual queue. Recovery evidence is a completed manual review, revoked credentials, preserved log and repeated synthetic test after correction.
Only the claims owner can restart, after each affected specialist has accepted the new evidence. Until a named author, fact-checker and qualified reviewers complete their work, this method remains on publication hold.
Before you act
- Define the narrow task and manual baseline before any test.
- Prepare synthetic drafts without personal or client-confidential data.
- Freeze rule set, prompt, model version and expected flags.
- Record missed-risk, false-alert, explanation completeness and boundary breaches.
- Obtain independent approvals from privacy, security, accessibility and financial-promotion reviewers.
- Agree stop conditions and rollback steps with the claims owner.
Common questions
What does the AI system actually do in this test design?
It flags passages in synthetic draft adverts that may contain an objective claim, price, comparison, endorsement, regulated-product statement or missing qualification. Each flag points to the exact text and rule category. It offers no legal conclusion and does not approve claims, choose an audience, upload data, set spend or publish.
What metrics are used to evaluate the AI step?
The design records missed-risk rate, false-alert rate, explanation completeness and action boundary breaches. Missed-risk rate is expected flags not raised divided by all expected flags. False-alert rate is incorrect flags divided by all flags raised. Explanation completeness is flags with an exact passage and rule category divided by all flags.
When must the process stop and roll back?
Stop if the system attempts a prohibited action, exposes protected data, misses a pre-declared critical example, loses traceability, or changes version without approval. Then disconnect its account access, quarantine its output and return every draft to the manual queue. Only the claims owner can restart after each affected specialist accepts the new evidence.
