Tools and providers
Part of Selecting paid social tools and suppliers through privacy, security and exit gates
Paid social agencies software selection: a sandbox test plan
A paid social agencies software selection test plan: one requirement row, a sandbox with synthetic records, failure cases and a manual baseline.
Paid social agencies software selection ends in a test, not a demonstration. A salesperson's walkthrough is supplier evidence. Only a trial run by the agency's own staff, on records that cannot hurt a client, counts as buyer acceptance testing and tells you whether the tool does the job.
What to take away
- Turn each requirement into one test with an expected result written before the trial begins.
- Use a sandbox or an empty client-owned account, with synthetic records, restricted roles and no authorised spend.
- Let an intended user complete a critical task by keyboard and with relevant assistive technology.
- Run failure cases on purpose and record expected result, observed result, tester, configuration and defect owner.
- Compare the shortlist with a manual baseline, and rehearse departure on paper before signature.
Start from one requirement row
A test needs something to test against, so write the requirement first. Name the operator, input, action, output, frequency and evidence retained for the workflow, then derive the trial from that one row.
Requirements File Fields
- Operator, input, action, output
- Frequency and evidence retained
- Account permissions and media payment
- Creative approval and audience handling
- Tracking, reporting and invoicing
- Account owner and stop authority
A completed row can look like this, and the trial then checks each cell:
| Workflow | Operator | Input | Action | Output | Frequency | Evidence retained |
|---|---|---|---|---|---|---|
| Weekly change record | Agency analyst | Client-owned Meta Ads account | Export named change fields for the past week | CSV with date, user, field, old value and new value | Weekly | Client drive file plus reviewer sign-off |
For this row the pass condition is plain. The analyst produces the CSV with every named field, and a reviewer signs it off. A tool that exports a different shape, or needs an administrator to do it, fails the row.
Build the sandbox
Shortlist only candidates that clear the written evidence gate, then trial them in a sandbox or an empty client-owned account. The trial uses synthetic records, restricted roles and no authorised spend, so a mistake costs the agency time and nothing else.
Synthetic records also keep personal data out of the test. If a trial seems to need real audience files, the question to settle first is the one raised in mapping data protection role by role, before any real record is loaded.
Restricted roles matter as much as the records. Give the tester the role an agency analyst would really hold, not an administrator's, because a feature that only works for an administrator will not work in daily delivery.
Test with keyboard and assistive technology
Ask an intended user, not the project lead, to complete a critical task with the keyboard and the relevant assistive technology. The task should be one the requirement row names, such as approving a change or exporting the audit record.
WCAG 2.2 offers testable criteria, but conformance cannot be inferred from a badge or an automated scan alone. Treat a supplier's accessibility statement as a claim to check, and write down what the user could and could not do.
Record the result in the same way as every other test: expected result, observed result, tester and configuration. An approval path that cannot be operated accessibly is a hard exclusion, not a defect to defer.
Run failure cases on purpose
A tool that works on a good day proves little. Break it deliberately, and write the expected result before each step so the observed result can disagree with it.
| Failure case | Expected result to write down | What to record |
|---|---|---|
| Revoke an operator | The operator can no longer act in the account | Tester, configuration, observed result |
| Reject an approval | The change does not go live | Observed result, defect owner |
| Disconnect an integration | The failure is visible to a named person | Observed result, who was told |
| Export the audit record | The record leaves the tool in a usable format | File, format, defect owner |
Each row needs the same fields: expected result, observed result, tester, configuration and defect owner. The wider habit of rehearsing stops before launch is described in running delivery through evidence gates.
Decide who owns each defect before the trial starts. A defect with no owner is simply a note, and the team roles model shows how an agency can name that person in advance.
Compare with a manual baseline
For an agency, compare the shortlist with a manual baseline: client-owned native accounts, controlled templates, local decision logs and scheduled exports. Include staff time and control gaps, so the comparison is honest about what the manual route costs.
Buying is justified only if the selected service solves the defined problem after all exclusions pass. If the manual baseline meets the requirement row with acceptable effort, the trial has still done its job by showing that.
Send each candidate the same evidence schedule, as described in the guide to ad account access gates, so the tool and the manual route are judged against identical questions. The privacy, security and exit gates for suppliers sit beside this test plan.
Make the decision reversible
Before signature, rehearse departure on paper. List the assets, formats, deletion evidence, retained legal records and revocation steps, then note the billing closure and the continuity owner.
Security and contract questions belong in the same pass. The NCSC publishes supplier assurance questions, including whether the contract states requirements for managing and reporting incidents.
For personal data, the ICO's guidance on contracts between controllers and processors explains what needs to be included in a contract. A named privacy reviewer should map each actual processing purpose against it.
The security lead, accessibility specialist, commercial reviewer and UK privacy or PECR adviser should each sign their unresolved items. Recheck the quote, contract and live supplier pages immediately before approval. What happens once a tool is chosen is covered in the guide to tool implementation.
Before you act
- Write one requirement row and the expected result for it.
- Set up a sandbox with synthetic records and restricted roles.
- Have an intended user complete a critical task by keyboard.
- Run the four failure cases and record observed against expected.
- Compare with the manual baseline and rehearse departure on paper.
Common questions
Why is a supplier demonstration not an acceptance test?
A demonstration performed by a salesperson is supplier evidence. Acceptance testing is run by the buyer, in a sandbox, on synthetic records, with the roles the agency's staff will really hold.
What should be recorded for each failure case?
Expected result, observed result, tester, configuration and defect owner. The expected result is written first, so the trial can show a difference instead of confirming what was assumed.
When is buying justified?
Only if the selected service solves the defined problem after all exclusions pass, when compared with the manual baseline of client-owned native accounts, templates, decision logs and scheduled exports.
