BroadOakData.UK – Data is power

An AI assessment is only as good as the facts behind it, and most of the time goes on chasing them. Here is what a report that holds up needs, what AWS’s published guidance says a sound programme looks like, and the five decisions that belong to your executive team.

Most of the effort in an AI assessment goes on chasing facts: baselines, access details, policies, and who signs off. Those facts feed five outputs, and a report that skips one leaves the decision half made. This post sets out what the five outputs are, which facts they depend on, what AWS’s published guidance says a sound programme looks like, and what each output looks like for a fictional insurer.

Meet Brightwater Mutual

Brightwater Mutual is a fictional UK insurer with about 1,200 staff, a claims team that reads every incoming document by hand, and a board that wants “an AI strategy” by the end of the financial year. Its operations director, Priya, asks for an AI assessment. Three ideas come to the first call:

Claims triage agent. Read a new claim, check the policy, request missing documents and route the claim to a handler.
Broker email extraction. Pull the policy number, cover type and requested change out of broker emails and fill the admin form for a person to check.
Policy wording search. Let staff ask questions of the policy wordings in plain English.

All three sound reasonable. The assessment exists to say which one comes first, which needs an agent, and what has to be true before anyone builds.

What an assessment should deliver

A complete assessment ends with five outputs. Each one answers a question the executive team has to settle.

Output	The question it answers	What it contains	Brightwater
1. Use case discovery	What could we use AI for, and which ideas deserve a closer look?	Candidate use cases described the same way (what it does, who uses it, what data it touches), then scored and shortlisted	Three ideas: claims triage, broker email extraction, policy wording search
2. Agentic readiness	Are our data, architecture, governance and organisation ready, and does the idea need an agent at all?	Readiness scored 0 to 4 on four dimensions, with blockers and a gap list, plus agentic fit scored out of 20	Claims triage: 1.9 of 4 and five blockers. Broker extraction: 2.5 and one blocker
3. Platform feasibility	Can we build and run it, and which AWS service fits?	A feasibility score out of 25 and a platform decision based on who builds it, who uses it and where it lives	Claims triage: 14 of 25, Amazon Bedrock AgentCore. Broker extraction: a plain model call. Policy search: Amazon Quick
4. ROI and business case	What does it cost, what does it save, and when does it pay back?	Saving, payback and return worked out from the numbers entered, with “To confirm” wherever a number is missing	Broker extraction pays back in anywhere from 12.6 to 37.1 months, depending on one unknown
5. Recommendation	What do we do first, and what must be true before we build?	A starting use case, a “foundations first” flag where blockers pile up, and a gap list with owners and dates	A small broker extraction proof of concept first. Claims triage waits until its foundations are closed

The rest of this post covers the facts these outputs rest on, then returns to each output with Brightwater’s numbers.

The facts a good assessment rests on

Area	What to collect	Typically held by (executive)	Send ahead?
Strategy	Top 3-5 priorities for the next 12-18 months, the event creating urgency, the sponsor and the budget window	CEO, COO	Yes
Process baseline	Items per month by type, manual minutes per item, loaded hourly cost, error and rework rate	COO, CFO	Yes, then check
Data	Where each source lives, volume, quality, freshness, and a labelled set that shows what “right” looks like	CDO, CIO	Partly
Systems	AWS account and Region, access management, deployment pipeline, monitoring, and whether each tool the agent needs has an authenticated API	CIO, CTO	Yes
Governance	Data classification, regulation and residency, who owns AI incidents, rules for third-party content, any under-18s or vulnerable groups	CRO, CISO, DPO	Partly
People	Who builds and runs it, who signs off before a proof of concept, tools staff already use	COO, CHRO, CIO	Yes
Review effort	Minutes a person needs to check one AI draft	Nobody yet	No: measure it in the pilot

The last row matters most. Every business case that promises a large time saving rests on it, and nobody can answer it before a pilot.

What goes wrong when the facts are thin

Scores without evidence. A use case gets a 5 for impact because the loudest person in the room said so. Without the baseline and the priorities beside it, nobody can challenge the score later.
A business case with no baseline. “80% less effort” is quoted from a vendor slide. Without minutes per item and review time, the figure cannot be checked, and the report either repeats it or quietly drops the numbers.
Unknowns that look like answers. A readiness table shows every cell filled in because the author guessed the gaps. The customer reads “conditionally ready” and never learns that nobody asked about permissions, regulation or an incident owner.

At Brightwater, a claims agent that reads outside documents cannot be called ready until someone has said which sources are allowed and who owns the response when it gets a claim wrong.

What AWS recommends

Four pieces of AWS guidance matter here. The Cloud Adoption Framework for AI asks whether the organisation is ready, across six perspectives: Business, People, Governance, Platform, Security and Operations. The Well-Architected Generative AI Lens covers how to build and run a solution, starting with scoping. The Well-Architected Agentic AI Lens, published in June 2026, covers what changes when AI acts on its own. The Responsible AI Lens, which AWS describes as the foundation for the other two, covers fairness, safety and accountability. This article is not an AWS product or endorsement, and none of it replaces a full architecture review.

Nine practices come out of that guidance. Each ends with a question an executive can put to the team.

Start from the outcome, with a pass mark. The lifecycle begins with scoping: define the problem, decide whether generative AI is the right tool, check feasibility, data and cost, and set success measures. Ask: which number moves, who owns it, and what counts as a pass?
Use an agent only where one is needed. AWS describes autonomy as a range, from an observer that only reads to an orchestrator of other agents. The lens says that autonomy calls for stronger security controls and human oversight, and agents that loop through many reasoning steps cost more to run. Ask: what is the simplest design that meets the outcome?
Give the AI its own identity and the least access that works. AWS recommends strong authentication for agents, permissions kept separate from those of the person using it, and regular access reviews. Ask: whose credentials does it use, and what can it touch?
Match human oversight to the consequence. Low-risk actions proceed alone, medium-risk ones notify someone, and high-risk ones need explicit approval. At Brightwater that might mean reading documents automatically, requesting missing documents after a handler approves, and never deciding or settling a claim. Ask: which actions can it take alone, and which must it never take?
Treat outside text as untrusted. A claim form or broker email can contain instructions aimed at the AI. AWS recommends layered input checks and prompt-injection defences, and Amazon Bedrock Guardrails offers filters for prompt attacks, denied topics and sensitive information, plus checks that an answer is grounded in its source. Ask: what happens if a document tells the AI to ignore its rules?
Build the test set before the system. AWS asks for layered testing, performance tracked over time, and validation by subject-matter experts. Ask: who labelled the examples, how many are there, and what score is a pass?
Make every action traceable from day one. Reasoning steps and tool calls need a single record that answers questions about cost, speed, quality and security. Ask: if it makes a wrong decision on Tuesday, can we replay exactly what it did on Wednesday?
Treat cost as a design input. Token-based pricing combined with an agent’s repeated reasoning steps behaves differently from ordinary cloud spend. AWS points to right-sizing models, caching, attributing cost and governing it continuously. Ask: what does one processed case cost, and what is the ceiling per case?
Name the owner after the pilot. AWS advises regular reviews as a system changes, and covers how prompts and releases are managed. Ask: who is called when it goes wrong, and who signs off a change to its instructions?

One unknown number can flip the business case

Take Brightwater’s broker email extraction and enter these invented inputs: 600 emails a month, 10 minutes each by hand today, £28 an hour fully loaded, a £15,000 build, £400 a month to run, and value starting in month 3. The only thing that changes below is the number of minutes a person needs to check each AI draft.

Review minutes per item	Net saving a month	Payback	Return after 12 months	Return after 24 months
3	£1,560	12.6 months	+19%	+91%
5	£1,000	18.0 months	-15%	+37%
7	£440	37.1 months	-49%	-18%

Payback is counted from today, including the three months before value starts. Return is the saving over the period minus the build and running cost, as a share of that cost. The same project looks like a sound investment, a slow one or a loss depending on a number nobody can know before a pilot. So the pilot’s first job is to measure review time on real work, and the assessment should leave that figure as “To confirm” until it has.

How a structured assessment removes the chasing

The chasing goes away when the fact list becomes a form and the completed form becomes the report. Four things do most of the work:

One question bank. Every assessment asks the same things in the same words, with the answer scales, scoring rubric and thresholds in one place.
Pre-work that goes out ahead. Questions the customer can answer without a workshop are marked and sent a week early, so the session starts with the gaps instead of the basics.
Scoring, readiness and gaps computed, not argued. Fourteen criteria are scored from 1 to 5 across value, agentic fit and feasibility. Any use case with an agentic fit under 12 of 20 is marked as simpler GenAI. Readiness is scored from 0 to 4 and banded: below 2.0 is not ready, 2.0 to 2.9 is conditionally ready and 3.0 or more is ready. Three or more blockers means foundations first.
A business case that admits what is missing. The formulas run on the numbers entered, and an empty box shows “To confirm” instead of a made-up figure.

Here is what that produced for Brightwater, using illustrative scores:

Use case	Value /25	Agentic fit /20	Feasibility /25	Total /70	Verdict	Platform decision
Claims triage agent	21	16	14	51	Agent	Amazon Bedrock AgentCore
Broker email extraction	18	9	19	46	Simpler GenAI	Amazon Bedrock model call, no agent runtime
Policy wording search	15	8	18	41	Simpler GenAI	Amazon Quick

The platform decisions follow from three answers: who builds each use case, who uses it, and where it lives. Claims triage and broker extraction are built by developers and embedded in an admin tool, so they point to Amazon Bedrock AgentCore (AWS’s managed service for running agents) or, where no agent is needed, a plain model call. Policy wording search is set up by business users in a workspace tool, so it points to Amazon Quick, AWS’s no-code AI workspace for employees.

Claims triage ranks first and is the only genuine agent. It also has the lowest feasibility score, and the readiness answers show why: no scoped identity for an agent, no authenticated API into the claims system, no list of permitted document sources, no handling for untrusted input and no oversight tiers. That gives readiness of 1.9 of 4 and five blockers, so the report says to close the foundations first. Broker extraction needs no agent. It reaches 2.5, conditionally ready, with one blocker (agent identity, a question the tool asks once for the whole organisation), and the sensible recommendation is a small proof of concept there that builds the test set and review routine the claims agent will later need.

How to read the report

Treat the ranking as a starting point. Scores are judgements entered by people. Ask for the evidence behind any 5, and challenge the top two.
Read blockers before bands. A use case can have an attractive total and still be marked foundations first.
Check what is still unknown. Unanswered readiness questions are left out of the average rather than counted as zero, so a dimension can look healthier than the facts justify. They also create no gap-to-action lines. Read the list of what is unknown as carefully as the scores.
Ask who owns each gap. The gap list is only useful when every line has a name and a date.

A note for UK insurers

This is general information, not legal advice, and it reflects sources available in early November 2026. The FCA confirmed in June 2026 that it will not write new AI-specific rules and will rely on the Consumer Duty, the Senior Managers and Certification Regime and its existing expectations on governance and controls, which keeps accountability for the outcomes of an AI-supported process with the firm and its named senior managers. From 5 February 2026 the Data (Use and Access) Act 2025 replaced the old UK GDPR Article 22 with new rules that loosen the limits on solely automated decisions but keep safeguards, with tighter conditions where special category data such as health information is used. If you sell into the EU, the timetable for the EU AI Act’s high-risk rules has moved to 2 December 2027, and that category includes some insurance uses. Take advice before relying on any of this.

What to do next

An assessment does not collect the facts for you. Someone still has to ask for the baseline, the access details and the policies, and the scores are only as honest as the people who enter them. It cannot tell you how long a person takes to review an AI draft. It does not replace a security test or legal advice. Start like this:

Name the sponsor and who signs off a proof of concept. Send the pre-work questions a week before the session, with an executive named for each group of facts.
Run the session against the form, scoring as the customer talks, and leave unknowns empty so the report shows “To confirm”.
Give every gap an owner and a date. Decide what the AI may do alone, which outside content it may read, and the running-cost ceiling.
Run a pilot on real work with named reviewers. Measure review minutes and cost per case against the pass mark you set at the start.
Decide whether to scale, change scope or stop, using measured numbers.

[Add a closing call to action here: how readers can request an assessment or a sample report.]

Sources

AWS guidance

Regulation (law-firm and trade-press summaries; check the primary text)

Brightwater Mutual is fictional. Its inputs are invented, and the scores, readiness figures and business-case outputs were calculated by an assessment tool from them.

Latest Article

Before you fund an AI project: the facts your assessment must rest on