By Daniel Sawyer and Rebecca Firth, KMPG, Risk and Regulatory Advisory

Imagine navigating a complex journey without a reliable map or building a house without regularly checking its foundations. For building societies, ensuring positive member experiences can sometimes feel just as intricate. With the advent of Consumer Duty, simply intending to deliver good outcomes is no longer enough; firms must actively evidence them. 

This is where Customer Outcomes Testing (COT) steps in – not as a burden, but as valuable insight to understand, measure, and ultimately enhance members' journeys.

Beyond traditional testing: What is COT?

Many societies are familiar with Quality Control (QC) and Quality Assurance (QA). While valuable, COT offers a distinct and crucial perspective.

 

QC	Typically involves reviewing specific, point-in-time colleague activity against policy and procedure.  Its objective is to assess performance and competence, identifying training needs.
QA	Focuses on the effectiveness of processes and controls. Ensuring they operate effectively against policy and procedure.
COT	Takes a broader, more holistic view.  It involves a regular review of discrete end-to-end customer journeys or product journeys over a period of time.

COT is not merely focused on policy/process adherence, point-in-time interactions, or individual agent performance. Its purpose is to identify, manage, and measure good customer outcomes, prevent foreseeable harm, and allow management to promptly address issues.

COT should be designed to:

• Observe the 'end-to-end' customer's journey, not just isolated 'parts'.
• Holistically review the customer's experience, considering whether it leads to a good or poor outcome.
• Identify patterns and themes that contribute to poor outcomes and foresee potential harm.

Three lines of defence: A collaborative approach

An effective COT Framework cuts across the three lines of defence, with defined roles and responsibilities:

• First Line (1LOD): Owns customer outcomes risks and is responsible for conducting outcomes testing.  Often performed by an ‘independent’ or ‘1.5 Line’ team.
• Second Line Compliance (2LOD): Provides independent oversight, advice, and challenge to COT activities, and conducts monitoring through a risk-based approach.
• Third Line Internal Audit (3LOD): Offers independent assurance on the design and operating effectiveness of the COT framework and its components.

By embracing a comprehensive and collaborative approach to COT, societies can not only meet regulatory expectations but also genuinely enhance customer trust and deliver fair value.

Key considerations for a COT Framework

A robust COT framework requires careful consideration across several components:

1. Scope: Clearly define which products, customer journeys, and service areas are in scope, and how this aligns with existing testing.
2. Roles and Responsibilities: Determine where the COT function will sit within the organisation, how it interacts with other departments, and ensure sufficient resources and capacity.
3. Defining Good Outcomes: Define what a good outcome looks like for a customer going through the journey and set good outcome standards.
4. Policies, Methodologies, and Checklists: Establish a single, consistent approach to COT, including detailed sampling methodologies and checklists aligned with customer outcome definitions.
5. Feedback Mechanisms: Implement effective feedback loops and root cause analysis to track issues to closure and ensure consistent reporting at appropriate levels of granularity.

By embracing COT not as a mere compliance exercise, but as a holistic, strategically targeted, and collaboratively managed framework across all lines of defence, societies can confidently demonstrate their commitment to delivering good outcomes, fostering deeper member trust, and securing a resilient future in an ever-evolving regulatory landscape.

Find out more: Read more about the challenges facing firms the actions to take and how we can help on KPMG’s website.

This article was first published in the Winter edition of Society Matters magazine (December 2025)