By Derek Alston, Partner, True North Partners LLP

The use of AI models, including LLMs like Claude, ChatGPT and Copilot, is becoming commonplace in our personal and professional lives. These models are taking on increasingly complex tasks and, as they do, they're building trust with their users.

As that use and dependence grows, particularly within financial services, the need to bring these models into the model inventory and subject them to model risk practices is becoming increasingly urgent.

The core principles of model risk management aren't complex, but they matter: identifying a model (identification), knowing who owns it (governance), where and how it can be used (use), confirming it's been reviewed by an independent expert (validation) and ensuring it continues working as intended (monitoring).

These principles still apply to AI models, but in many cases the complexity increases. This is partly because of the nature of the models themselves, partly because they're often provided by third parties with limited documentation and controls and partly because their broad use makes them harder to govern.

Given the increasing complexity, these principles must remain a focus. One area that gets too little attention, however, is AI, including LLM, model monitoring. Yet, this is arguably the most important principle, and so we must address it. To succeed in this area, three primary pillars should be covered:

1.    Stability. Responses shift between versions of publicly available models. Likewise, retraining internal AI models on new data materially changes their output. Confirming that responses remain appropriate across model versions is a standard step in a traditional model rollout, but it's routinely skipped for AI models, especially public LLMs

2.    Accuracy. This is likely the most critical of the three. Answering "can I trust this model for its intended purpose?" requires a defined scope that isn't breached and responses tested against expected outcomes

3.    Cost. Token usage with third-party LLM and AI providers will become a material cost for financial institutions. At the enterprise level, where security and custodianship of data is required, the price of these models scales exponentially from the heavily subsidised consumer prices.  These costs add up, and providers are incentivised to drive greater engagement with their models. Monitoring prompts and the models’ outputs through a lens of response quality is worthwhile.

Across all three dimensions, the real challenges are data quality and choosing the right metrics to report. The good news: many of the best tools and frameworks for this already exist which is the ones banks and financial institutions already use.

Automating data gathering, calculation and reporting is what separates a strong monitoring practice from a weak one. Good monitoring, in turn, reinforces the rest of the framework: governance (teams are more comfortable owning something they can show is working), validation (data and metrics don't need to be sourced independently) and use (confirming models operate within their intended boundaries).

This space is still evolving, but the case for building a strong monitoring framework sooner, rather than later, and then embedding monitoring as a key pillar as part of the implementation plan for every new model feels like a wise strategy. While models are few and use cases are still emerging, AI model monitoring can be adapted and evolved over time. Doing so positions us well for the growth we anticipate in the AI model race.

Find out more: Visit True North Partners LLP