Pim Poppe: AI in risk management

December 2, 2025

_{This column was originally written in Dutch. This is an English translation.}

By Pim Poppe, Managing Partner at Probability & Partners

On 20 November, I had the privilege of chairing the seminar “Artificial Intelligence, Blockchain, Crypto & Digital Assets” for Financial Investigator. In my introductory talk, I gave a brief overview of how AI has changed Probability & Partners over the past eighteen months. When you make a list like that, you realise how quickly the world is changing and how the work of a risk management consultancy firm is adapting to the possibilities and needs of today.

In March 2024, we began to systematically adapt our business operations to the new possibilities offered by large language models. That was quite recently. This adaptation process involved trial and error, experimenting with use cases in a controlled environment, sharing successes and failures, and sharing tips and tricks. Some tasks are going great, while other applications that you had high expectations for somehow aren't working out. Sometimes the AI solutions require so much manual rework that the AI solution is not worthwhile. We have learned from our failures and embedded our successes into our business operations.

Without wanting to be completely or truly accurate, I will give you the voice-over for the slide I used (see Figure 1).

Figure 1:How is AI changing Probability's consulting business?

Advantages of AI for existing client work

We create risk models, and it takes a lot of programming work to build working prototypes or production software. With AI, this is now many times faster. Depending on the problem, it can save 50% to 80% in time. More time savings if the mathematical challenges are significant, less time savings if there are many interfaces. What the modellers/programmers have also found is that thinking ahead and instructing the LLM properly yields enormous benefits. In other words, you need to have a better overview at a higher level of abstraction in order to reap the AI efficiency gains.

Another application is generating documentation based on existing code. This is actually the wrong way around, but good documentation is useful to have for internal use, and regulators consider documentation important. This is also an area of expertise that requires some practice. Some high-level instructions are needed to accompany the code. Otherwise, you will not get the right documentation. Here, too, overview and understanding at a higher level of abstraction are very valuable and even necessary for the successful use of LLMs to generate the right documentation.

What also works well is converting regulations into policy. Here too, the trick is to create good prompts and provide structures for the policy. Uploading existing policy as an example for the new policy can also work. All in all, significant gains can be made in terms of costs and lead times in this area. It is important to keep a close eye on the source reference.

The list of aspects of client work that we now perform differently than before is long. We will not go through points 5 to 10 one by one. The trend is clear: the work can be done faster, often better, and more cheaply than before.

New client work through AI

In addition to the old client work, which is now being done faster and better, there is also new work. Work that did not exist before the LLM revolution. The first is the validation of numerical AI models (think of supervised and unsupervised machine learning models). These AI models face new and different challenges than the classic models, such as non-linearity, instability, undesirable and undetected biases, lack of intuitive comprehensibility, etc.

To mitigate these problems somewhat, a set of techniques and frameworks has emerged that fall under the umbrella of Explainable AI (XAI). Based on this, we have created dashboarding, which allows you to better understand (machine learning) models by providing insight into inputs and outputs, usually in graphical form. The great thing about this XAI toolkit is that you can visualise the operation of classic models alongside AI models in the same way, enabling comparisons between old and new models. Think, for example, of credit risk models for the likelihood of default or fraud.

Another new domain is helping to build tooling that can perform specific tasks for customers. Roughly speaking, three things are important here: first, teaching domain-specific knowledge; second, performing specific tasks efficiently; and third, ensuring a secure environment.

The latest theme in customer work driven by AI is of a completely different order. It concerns the enormous profit growth and PE expansion of AI-related shares. Is this a bubble? Will the investments in models, data, energy and water supply pay for themselves? Should something be done about the concentration in this sector? What are the scenarios for the coming years? Should the weightings in the portfolios of pension funds or insurers be capped in some way? Is market-weighted investing still wise in a potentially bubble-prone and geopolitically driven world? The question is how to invest in AI.

Business operations and customers

In short, the AI revolution is having a significant impact on our business operations. On the one hand, existing work is being done faster and, on the other hand, new work has been added. It is a fact that we employ proportionally fewer juniors. We use LLMs as juniors. Not very pleasant, but very efficient. What applies to juniors also applies to LLMs. Find the most suitable model or junior for the task, give clear instructions to the junior or LLM and monitor the junior or LLM closely.

Replacing juniors with LLMs is not something we are alone in doing. The declining number of vacancies for junior risk managers and consultants in the market is probably a reflection of the same phenomenon. A second important conclusion is that we have the same turnover with fewer people: machines are replacing humans. This is apparently also the case at our company. And the last – and, I believe, most important – conclusion is that customers are getting more value for their money. The help of AI in our work makes us faster with at least the same quality.

More columns by Pim Poppe