How do we govern AI models sourced from third-party vendors?

Many delegated authority organisations first encounter AI not through a project of their own, but through a vendor's product.

Bordereaux processing platforms, coverholder portals and MGA-provided systems increasingly use AI to extract, validate or transform data before it ever reaches an insurer's own systems.

That creates a governance question many frameworks were not built to answer: how do you oversee an AI model you did not design, cannot inspect, and do not control?

The organisation remains accountable to regulators, to Lloyd's and to its own oversight committees for the accuracy of the data flowing through these tools, regardless of who built the underlying model.

Closing that gap requires extending existing governance practices to explicitly cover vendor AI, rather than assuming internal policies already do so.

Why vendor AI creates a distinct governance challenge

Internally developed AI can be inspected. Governance teams can ask what data trained it, how it was tested, and how its outputs are validated, because the organisation controls those decisions directly.

Vendor-sourced AI is different. It typically arrives as a finished product, embedded within a bordereaux platform or coverholder system. The organisation using it usually cannot see the training data, the model architecture or the specific logic behind an output.

This lack of visibility does not reduce the organisation's responsibility. Regulators and Lloyd's oversight expectations focus on outcomes: the accuracy of bordereaux data, the integrity of reporting, and the quality of underwriting information. Whether an error originates from a spreadsheet macro, a vendor's AI model or a manual process makes little difference to that accountability.

Many internal AI governance frameworks were written with internally built models in mind. They assume the organisation controls the model lifecycle. When AI arrives embedded in a vendor tool, that assumption breaks down, and the framework often has nothing specific to say about it.

Traditional approaches to vendor oversight

DA organisations already have established ways of governing vendor relationships. Service level agreements set expectations for uptime, accuracy and support. Periodic audits and reviews check that a vendor continues to meet agreed standards. Onboarding due diligence assesses financial stability, security practices and operational resilience before a relationship begins.

These approaches remain valuable and should not be discarded. However, they were largely designed around traditional software, where behaviour is fixed unless a vendor deliberately changes the code, and changes are usually visible through version numbers or release notes.

AI-enabled tools behave differently. A vendor can update or retrain the underlying model in ways that change how the tool interprets data, without necessarily changing the visible interface or issuing a formal software release. Traditional vendor oversight, built around periodic checks and static service agreements, can miss this kind of change entirely unless it is explicitly built in.

Extending governance to cover vendor AI

Rather than creating a separate framework, most organisations are better served by extending what they already do, with AI-specific questions and assurances added at each stage.

At due diligence, this means asking vendors:

• What does the AI component actually do within the tool, in plain terms?
• At a high level, what kind of data was used to develop or train it?
• How are outputs checked for accuracy, and what error rates are considered normal?
• How are errors identified, reported and corrected when they occur?
• What happens when the model is updated or retrained?

Contractually, organisations should seek commitments that go beyond general service levels. A vendor should be required to notify the organisation of material changes to the AI model, not just to the software platform. Contracts should also clarify what evidence the vendor will provide to support ongoing oversight, such as periodic accuracy reporting or access to error logs.

The goal is not to turn every DA operations team into AI auditors. It is to ensure that the organisation has enough visibility to exercise proportionate oversight, calibrated to how material the tool's output is to underwriting, claims or regulatory reporting.

Ongoing monitoring and escalation

Approving a vendor's AI-enabled tool once is not sufficient. Because the model can change after approval, oversight needs to continue for as long as the tool is in use.

Practical, non-technical monitoring can include:

• Periodic sample checks comparing tool output against source bordereaux.
• Tracking exception rates and error patterns over time to spot unexpected shifts.
• Scheduled review meetings with the vendor to discuss performance and any planned changes.
• Clear internal ownership for who reviews vendor AI performance and how often.

When something goes wrong, whether a data quality issue traced back to the tool, or a vendor notification of a model change, there should be a defined escalation path. This typically includes assessing the scale of any affected data, informing relevant oversight committees, and agreeing corrective action with the vendor, including, where necessary, reverting to manual checks until confidence is restored.

Treating vendor AI governance as a continuous process, rather than a one-off approval, is what closes the gap between traditional vendor oversight and the realities of AI-enabled tools.

Example

A managing agent in London uses a bordereaux processing platform supplied by a third-party vendor to extract and validate data from a coverholder's monthly submissions.

The vendor recently updated the AI model underpinning its extraction engine, but did not proactively inform the managing agent. Data quality issues emerge that the oversight team traces back to the model update.

The managing agent works with the vendor to establish a formal notification process for future model changes and introduces a quarterly review of the platform's output accuracy, closing the gap between vendor updates and internal oversight visibility.

FAQs

• Can we simply rely on the vendor's own assurances about their AI model?

  Vendor assurances are a useful starting point, but they do not remove the organisation's own accountability for data accuracy and regulatory compliance. Independent verification, such as sample checks against source data, and ongoing monitoring remain necessary alongside whatever the vendor tells you.

• What should we ask a vendor before approving an AI-enabled tool?

  Ask what the AI component actually does, at a high level what data it was developed or trained on, how its outputs are checked for accuracy, how errors are identified and corrected, and how the vendor will notify you of future changes to the model. None of this requires technical AI expertise to ask or understand.

• Who is responsible if a vendor's AI model causes a bordereaux data error?

  Regulatory and contractual accountability typically remains with the insurer or managing agent, even where the technical fault originates with the vendor's AI model. This is why governance and ongoing monitoring of vendor AI tools are essential rather than optional.