Rail Governance

AI Vendor Due Diligence Questionnaire Criteria for Banks

A bank's AI vendor questionnaire must probe model behavior, not just security.

· 9 min read
Cover illustration for “AI Vendor Due Diligence Questionnaire Criteria for Banks”
Vendor AI Risk Evaluation · November 12, 2026 · 9 min read · 2,016 words

AI vendor due diligence for banks needs criteria built around model behavior alongside software security. A sound questionnaire must establish how a vendor controls drift, hallucination, bias, data access, and autonomous action, then turn those answers into contract terms and ongoing evidence. Outsourcing the model does not outsource the bank’s responsibility.

Why standard TPRM questionnaires fail when the vendor's core product is an AI model

A compliance team receives a partner questionnaire asking, for the first time, how it assesses its AI vendors. The team already has a third-party risk management program that covers uptime, security, data handling, and business continuity. Those controls remain necessary, but they answer a different question. They assess whether the service operates securely and reliably.

The gap is structural. Model outputs depend on training data, parameter choices, bias characteristics, and behavioral stability. A bank could have a mature 300-vendor program without a questionnaire that mentions hallucination. Before assessing those risks, it must also discover where AI is being used, because vendors may not disclose it unless asked directly.

The regulatory moment that turned this gap into an institutional obligation

The Federal Reserve, OCC, and FDIC issued SR 26-2 on April 17, 2026, replacing SR 11-7 with revised interagency model risk management guidance. Section VII applies model risk principles even when a bank cannot get a vendor’s code, data, or methodology. It says vendor validation, ongoing monitoring, and outcomes analysis are sound practice.

The scope needs equally careful attention, however. OCC Bulletin 2026-13 states that generative and agentic AI are “not within the scope of this guidance” because they are novel and rapidly evolving. SR 26-2 also “does not set forth enforceable standards or prescriptive requirements” and is expected to be most relevant to banks with over $30 billion in total assets. A vendor’s claim of compliance therefore cannot settle the assessment, especially for the categories the guidance excludes.

The U.S. Department of the Treasury released the Financial Services AI Risk Management Framework on February 19, 2026. Developed by the Cyber Risk Institute with the FSSCC, it adapts the NIST framework to financial services through 72 subcategories and 230 control objectives. It is non-binding, but its evidence-based approach includes independent testing, bias audits, hallucination measurement, and security testing. A completed questionnaire alone is not enough.

OCC examiners are also probing whether models access unauthorized data and whether controls operate at both the model layer and the database layer. NCUA has no AI-specific rules, but examines AI through existing BSA/AML, fair lending, and third-party management frameworks. These developments make institutional judgment and evidence essential, so a bank’s framework must address actual use, model behavior, and enforceable oversight.

Tiering AI vendors before writing a single question

“In which parts of the service you provide us does AI contribute to outputs or decisions?” The answer establishes the service’s exposure before the bank assigns review depth. The 2023 interagency third-party guidance expects more rigorous oversight where failure could create significant risk, customer impacts, or effects on the bank’s financial condition.

Tier 1 covers critical AI vendors affecting credit decisions, fraud determinations, compliance screening, customer communications about financial products, or regulatory reporting. These relationships warrant documentation review, independent testing where feasible, and enhanced contracts. A service that reads customer data or drafts material for customers is more likely to belong in this tier. SR 26-2 validation principles apply where the models fall within its scope.

Tier 2 covers uses that matter but are not critical to decisions, including document review, risk monitoring, and workflow automation that involves customer data or regulated activities. These warrant standard AI questionnaire coverage, annual review, and AI contract provisions. Tier 3 covers incidental features such as search, auto-fill, and background analytics. Contracts should require disclosure and notification, with quarterly scanning for expanded AI use that warrants a higher tier.

The six AI-specific due diligence domains a questionnaire must cover

The first domain is model inventory and documentation. Vendors should name every model that contributes to the service, including foundation models, fine-tuned layers, and RAG knowledge bases. Model cards or equivalent records should describe intended use, training data, performance, and known limitations. Missing documentation is itself a governance signal. Without it, the bank cannot establish what it is assessing.

The second domain is training data sourcing and bias controls. The assessment should establish which datasets trained or fine-tuned the model, how they were screened for bias across relevant protected characteristics, and whether demographic impact analyses precede updates. The FS AI RMF treats bias audits as an expectation. Institution data needs a separate, explicit boundary. Its use to train or fine-tune the vendor’s general model must be prohibited by contract.

The third domain is drift monitoring and model change notification. Criteria should cover how behavioral drift is detected, what triggers a retraining review, how new versions are announced in advance, and whether the bank can pin a version. Output stability is only part of the question. Changes in production data access also need scrutiny because examiners ask what the model actually accesses.

The fourth domain is hallucination measurement and deterministic fallback. For generative features, vendors should provide measured hallucination rates and the conditions under which they were measured. Compliance decisions need deterministic logic rather than reliance solely on probabilistic generation. Neurosymbolic design separates these roles. A neural component interprets messy documents, while a symbolic component executes deterministic rules, preventing a hallucinated compliance pass through that rules-based decision path. The OCC identifies lack of explainability as a distinct risk, making the reasoning chain part of the evidence.

The fifth domain is incident notification when output is harmful, bias is detected, a model fails, or data boundary breaches occur. The bank needs defined triggers, timelines, and accountable people, with the notification window written into the contract. The sixth domain is the right to audit how well the model performs. It should set the terms for independent testing and access to AI-related records, including what the vendor commits to provide when regulators ask. Access cannot remain an unspecified support process.

Fourth-party AI risk: when your vendor's vendor is the model

A service provider may embed another company’s language model, classifier, or AI API without proactively disclosing it. That creates a fourth-party dependency subject to oversight. The questionnaire must ask which foundation models and fine-tuning providers the service uses and where its RAG knowledge bases come from. If the bank does not know the upstream provider, how can it demonstrate what data the model accesses in production?

The 2023 guidance explicitly addresses subcontractor selection, oversight, and concentration risk from dependence on a single provider. Fannie Mae LL-2026-04 requires lenders to manage subcontractor and vendor AI/ML risks to a standard no less protective than their internal controls. It does not contain an explicit indemnification clause. Identity needs scrutiny at this layer too. The questionnaire should establish what identity context reaches upstream providers, rather than accepting a single service account as an adequate account of who authorized access.

The contract provisions that protect a bank when a vendor's model fails

For Tier 1 vendors, legal teams should treat eight provisions as non-negotiable minimum requirements. Performance terms should define accuracy and error-rate thresholds that trigger remediation. Bias testing terms should specify the frequency, scope, and reporting of demographic impact analyses. Incident terms should set a notification window for when harmful outputs occur, models fail, or bias is found. These commitments need to survive procurement approval.

Audit rights should allow independent performance testing, set out what notice is required, and let the bank hire outside validators. Change provisions should require advance notice with a stated lead time and version-pinning rights where stability is needed during examinations. Data restrictions should prohibit training or fine-tuning the vendor’s general model with institution data. Sub-processors must be bound to that restriction too.

Log provisions should set an institution-defined retention period and require export to the bank’s SIEM in a format it controls, without mediation through vendor support. Termination provisions should require data export followed by deletion within a stated period, including sub-processor copies. Written confirmation is necessary. The contract turns due diligence into due care by binding the vendor to the standards its answers describe.

Agentic AI demands a separate authorization layer in the questionnaire

Agentic AI changes the object of the assessment. The bank is no longer assessing only an output, but authority to take multiple actions. The bank must be able to trace that authority to a human decision-maker, even though SR 26-2’s scope leaves out agentic models. Banks must govern both their own delegated agents and the risks of customers using external agents to interact with banking services.

How are multi-step tool use and autonomous workflows authorized? Which actions require approval, how quickly can access be revoked, and can every action be traced to its human initiator? Strong answers require approval for writes and external sends, revocation in minutes, and action logs containing initiating user identity. The authorization model must be configurable and human-governed.

The IMF’s 2026 note on agentic AI in payments identifies authorization traceability, opacity, correlated behavior, cybersecurity, and unresolved liability questions as core risks. It also identifies an unresolved Regulation E question about whether granting an agent bank account credentials satisfies authorization requirements. For payment execution, transfer authorization, and account analysis, the vendor must support institutional governance and a complete action trail.

Near-standard instruments that exist and what they actually cover

No single AI vendor questionnaire is mandatory. Available instruments offer useful starting points, but none fully resolves the generative and agentic coverage gap. CSA AI Controls Matrix v1.1 and AI-CAIQ, which were released June 22, 2026, have 247 control objectives across 18 domains. The question set maps to the matrix, and vendors can publish completed responses in CSA’s STAR Registry. That gives banks a public artifact to examine, with the effectiveness of relevant controls still to be established.

The KY3P Best Practices Questionnaire has a version used by at least one bank with 51 AI-specific questions. ISO/IEC 42001:2023 sets out what an AI management system must have. Its certification scope determines what a vendor’s claim proves. A narrow certification does not establish coverage of the entire service.

NIST AI RMF 1.0 and the July 2024 NIST AI 600-1 Generative AI Profile provide another basis for gap analysis. The profile covers how to assess generative AI vendors and set contract terms, including who is responsible for incidents and who must be notified. A practical approach is to use AI-CAIQ as the baseline evidence request, supplement it with the six domains, and require authorization questions for vendors offering multi-step or autonomous workflows.

Turning the questionnaire into an ongoing monitoring program, not a one-time gate

The interagency third-party guidance spans planning, due diligence and selection, contract negotiation, ongoing monitoring, and termination. Oversight and accountability, independent reviews, and documentation and reporting run across that life cycle. SR 26-2 also calls for outcomes analysis of vendor model performance within its scope. Initial approval is only the starting point.

Contracts should require a new review when a model changes without advance notice, even if the change is a minor update. A new sub-processor or underlying model provider should also prompt review, as should an AI incident even if it did not affect the bank’s instance. The bank should also review material changes in how AI is used or in the scope of data processing. The assessment must follow the service as it changes.

Runtime evidence connects that review to actual behavior. Per-decision records should show who initiated a request, what data and policy applied, and what outcome occurred. Where the bank controls the AI request boundary, an identity-bound enforcement layer can produce this evidence. Where the vendor controls it, accessible audit records must be contractual. Agentic monitoring should also test revocation against the promised time frame and examine logs for actions without a traceable human initiator.

Institutional accountability provides the foundation for responsible AI deployment. Banks that build evidence, authorization, and ongoing review into vendor relationships can move faster than peers that must reconstruct those controls after procurement. The questionnaire becomes useful not as a finished form, but as the beginning of a governed relationship.

Sources

  1. What a Bank's Third-Party Risk Review Asks an AI Vendor

More in Vendor AI Risk Evaluation