ProcurementSecurityRisk management
AI Vendor Evaluation Checklist
A procurement-ready checklist to vet AI vendors on privacy, safety, accessibility, and exit strategies before signing a contract.
Applies to: Procurement, IT/security, Legal, Program leadsPolicy owner: Procurement & IT/security leadLast updated: 2025-02-12
Baseline requirements
- Vendor must support data isolation for each tenant and confirm data is not used to train public models.
- Contract should include SLAs for uptime, incident response, and notification timelines.
- Accessibility conformance (WCAG 2.1 AA) is required for patron-facing interfaces.
Data handling & retention
- Where is data stored and processed? Can we select regional hosting?
- What is the default retention period for prompts, outputs, and analytics? Can we purge on demand?
- How is PII detected, redacted, and prevented from leaving the tenant boundary?
- Is data encrypted in transit and at rest? Which keys do we control?
Model behavior & safety
- Which base models or providers are used, and how are they patched or updated?
- Can we require citation-style responses and block high-risk topics (legal, medical, immigration)?
- What guardrails exist against bias, hallucinations, and prompt injection?
- How are jailbreak attempts detected and reported to administrators?
Access, auditing, & reporting
- What authentication options are supported (SAML, SCIM, MFA)?
- Can we segment access by role (staff, supervisors, volunteers) with different guardrails?
- Are administrator logs available for prompts, changes to guardrails, and data exports?
- How can we export audit logs for compliance reviews without exposing PII?
Support, onboarding, & training
- What onboarding services are included (template setup, policy tuning, staff training)?
- How are support tickets prioritized, and what are typical response times?
- Is there a named account owner or success manager for escalations?
Exit strategy & portability
- How do we export prompt libraries, knowledge bases, and configuration if we leave?
- What is the process and timeline for certifying data deletion after termination?
- Are there any fees tied to export, migration, or data destruction certificates?
Pricing & procurement fit
- Is pricing flat-rate, usage-based, or hybrid? How are overages handled?
- Can the vendor accommodate grant-funded pilots, co-op purchasing, or short-term trials?
- What is the minimum contract length, and are there opt-out or renewal notification clauses?
Want these customized?
We can adapt language and policies to your specific programs.