This is part four of a four-part series highlighting key learnings from the IAPP Privacy. Security. Risk. + AI Governance Global conference.

AI Governance: Managing Changes After Approval

AI governance was another major conference topic. Sessions addressing AI-enabled software and agentic AI highlighted how much can change after a company initially approves an application.

A vendor may substitute a new model, expand access to customer information, or enable features that weren't considered during the original review. Existing vendor agreements may not provide adequate notice or give the customer meaningful rights to assess those changes.

In-house counsel should pay particular attention to data use and model training restrictions, confidentiality, IP rights, and vendor obligations when AI functionality changes or an incident occurs.

Agentic AI adds another layer of risk. Agents may retrieve confidential documents, access business applications, or take actions on behalf of employees. A poorly configured agent can exceed its intended authority even without a traditional cybersecurity breach.

Legal teams should work with IT and security to establish boundaries around the information agents can access and the actions they're authorized to take. Higher-risk activities, particularly in HR and financial workflows, may warrant human approval before an agent acts.

Practical Takeaway

Review AI vendor contracts and establish a process to reassess material changes in approved systems. For agents, document access permissions, delegated authority, and approval requirements. Retain vendor change notices and the resulting review decisions.

AI Incidents: Planning Beyond Data Breaches

The session Harm or Hazard? When Your AI Event Becomes an AI Incident addressed another gap in many existing compliance programs. An AI agent might initiate an unauthorized transaction. An employment tool could produce discriminatory results. A customer-facing application might generate materially inaccurate information that causes harm.

These events don't necessarily constitute reportable data breaches, but they may create exposure under other laws or contractual obligations.

Incident response teams need a process to determine when an AI problem warrants legal escalation, what evidence to preserve, and whether the system should be suspended. Where a third-party model or application is involved, investigating the event may depend on contractual rights to obtain technical records and vendor cooperation.

Insurance deserves attention as well. Legal and risk management teams should review cyber and errors and omissions policies for AI-specific exclusions, limitations involving statutory privacy or wiretap claims, and the scope of regulatory proceeding coverage.

Practical Takeaway

Incorporate AI events into existing incident response procedures and test them through a tabletop exercise. Keep records of investigations, escalation decisions, vendor communications, and actions taken to contain or correct the problem.