Contracting to Address Agentic AI Governance

Background

Agentic AI has the ability to take actions that alter the business environment. In an enterprise setting, autonomy is the degree to which a company allows an AI agent to initiate or complete actions within its systems, with third-party systems, or with individuals. A single AI agent may have different autonomy levels for different tasks. The governance challenge posed by such autonomy affects virtually every company that contracts for agentic AI, because what must be governed is not merely what the AI agent does, but what the AI agent is authorized to do on the company’s behalf.

Why Agentic AI Contracts Matter Now

Evolving Regulatory Landscape

The AI regulatory landscape is evolving rapidly. In the United States, AI is currently regulated via a patchwork of sector-specific¹ and state laws, such as the Colorado Automated Decision-Making Technology Act (Colorado ADMT Act).² In the European Union, AI is regulated by the EU AI Act (Regulation (EU) 2024/1689).³ These regulations impose obligations based on risk and the AI system’s (including AI agent’s) potential to impact consequential areas including education, financing, lending, employment, insurance, healthcare, real estate, essential services, critical infrastructure, and regulated products.

Under comprehensive AI regulations such as the EU AI Act and Colorado ADMT Act, compliance obligations are divided based on whether the regulated entity is, among other things, a developer or deployer. For example, the developer may be responsible for training and testing the agentic AI system, ensuring its accuracy and robustness, maintaining technical documentation and logs, and providing the deployer with appropriate instructions for use and system-related information. Deployers, on the other hand, may be responsible for deciding how to use the agentic AI, providing transparency and explainability notices to end users, adhering to the developer’s instructions of use and documentation, managing the input prompts for the agentic AI, and handling end-user rights requests under data privacy and AI regulations. Robust contracts often clearly allocate which party bears which obligation and provide mechanisms for the parties to support each other’s compliance.

Along with these laws, some countries have developed or are developing specific agentic AI governance frameworks and standards, such as Singapore’s IMDA Model AI Governance Framework for Agentic AI⁴ and the United States’ AI Agent Standards Initiative launched in February 2026 by the National Institute of Standards and Technology.⁵

Given the speed of agentic AI development, legal and regulatory obligations that apply to AI systems are constantly shifting and could vary by geography, industry, or other characteristics. As a result, robust contracts are often drafted to accommodate this regulatory flux, by having mechanisms in place to adjust the parties’ respective obligations as new requirements emerge and through requirements addressing compliance with laws.

Security Risks Unique to Agentic AI

Agentic AI systems face unique risks including indirect prompt injection, agent hijacking, unauthorized tool use, and specification gaming. Because agents take action and do not merely produce outputs, a successful attack can result in unauthorized transactions, data exfiltration, or communications sent on behalf of the enterprise. Companies can take steps to protect against these significant security risks, including through contractual provisions requiring specific technical controls, incident response obligations, and liability allocation based on autonomous agent behavior.

Compliance Controls in Contracts for Agentic AI

Companies can address the unique risks present in agentic AI by incorporating compliance controls into their agreements with providers. This includes provisions for:

Party-role obligations under applicable laws

• Data governance
• Transparency
• Human oversight
• Technical controls
• Liability
• Other compliance controls

Alignment with Party Roles under Applicable Laws

If a company is entering into an agentic AI licensing contract, it can benefit from evaluating its party role (developer or deployer) and the parties’ intended usage of the AI system. Such a contract may include terms requiring each party to manage compliance and risks based on their party-role obligations and whether the agentic AI is intended to be used in consequential decision-making. Companies can review deployed AI agents periodically—and in any event when material changes occur—to ensure alignment with legal obligations.

Data Governance

Companies deploying agentic AI can consider applying the rule of least privilege contractually and internally so that agents operating autonomously cannot access systems with sensitive data and trade secrets, reducing the risk of security incidents and data privacy violations. Under a robust agreement, agents can receive only necessary access, applying least-privilege, role-based access controls, just-in-time permissions, and tool allowlists.

Transparency

Companies can contract for transparency, or the ability to implement transparency, as determined by the companies, including to foster trust with users and facilitate proper use. For example, transparency could include:

• At any interaction point, the agent discloses in the user interface that the user is interacting with an AI agent.
• If the user’s information will be available to the AI provider (in addition to the company), disclosing the extent of the AI provider’s data rights. 
• Terms of use and acceptable-use policies (a) flow down any obligations from the contracts with agentic AI providers and (b) inform users of the agent’s range of actions; data collection, storage, and use; and the user’s responsibilities, such as verifying information the agent provides.
• Identifying responsible human-contact points at the company whom users can alert if the agent malfunctions or if they are dissatisfied with a decision.

Human Oversight

Contractual controls can allow the company to exercise human oversight of the agentic AI or set autonomy levels reflecting legal effect, reversibility, data sensitivity, system criticality, and reputational impact. Human oversight thresholds would benefit from being specific to the workflow level and use case. An AI agent’s autonomy level might allow it to operate without prior approval, subject to post-execution sampling, under human supervision with override capability, or require affirmative approval before execution.

Technical Controls

Companies can contractually require technical controls to limit an agent’s reach by using, for example, sandboxing, staged deployment, read-only defaults, transaction caps, rate limits, and rollback functionality.

Companies can also contractually require technical controls to allow monitoring of the agent’s actions, such as tool calls, approvals, policy exceptions, outputs, and security events, with auditable records and logs that allow for reconstruction of agent actions and verification of whether the agent acted within its authorized action profile.

The technical controls may include the agent’s authorized action profile, including the agent’s business purpose, accessible systems and data sources, actions permitted without prior approval, actions requiring human approval, and prohibited actions.

Prohibited actions should be defined with specificity, covering consequential decisions, access to sensitive databases, data exfiltration, production-code changes, and any action violating applicable law or company policy.

Liability Allocation

Because agentic AI can take autonomous actions that affect third parties, enterprise systems, and business relationships, contracts can benefit from clearly allocating liability for AI agent behavior that exceeds contractually authorized boundaries. A typical limitation of liability provision may already cover unintended consequences of an agent’s unauthorized actions, such as initiating a transaction, disclosing confidential information, or taking action that violates applicable law. However, companies can take prudent steps to confirm that such actions would constitute a breach under the agreement and that the resulting harm falls within the scope of the agreement’s liability provisions. The agreement may also address, for example, which party bears responsibility when an agent acts within its authorized boundaries but causes unintended consequences versus when it acts outside its authorized boundaries.

Other Compliance Controls

As with any area of high regulatory risk, companies can establish broader compliance requirements both in their agentic AI contracts and in their internal AI governance approaches.

While a company can seek to contractually require that agents comply (or can be configured by the company to comply) with the company’s written policies and procedures intended to bind agent behavior, define permissions and authorizations, and constrain agentic workflows to defined steps, leading providers may not agree to this commitment today. Where a provider does not commit, the company can confirm if agents can be configured for compliance and document gaps with its AI governance program.

A recent US state AI law may provide companies with a defense to AI enforcement actions when their own testing or feedback surfaces the violation first. Specifically, the Texas Responsible Artificial Intelligence Governance Act, which became effective earlier this year, provides that a defendant may not be found liable if it discovers a violation through feedback, testing (including adversarial testing or red-team testing) or, if the defendant substantially complies with a recognized AI risk-management framework, an internal review process.⁶ Companies can benefit from conducting periodic reviews and audits of agentic AI deployments, including the effectiveness of human oversight to guard against automation bias and confirm that agents continue to operate within authorized boundaries.

Takeaways

As agentic AI capabilities evolve and regulatory frameworks mature, the contractual provisions governing these systems will become increasingly critical to effective enterprise risk management. Companies that proactively incorporate robust compliance controls will be better positioned to harness the benefits of agentic AI while mitigating its unique risks. By treating contracts as dynamic governance instruments that adapt to regulatory change and technological advancement, companies can build the foundation for responsible and sustainable agentic AI deployment.

 

---

 

¹ See, e.g., Colo. Rev. Stat. § 10-3-1104.9.

² See, e.g., Act of May 14, 2026, ch. 131, 2026 Colo. Sess. Laws 569, 569–85 (to be codified at Colo. Rev. Stat. §§ 6-1-1701 to 1709); Texas Responsible Artificial Intelligence Governance Act (TRAIGA), Tex. Bus. & Com. Code § 552.051 (2025).

³ Regulation 2024/1689, of the European Parliament and of the Council of June 13, 2024 Laying Down Harmonised Rules on Artificial Intelligence 2024 O.J. (L 2024/1689).

⁴ Infocomm Media Dev. Auth., Model AI Governance Framework for Agentic AI 13 (Version 1.5, May 20, 2026) (organizing the framework around four dimensions: risk-bounding, meaningful human accountability, technical controls and processes, and end-user responsibility).

⁵ Nat’l Inst. of Standards & Tech., AI Agent Standards Initiative (Feb. 17, 2026), perma.cc/LZU9-2BCN (stating that the initiative is intended to support trusted, interoperable, and secure agentic AI adoption).

⁶ See, e.g., Tex. Bus. & Com. Code § 552.105(e).