🏛️ Issues and Advocacy Training™
Courses › Large Organization Security Programs, Governance and Compliance™

Lesson 13 of 14 · about 60 minutes · Updated November 15, 2026

AI Governance and Security in a Large Organization

You'll be able to

  • Build and maintain an AI inventory and sort AI uses into risk tiers using the OMB high-impact test and the EU AI Act high-risk categories.
  • Explain the NIST AI RMF functions, the 12 risks in NIST AI 600-1, the draft Cyber AI Profile and what ISO/IEC 42001 certification does and does not prove.
  • Describe the main security risks of AI systems, including prompt injection, data leakage, supply chain and excessive agency, and the management controls that contain them.
  • State the EU AI Act timeline after the 2026 AI Omnibus and the status of U.S. federal policy and the Colorado, California, Texas, Utah, Illinois, New York City and Idaho laws as of November 2026.
  • Write a generative AI acceptable use policy and AI vendor contract terms suited to a large organization.
  • Design board reporting and incident response for AI, drawing on the Mata, Air Canada, Rite Aid, SEC AI washing and Texas Pieces cases.

Start here: the short version

  • Start with an inventory. You cannot govern AI you do not know about. Federal agencies must list every AI use case each year (3,611 of them across 56 agencies in the 2025 count), Idaho's state IT office requires agencies to register every AI tool, and almost every private framework begins in the same place.123
  • Use the free federal frameworks as the spine. The NIST AI Risk Management Framework 1.0 (Jan. 26, 2023) and its Generative AI Profile, NIST AI 600-1 (July 26, 2024), give a vocabulary boards, auditors and regulators recognize. ISO/IEC 42001:2023 turns the same ideas into a management system an outside body can certify.456
  • The rules moved in 2025 and 2026. The EU delayed its high-risk AI duties to Dec. 2, 2027 (and Aug. 2, 2028 for AI built into regulated products) in a law that took effect July 27, 2026. Colorado replaced its 2024 AI Act with a narrower law effective Jan. 1, 2027. Washington revoked the 2023 AI executive order, issued new agency rules in April 2025 and told the Justice Department to challenge state AI laws, but no federal statute preempts them.789
  • AI brings new security failure modes. Prompt injection, leaks of sensitive data, poisoned models and data, and AI agents with too much authority top the OWASP list for 2025. Most of the defenses are management decisions: what data the tool may see, what it may do without a person's approval, and who checks the output.10
  • The cases so far are about ordinary duties. Regulators and courts have punished false claims about AI, unverified AI output, chatbots that gave customers wrong answers and AI tools deployed without testing. None of those required a new AI law.111213
  • Buy carefully. Most large organizations get most of their AI from vendors. The contract decides whether your data trains someone else's model, whether you hear about a model change, and whether you can leave. The federal government's April 2025 buying rules are a usable model.14
  • The board needs a short, regular AI report: the inventory, the high-risk uses, incidents, vendor exposure, legal changes and any claims the company makes in public about its AI.

Rules and standards described: NIST AI Risk Management Framework 1.0 (NIST AI 100-1, Jan. 26, 2023), NIST AI 600-1 Generative AI Profile (July 26, 2024), NIST AI 100-2 E2025 (March 24, 2025) and the initial preliminary draft of NIST IR 8596, the Cyber AI Profile (Dec. 16, 2025); ISO/IEC 42001:2023 (described, not reproduced); OWASP Top 10 for LLM Applications 2025; the EU Artificial Intelligence Act, Regulation (EU) 2024/1689, as amended by the AI Omnibus in force July 27, 2026; Executive Orders 14148, 14179 and 14365; OMB Memoranda M-25-21 and M-25-22 (April 3, 2025); America's AI Action Plan (July 2025); Colorado SB 26-189 (2026); California SB 53 (2025); Texas HB 149 (2025); the Utah Artificial Intelligence Policy Act as amended in 2025; Idaho Senate Bills 1227 and 1297 (2026) and the Idaho Office of Information Technology Services AI Governance Policy, Standard and Guideline, Version 1.0 (August 2025); all read Nov. 15, 2026. AI rules are changing faster than any other area in this module. Check the regulator's current text before relying on any point.

Before you change anything. Plan changes to production systems, test them first, keep a way back, and follow your change management process. For clinical, plant-floor or other safety systems, involve the vendor, your safety lead and qualified engineers (see modules 109 and 111).

The vice president of customer service at a regional insurer wants to put a generative AI assistant on the company website by spring. Marketing already pays for an AI writing tool on a corporate card. The claims unit is testing a vendor's model that scores claims for possible fraud. Engineering has connected an AI coding assistant to the company's source code repository. Human resources bought an applicant screening product whose sales deck says it is "AI-powered." Nobody has a list of these, and the board's risk committee has just asked the CISO, the chief privacy officer and the general counsel's office a simple question: what AI do we use, what could go wrong with it, and who is in charge?

That question is the subject of this lesson. AI governance is the set of decisions, roles, rules and records that let a large organization use AI on purpose rather than by accident: knowing what it uses, deciding what it will not do, testing what it does, buying it under terms it can live with, watching it once it runs, and being able to explain all of that to a regulator, a court, a customer or a reporter. AI security is the part of that work that protects AI systems from misuse and attack, and protects the organization from AI used against it. Both belong in the security program this module describes, not in a separate silo, because they use the same machinery: inventories, risk assessment, policy, vendor management, testing, incident response, audit and board reporting.

This lesson is written for executives and board members, audit and risk committee members, the general counsel's office and compliance officers, CISOs and their teams, chief privacy officers and new chief AI officers, internal auditors, procurement staff, public officials, and the reporters, advocates and legislators who ask them questions. It stays at the level of governance and defense. It describes attacks on AI systems only as far as a manager needs to understand why a control exists.

It builds on several earlier lessons and does not repeat them. The AI rules in privacy law, including California's automated decisionmaking rules and Colorado's 2026 reset, are summarized in Module 108, Lesson 12. AI in legal and accounting work, including the duty to check AI output before relying on it, is in Module 110, Lesson 2. AI vendors that handle health information are in Module 109, Lesson 6. The small business version of AI governance is in Module 73, Lesson 13. Inside this module, the policy machinery is in Lesson 4, board reporting in Lesson 3, privacy assessments in Lesson 10, ISO certification (including ISO/IEC 42001) in Lesson 6, and the maturity checklist that includes AI items in Lesson 14.

Why AI governance belongs in the security program

What counts as AI

The definitions matter because they decide what goes on the inventory. Federal law defines artificial intelligence as "a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments" (15 U.S.C. 9401(3)), and the 2025 executive order on AI adopts that definition.15 The EU AI Act uses a similar idea: a machine-based system designed to operate with some autonomy that infers from its inputs how to generate outputs such as predictions, content, recommendations or decisions.16 Both definitions reach far beyond chatbots. A credit scoring model built in 2012, a résumé screener, a fraud score, a demand forecast and a camera system that matches faces are all AI under these definitions.

Inside that broad category, four kinds of systems raise the questions this lesson covers:

  • Predictive or "traditional" machine learning models, which score, classify or forecast: fraud scores, credit decisions, readmission risk, insurance pricing, spam filters. Banks have governed these as "models" for years.
  • Generative AI, which produces text, images, audio, video or code in response to a prompt. Large language models (LLMs) are the best known: they predict the next piece of text based on patterns learned from very large amounts of training data, which is why they can write fluently and also state false things with confidence.
  • AI features inside products you already own. Email, office suites, customer relationship systems, help desks, electronic health records and security tools now ship AI features, often turned on by a vendor update rather than by a purchase decision.
  • AI agents: systems that use a model to plan and take actions (send email, change records, run code, buy things, call other software) with limited human involvement. Agents turn a wrong answer into a wrong action, which is why they get their own section below.

Where AI enters a large organization

In practice AI arrives through five doors, and a governance program has to watch all of them:

  1. Enterprise purchases of AI products and AI-heavy services, through procurement.
  2. Vendor updates that add AI to software already under contract, often with new data flows the original security review never saw.
  3. Internal builds: data science teams training models, and software teams building applications on a commercial or open model through an application programming interface (API).
  4. Individual use of consumer AI tools on personal or corporate accounts, often called "shadow AI." It is the same problem as shadow IT, with the added risk that what employees paste into the tool may be kept or used for training.
  5. Third parties using AI on your behalf: the outsourced call center, the claims administrator, the marketing agency, the law firm, the payroll processor.

Who owns AI risk

No single office has the skills to govern AI alone. The model that has emerged in large organizations is a cross-functional committee with a named executive owner, and it looks a lot like the privacy and security governance structures in Lesson 1:

RoleWhat it brings to AI governance
Board or a named committeeOversight of AI as a strategic and risk matter; approval of the top-level AI policy and risk appetite; questions about disclosures and incidents.
Chief AI officer or executive sponsorOwns the AI program, the inventory and the review process. Federal agencies must name one (OMB M-25-21); many companies have created the title.1
CISOSecurity of AI systems and the data they touch; AI in incident response; vendor security reviews; monitoring of use; defense against AI-enabled attacks.
Chief privacy officerPersonal data in training and prompts; privacy and data protection impact assessments; automated decisionmaking notices and opt-outs (see Lesson 10).
General counsel's office and complianceWhich laws apply; contract terms; intellectual property; employment and consumer protection rules; litigation holds; public statements about AI.
Model risk management (banks and insurers)Independent validation of models, ongoing monitoring and model inventories under existing supervisory guidance.
Business ownersAccountability for each use case: the purpose, the people affected, the decision to deploy and the human review.
Procurement and vendor managementIntake questions, contract clauses, renewal reviews.
Internal auditIndependent assurance that the AI program works as described (see Lesson 8).
Human resources and communicationsWorkforce training, employee use rules, labor relations, public statements.

Banks and some insurers have a head start. The Federal Reserve and the OCC issued their Supervisory Guidance on Model Risk Management (Federal Reserve SR 11-7) on April 4, 2011. It asks for a model inventory, development standards, independent validation by people who did not build the model, ongoing monitoring, and board and senior management oversight.17 Those are most of the pieces of an AI program. A bank's AI governance usually extends its model risk management function to cover generative AI and vendor AI rather than starting over.

Two design choices matter more than the organization chart. First, someone must have the authority to stop a use case that has not been reviewed, and procurement and the software release process must enforce it. Second, the review must scale. A program that sends every AI request to a monthly committee meeting will be bypassed. The common answer is a tiered review: a short intake form for every use, a light review for low-risk uses, and a full assessment only for high-risk ones.

Governance policy, committee, board reporting, training 1. Intake and inventory register every AI use, its owner and vendor 2. Risk tiering low, medium or high, by people affected, data and decision 3. Assess and test impact, security, accuracy and bias, before launch 4. Approve and contract owner and review sign-off; vendor terms in place 5. Deploy with controls human review, logging, access limits, user notice 6. Monitor and retire incidents, drift, vendor changes; reassess yearly; retire unused
The AI governance lifecycle: every use case enters through intake, gets a risk tier, and comes back around for reassessment. Original graphic for this module; steps drawn from the NIST AI RMF 1.0 (Jan. 26, 2023), OMB M-25-21 (April 3, 2025) and the Idaho ITS AI Governance Standard (August 2025), read Nov. 15, 2026.

The AI inventory

Why the inventory comes first

Every serious AI rule assumes the organization knows what AI it uses. OMB Memorandum M-25-21, issued April 3, 2025, requires federal agencies (other than the Defense Department and the intelligence community) to inventory their AI use cases at least once a year, report them to OMB and post a public version.1 The 2025 inventories listed 3,611 use cases across 56 agencies, up from 1,757 in 2024, a 105 percent increase; 445 were rated high-impact, and the Department of Veterans Affairs reported the most, 215.3 Independent agencies publish them too; the Federal Reserve keeps its own AI use case inventory online.18 Idaho's AI governance standard for state agencies requires each agency to keep an inventory of its AI systems and to register generative AI tools in both the agency and the statewide inventory.2 In the private sector, the inventory is the starting point for a Colorado or California automated decisionmaking notice, an EU AI Act classification, a SOC 2 or ISO/IEC 42001 audit, and an honest answer to the board.

It is also the most common gap. Organizations that have governed models for years still find, when they start, that half their AI arrived through vendor updates and employee sign-ups that no one reviewed.

What the inventory records

The inventory is a register, usually kept in the governance, risk and compliance platform or the same system as the data inventory in Lesson 10. A workable set of fields:

FieldWhat to recordWhy it matters
Name, ID and descriptionPlain-language name and what it doesEveryone can tell which system is meant
Business owner and technical ownerNamed people, not departmentsAccountability; who answers questions
Purpose and decisionWhat the output is used for; whether it informs or makes a decision about peopleDrives the risk tier and legal triggers
People affectedCustomers, patients, applicants, employees, the public, childrenEmployment, credit, health, housing, insurance and education decisions carry legal duties
SourceBuilt in house, vendor product, feature in existing software, open model, APIWho can fix it; what the contract says
Model and versionUnderlying model, provider and version; how updates arriveVendor model changes can change behavior overnight
Data inData classes used for training, fine-tuning, retrieval and promptsPrivacy, confidentiality, HIPAA, GLBA, trade secrets
Data out and retentionWhere outputs and prompts go, how long they are kept, whether the vendor may train on themLeakage, records retention, legal holds
Actions and accessWhat the system can do on its own: send, change, delete, buy, run code; what accounts and permissions it usesThe blast radius if it is misled or wrong
Human oversightWho reviews outputs, when, and with what authority to overrideRequired for high-risk uses by OMB, the EU and several states
Risk tier and assessmentsTier; dates of impact assessment, security review, testingShows the review happened
Laws and commitmentsWhich laws, contracts and public statements applyCompliance mapping; avoids false claims
Status and datesPilot, production, retired; next review dateKeeps the inventory alive
One AI inventory record: the fields that matter Identity Name and ID Business owner, technical owner Status: pilot, production, retired Next review date Purpose and people What it does Informs or makes decisions about people? Who is affected: customers, patients, applicants, employees, children Technology and data Source: built, bought, embedded, API Model, provider and version Data in: training, retrieval, prompts Data out: retention, vendor training? Control and risk Actions it can take on its own Accounts and permissions; human oversight Risk tier and assessment dates Laws and public commitments that apply The inventory feeds board reports, regulatory notices, audits and incident response.
The fields of an AI inventory record. Original graphic for this module; fields drawn from OMB M-25-21 (April 3, 2025), the Idaho ITS AI Governance Standard (August 2025) and the NIST AI RMF Map function (Jan. 26, 2023), read Nov. 15, 2026.

Finding AI you did not know about

A first inventory is built from several directions at once, because no single source catches everything:

  • Ask. A short survey of every business unit leader, repeated each year, with examples so people recognize AI features they would not call AI.
  • Follow the money. Purchase orders, contract renewals, corporate card and expense reports for AI subscriptions.
  • Read the vendor notices. Release notes and contract amendments that announce new AI features or new subprocessors. Require procurement to flag them.
  • Use the identity system. Single sign-on records show which cloud applications people sign into with company accounts.
  • Use existing security monitoring. Web gateways, cloud access security brokers and data loss prevention tools the organization already runs can report traffic to known AI services and uploads of sensitive data to them. This is an ordinary use of tools the security team already operates under the organization's monitoring notice, not a new surveillance program; the privacy office and HR should agree on how the information is used.
  • Check the code. Software teams' dependency lists and cloud accounts show which model APIs and model files are in use.

The goal of discovery is not punishment. Programs that treat the first sweep as an amnesty, moving useful tools onto approved enterprise accounts, find more and keep more trust than programs that start with discipline.

Risk tiers

Once a use is on the list, it needs a tier, because the review must be proportionate. The federal and EU rules give usable tests. OMB M-25-21 calls AI "high-impact" when its output is a principal basis for decisions or actions with a legal, material, binding or significant effect on civil rights, civil liberties, privacy, access to education, housing, employment and other programs, human health and safety, critical infrastructure or strategic assets, whether or not a human is in the loop. It presumes certain uses are high-impact, including safety-critical functions of critical infrastructure, decisions about eligibility for benefits and decisions about federal employment.1 The EU AI Act's high-risk list (its Annex III) covers similar ground: biometrics, critical infrastructure, education, employment, access to essential private and public services including credit scoring and life and health insurance pricing, law enforcement, migration, and the administration of justice.16

A three-tier scheme works for most large organizations:

TierTypical examplesReview required
HighDecisions or recommendations about people in employment, credit, insurance, housing, health care, education, benefits or law enforcement; safety functions; agents that can move money or change production systems; anything public-facing that gives advice people will rely onFull impact assessment, security review, accuracy and bias testing before launch, documented human oversight, committee approval, annual reassessment, board visibility
MediumInternal tools that handle confidential or personal data; customer-facing content that a person reviews before release; coding assistants connected to source codeSecurity and privacy review, approved enterprise account, usage rules, owner sign-off
LowDrafting and summarizing public or internal non-sensitive material, with human reviewRegistration on the inventory and compliance with the acceptable use policy

The frameworks: NIST and ISO

Large organizations rarely need to invent an AI governance method. Two families of documents do most of the work: the free NIST publications, which most U.S. regulators, insurers and auditors recognize, and the paid ISO/IEC standards, which give a certifiable management system and are widely used by companies that sell into Europe and Asia. Module 108, Lesson 3 shows how these fit next to the cybersecurity standards.

The NIST AI Risk Management Framework

NIST released the AI Risk Management Framework (AI RMF 1.0, published as NIST AI 100-1) on Jan. 26, 2023, after a public process ordered by Congress in the National AI Initiative Act of 2020. It is voluntary, free and written for any organization that designs, develops, deploys or uses AI.4 It has two parts.

The first part describes what "trustworthy" AI looks like, in seven characteristics: valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair, with harmful bias managed. NIST treats validity and reliability as the base the others rest on, and accountability and transparency as running through all of them.4

The second part is the core, organized into four functions:

FunctionWhat it asks of the organizationTypical evidence
GovernA culture and structure for AI risk: policies, roles, accountability, workforce diversity and training, a process for third-party and supply chain AI risk. NIST says Govern runs through the other three.AI policy, committee charter, RACI, training records, vendor terms
MapUnderstand the context of each AI system: its purpose, users, people affected, limits, and the benefits and harms it could cause.Inventory record, intake form, impact assessment
MeasureTest and track the risks identified: accuracy, resistance to attack, bias, privacy, security, with metrics and independent review where the risk warrants it.Test plans and results, red team findings, monitoring dashboards
ManageDecide and act on the risks: prioritize, treat, accept or avoid; plan for incidents; monitor deployed systems; decommission safely.Risk decisions, incident playbooks, change logs, retirement records

NIST publishes a companion AI RMF Playbook with suggested actions for each subcategory, and runs the Trustworthy and Responsible AI Resource Center, launched March 30, 2023, to collect guidance and crosswalks. NIST's AI RMF page says the framework "is being revised as part of the White House AI Action Plan," which directed NIST to remove references to misinformation, diversity, equity and inclusion, and climate change; no revised version had been released when this lesson was checked on Nov. 15, 2026. NIST also released a concept note on April 7, 2026 for an AI RMF profile on trustworthy AI in critical infrastructure.1920 An organization that adopts AI RMF 1.0 today should expect to update its mapping when the revision arrives, but the four functions are unlikely to change.

The Generative AI Profile, NIST AI 600-1

On July 26, 2024, NIST published NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. It applies the AI RMF to generative AI and lists 12 risks that are "unique to or exacerbated by" it, then maps suggested actions to each.5 The 12 risks give a governance committee a usable checklist:

NIST AI 600-1 riskWhat it means for a large organization
1. CBRN information or capabilitiesEasier access to dangerous chemical, biological, radiological or nuclear information. Mostly a model developer concern; relevant to labs, universities and chemical and energy firms.
2. ConfabulationConfident but false output, commonly called "hallucination." The source of most real cases so far.
3. Dangerous, violent or hateful contentA public-facing assistant producing harmful or offensive content in the company's name.
4. Data privacyLeakage, unauthorized use or re-identification of personal data in training data, prompts or outputs.
5. Environmental impactsEnergy and water use of training and running models; relevant to sustainability reporting.
6. Harmful bias or homogenizationDiscriminatory outcomes and uneven performance across groups or languages.
7. Human-AI configurationOver-reliance on AI, automation bias, or people mistaking a system for a human.
8. Information integrityContent that blurs fact and fiction, including deepfakes used against the organization.
9. Information securityAI that lowers the cost of attacks, and attacks on AI systems themselves, such as prompt injection and data poisoning.
10. Intellectual propertyOutput that reproduces copyrighted or trade secret material; exposure of the organization's own secrets.
11. Obscene, degrading or abusive contentIncluding synthetic sexual imagery, now banned by several laws.
12. Value chain and component integrationThird-party models, data and components whose origin and quality are unclear.

The profile is long and detailed. Its most practical use for a large organization is as a review checklist: for each high-risk generative AI use, the committee records which of the 12 risks apply and what controls address them.

The Cyber AI Profile

The AI RMF is about AI risk generally. For the security team, NIST is also building a profile of its Cybersecurity Framework 2.0 for AI. The initial preliminary draft of NIST IR 8596, Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile), was published Dec. 16, 2025, with comments due Jan. 30, 2026.21 It is organized around three focus areas: securing AI system components (Secure), using AI to improve cyber defense (Defend), and protecting against AI-enabled attacks (Thwart). NIST reported more than 1,400 comments from 144 submissions on the draft, and its May 2026 roadmap listed further working sessions and a possible second preliminary draft before a final version.22 Because it uses the CSF 2.0 functions and categories the security program already reports against, it lets a CISO add AI to the existing program profile instead of running a separate one. NIST is also developing SP 800-53 control overlays for securing AI systems (a project called COSAiS) for organizations that use the federal control catalog; check its status before relying on it.22

ISO/IEC 42001:2023, Information technology: Artificial intelligence: Management system, was published by the International Organization for Standardization and the International Electrotechnical Commission in December 2023. It is the first international standard for an AI management system, and it follows the same high-level structure as ISO/IEC 27001 for information security, so an organization certified to 27001 will recognize its clauses: context, leadership, planning, support, operation, performance evaluation and improvement.6 In outline (the standard itself is copyrighted and sold by ISO and national bodies), it requires an AI policy, defined roles, an AI risk assessment and risk treatment process, AI system impact assessments that consider effects on individuals and society, a statement of which controls in its annex apply and why, documented objectives, internal audit and management review. Its annexes list control objectives and implementation guidance covering topics such as data quality, the AI system life cycle, responsible use and third-party relationships.

Like ISO/IEC 27001, it is certifiable: an accredited certification body audits the management system in two stages and then performs surveillance audits over a three-year cycle. Lesson 6 explains how ISO certification works and how to check a certificate's scope and accreditation. Related ISO/IEC standards a program may hear about include ISO/IEC 23894:2023 (guidance on AI risk management), ISO/IEC 42005 (AI system impact assessment) and ISO/IEC 42006 (requirements for bodies that audit and certify AI management systems), the last two published in 2025.6

What certification proves is limited in the same way as any management system certificate: that a defined system of policies and reviews exists and is followed within the stated scope. It does not certify that any particular AI system is accurate, fair or secure. Buyers should read the scope and ask for the impact assessments behind it.

How the pieces fit

DocumentTypeCostBest use
NIST AI RMF 1.0 and PlaybookVoluntary frameworkFreeThe spine of the program; common vocabulary with U.S. regulators
NIST AI 600-1Profile for generative AIFreeChecklist of 12 generative AI risks and suggested actions
NIST IR 8596 (draft)CSF 2.0 profile for AIFreeAdding AI to the security program's CSF profile
NIST AI 100-2 E2025Taxonomy of attacks on AIFreeShared terms for the security team and testers
ISO/IEC 42001:2023Certifiable management systemPaid standard; certification audit feesProving a program to customers, especially internationally
OWASP Top 10 for LLM ApplicationsCommunity risk listFreeRequirements for developers and vendors of LLM applications
MITRE ATLASKnowledge base of adversary tactics against AIFreeThreat modeling and test planning by the security team
The European Parliament hemicycle in Brussels
The European Parliament's hemicycle in Brussels, 2011. The Parliament adopted the EU AI Act in 2024. Photo: MichalPL. Wikimedia Commons, CC BY-SA 4.0.

Security risks of AI systems

This section explains, at the level a manager or board member needs, how AI systems fail under attack or misuse and which controls address each failure. It does not describe how to carry out any attack. The security team will use the detailed references (OWASP, MITRE ATLAS and NIST AI 100-2) cited here.

Why AI security is different

Traditional software follows instructions written by its programmers. A large language model follows instructions written in ordinary language, and it cannot reliably tell the difference between instructions from its owner and text that merely looks like instructions inside a document, web page or email it has been asked to read. That single fact explains most of the new risk. It means that anything the model reads is a possible source of instructions, and anything the model can do is a possible consequence. It also means that some of these risks cannot be fully fixed with a patch today; they must be contained by limiting what the model can see and do.

NIST's adversarial machine learning taxonomy, NIST AI 100-2 E2025 (March 24, 2025), written with researchers from Northeastern University, Cisco and the U.S. and U.K. AI safety and security institutes, sorts attacks on AI by the stage of the life cycle they target, the attacker's goals and the attacker's knowledge and access. Its broad categories include evasion (inputs crafted to fool a model), poisoning (corrupting training data or the model), privacy attacks (extracting training data or information about it) and misuse or abuse of generative systems.23 MITRE ATLAS is a free knowledge base of adversary tactics and techniques against AI systems, modeled on the MITRE ATT&CK framework security teams already use, with case studies drawn from real incidents and research.24

The OWASP Top 10 for LLM Applications

The Open Worldwide Application Security Project (OWASP), a nonprofit community, publishes the most widely cited list of security risks for applications built on large language models. Its official site listed the 2025 version on Nov. 15, 2026:10

OWASP 2025 entryPlain meaningMain management controls
LLM01 Prompt InjectionText, typed by a user or hidden in content the model reads, changes the model's behavior against the owner's intentTreat all content as untrusted; limit the model's access and actions; require human approval for consequential actions; monitor
LLM02 Sensitive Information DisclosureThe model reveals personal data, secrets or confidential business informationKeep sensitive data out of training and retrieval unless needed; enforce the user's own access rights in retrieval; output filtering; data loss prevention
LLM03 Supply ChainCompromised or low-quality third-party models, datasets, plugins or librariesApproved model sources; vendor review; inventory of models and components; integrity checks
LLM04 Data and Model PoisoningTraining, fine-tuning or retrieval data is corrupted to plant errors or hidden behaviorsControl who can change data sources; provenance records; testing before and after updates
LLM05 Improper Output HandlingOther systems trust model output without checking itTreat output as untrusted input to downstream systems; validate it; ordinary secure coding
LLM06 Excessive AgencyThe system has more permissions, functions or autonomy than its job needsLeast privilege; narrow tools; human approval steps; separate accounts for agents
LLM07 System Prompt LeakageThe hidden instructions given to the model are exposed, sometimes with secrets in themNever put passwords, keys or sensitive rules in prompts; enforce controls outside the model
LLM08 Vector and Embedding WeaknessesWeaknesses in the stores used for retrieval-augmented generation, such as mixed access rights or poisoned entriesAccess control at the document level; separate stores by sensitivity; review what is indexed
LLM09 MisinformationFalse or misleading output that people rely onHuman review; citations to sources; clear notice to users; accuracy testing
LLM10 Unbounded ConsumptionUncontrolled use that runs up costs, degrades service or allows copying of the modelRate limits, quotas, budget alerts, monitoring

An October 2026 industry research note described a 2026 revision of the list that moves excessive agency up to third place and renames system prompt leakage as "hidden context exposure"; the OWASP project site still showed the 2025 version when this lesson was checked, so treat the 2026 ranking as unconfirmed until OWASP publishes it.25

Prompt injection in plain terms

Prompt injection comes in two forms. In direct injection, a user types instructions meant to override the system's rules. In indirect injection, the instructions are hidden in material the AI is asked to process: an email it summarizes, a web page it reads, a shared document, a calendar invite. Indirect injection is the larger risk for a large organization, because enterprise assistants are designed to read exactly those things, and an outsider can put text in front of the model simply by sending an email.

The enterprise assistant case is not hypothetical. In June 2025 Microsoft assigned CVE-2025-32711 to an information disclosure flaw in Microsoft 365 Copilot that researchers showed could be triggered through crafted content the assistant processed, and fixed it on the service side.26 The general lesson for managers is in the design questions, not the details: what can the assistant read, what can it send or link to, and what does it do with instructions it finds inside content?

Because no vendor can promise a model will never follow injected text, the controls are about containment:

  • Give the system access only to the data the current user is already allowed to see, enforced by the underlying systems, not by the model's good behavior.
  • Separate reading from acting. A tool that summarizes outside email should not also be able to send email or move files without a person's confirmation.
  • Require a human approval step for actions with consequences: payments, deletions, external messages, changes to production systems, access grants.
  • Log prompts, retrieved content and actions so incidents can be investigated, under a retention rule the privacy office approves.
  • Test before launch and after major changes, including testing by people trying to make the system misbehave (AI red teaming), and keep the results.

Data leakage and confidentiality

The most common AI security problem in large organizations is not an exotic attack. It is employees pasting confidential material into AI tools whose terms let the provider keep it, review it or train on it, and enterprise assistants that surface documents to people who were never meant to see them because file permissions were too broad to begin with. The first is solved mostly by policy, training and enterprise accounts with contractual protections. The second is solved by cleaning up access rights before turning on an assistant that searches everything a user can reach. Organizations that rolled out enterprise assistants found that the assistant made years of over-shared folders easy to find. A readiness review of permissions and data labels belongs in the project plan, and it is far cheaper than an incident.

AI agents and excessive agency

Agents are AI systems given tools and permission to act. They can save real work, and they concentrate risk: a wrong or manipulated decision becomes an action at machine speed. The same autonomy helps attackers. On Nov. 13, 2025, the AI company Anthropic reported that in mid-September 2025 it had detected a cyber espionage campaign, which it assessed with high confidence was run by a Chinese state-sponsored group, that used its Claude Code tool to attempt intrusions into about 30 organizations, including large technology companies, financial institutions, chemical manufacturers and government agencies, succeeding in a small number of cases. Anthropic described it as the first documented case of a large-scale cyberattack executed largely without human intervention, and said it banned the accounts, notified affected organizations and coordinated with authorities over the following ten days.27

For a large organization the governance rules for agents are straightforward to state, even if they take work to enforce:

  1. Every agent is on the inventory with a named human owner.
  2. Every agent runs under its own identity, not a person's account, with the least privilege its task needs, managed like any other privileged non-human account (see Module 108, Lesson 5).
  3. Actions above a defined threshold (money, deletion, external communication, production changes, access grants) need a person's approval.
  4. Agents' actions are logged and reviewed, and there is a tested way to stop an agent quickly.
  5. Agents are tested against misuse before launch, and retested when the model, tools or permissions change.

Where a recommendation in this lesson involves changing production systems, such as connecting an assistant to company data, changing permissions or adding an approval step, plan the change, test it in a non-production setting first and keep a way back.

AI used against the organization

The other half of AI security is defending against attackers who use AI. Better-written phishing, voice clones of executives and fake video calls make old verification habits fail. In early 2024 an employee in the Hong Kong office of the engineering firm Arup joined a video call with what appeared to be the chief financial officer and other colleagues, all of them deepfakes, and sent 200 million Hong Kong dollars (about US$25.6 million) in 15 transactions; Arup confirmed it was the victim in May 2024.28 The defenses are mostly procedural: payment changes and urgent transfers verified through a separately known channel, dual approval for large payments, code words or call-backs for executive requests, and training that uses current examples. Module 110, Lesson 6 covers payment fraud controls in detail, and Module 108, Lesson 9 covers training.

Risk Main controls Owners Prompt injection Least privilege; human approval for actions; treat content as untrusted CISO, system owner Data leakage Enterprise accounts; data rules; clean up permissions first CISO, privacy office Supply chain and poisoning Approved model sources; vendor review; testing after updates Procurement, engineering Excessive agency Separate agent identities; approval thresholds; a tested stop switch Identity team, business owner AI-enabled fraud and deepfakes Call-back verification; dual approval; training with current examples Finance, CISO
Five AI security risks and who owns the main controls. Original graphic for this module; risks drawn from the OWASP Top 10 for LLM Applications 2025 and NIST AI 600-1 (July 26, 2024), read Nov. 15, 2026.

The rules as of November 2026

AI law in late 2026 is a patchwork: one broad foreign law that reaches many U.S. companies, a federal government that has chosen guidance and procurement over regulation and is pushing back on the states, a handful of state laws with different targets, and sector regulators applying the laws they already have. A large organization's compliance office needs a map of all four.

The EU AI Act

The EU Artificial Intelligence Act, Regulation (EU) 2024/1689, entered into force Aug. 1, 2024.16 It reaches U.S. organizations in three main ways: when they place an AI system or general-purpose AI model on the EU market, when they deploy AI systems inside the EU, and when they are outside the EU but the output of their AI system is used in the EU. A U.S. bank with a branch in Dublin, a software company selling an AI product to European customers, and a manufacturer whose machines with AI safety features are sold in Germany are all within reach.

The act sorts AI by risk:

  • Prohibited practices (Article 5), banned since Feb. 2, 2025: among them, manipulative techniques that cause significant harm, exploiting people's vulnerabilities, social scoring, predicting crime from profiling alone, untargeted scraping of facial images to build face recognition databases, emotion recognition in workplaces and schools (with medical and safety exceptions), biometric categorization to infer sensitive traits, and real-time remote biometric identification in public spaces for law enforcement except in narrow cases.16
  • High-risk AI systems: AI used as a safety component of products already regulated in the EU (its Annex I, such as medical devices and machinery) and AI used in the areas listed in Annex III (biometrics, critical infrastructure, education, employment, essential services including credit scoring and life and health insurance pricing, law enforcement, migration and justice). Providers must run a risk management system, govern training data, keep technical documentation and logs, design for human oversight, meet accuracy, resilience and cybersecurity requirements, and pass a conformity assessment. Deployers must use the system according to its instructions, assign trained people to oversee it, monitor it, keep its logs for at least six months, and tell workers before using it on them; public bodies and certain private deployers, including those that score credit or price life and health insurance, must do a fundamental rights impact assessment first.16
  • Transparency duties (Article 50): people must be told when they interact with an AI system unless it is obvious, AI-generated synthetic content must be marked in a machine-readable way, and deepfakes must be disclosed.
  • General-purpose AI models: providers of large models must keep technical documentation, give information to downstream providers, have a copyright policy and publish a summary of training content; the most capable models, those with "systemic risk," must also evaluate and mitigate those risks, report serious incidents and maintain cybersecurity. These duties have applied since Aug. 2, 2025.29
  • Everything else (most AI) carries no specific duties beyond the act's requirement that providers and deployers take measures to support AI literacy among their staff.

The schedule changed in 2026. The European Commission proposed a "Digital Omnibus" simplification package in November 2025 that included delaying the high-risk duties. The AI part of it was signed July 8, 2026, published in the Official Journal on July 24, 2026 and entered into force July 27, 2026. It moved the high-risk duties for Annex III systems from Aug. 2, 2026 to Dec. 2, 2027, and for Annex I product systems from Aug. 2, 2027 to Aug. 2, 2028; kept most Article 50 transparency duties at Aug. 2, 2026, with generative AI systems already on the market before that date given until Dec. 2, 2026 for content marking; added bans on AI systems that generate child sexual abuse material or non-consensual intimate images of identifiable people; and changed the AI literacy duty so that providers and deployers must "take measures to support" literacy.729 General-purpose AI models placed on the market before Aug. 2, 2025 must comply by Aug. 2, 2027.29 Other parts of the Digital Omnibus, including GDPR changes, were still under negotiation when this lesson was checked.7

Penalties are set in Article 99: up to 35 million euros or 7 percent of worldwide annual turnover, whichever is higher, for prohibited practices; up to 15 million euros or 3 percent for most other violations by providers, deployers and others; and lower amounts for supplying incorrect information to authorities.30

Prohibited practices High-risk systems Annex III and Annex I Transparency duties chatbots, deepfakes, synthetic content Minimal risk: most AI AI literacy measures only Banned since Feb. 2, 2025 New CSAM and intimate image bans: Dec. 2, 2026 Annex III: Dec. 2, 2027 (was Aug. 2, 2026) Annex I products: Aug. 2, 2028 (was Aug. 2, 2027) Aug. 2, 2026; marking for older generative systems by Dec. 2, 2026 General-purpose models duties since Aug. 2, 2025; older models by Aug. 2, 2027 Top fine: 35 million euros or 7 percent of worldwide turnover
EU AI Act risk tiers and application dates after the AI Omnibus. Original graphic for this module; dates from Regulation (EU) 2024/1689 and the AI Omnibus as summarized by White & Case (Aug. 4, 2026) and the Future of Privacy Forum (July 28, 2026), read Nov. 15, 2026.

United States federal policy

There is no general federal AI statute. Federal policy since January 2025 has worked through executive orders, OMB rules for agencies, procurement and a push against state regulation:

ActionDateWhat it does
Executive Order 14148Jan. 20, 2025Revoked a list of earlier orders, including Executive Order 14110 of Oct. 30, 2023, the prior administration's main AI order.31
Executive Order 14179, "Removing Barriers to American Leadership in Artificial Intelligence"Jan. 23, 2025 (90 Fed. Reg. 8741)States a policy "to sustain and enhance America's global AI dominance" and directs review of actions taken under the revoked order.15
OMB M-25-21, "Accelerating Federal Use of AI through Innovation, Governance, and Public Trust"April 3, 2025Replaced M-24-10. Agencies must name a chief AI officer within 60 days; large agencies must convene AI governance boards within 90 days and publish AI strategies within 180 days; agencies must inventory AI annually; high-impact AI must have pre-deployment testing, an AI impact assessment, human oversight, remedies or appeals for people affected, and must be paused if it fails the minimum practices, documented within 365 days (by April 3, 2026).132
OMB M-25-22, "Driving Efficient Acquisition of Artificial Intelligence in Government"April 3, 2025Buying rules: contracts should bar vendors from using government data to train or improve AI without agency consent, require knowledge transfer and data and model portability to avoid lock-in, allow ongoing monitoring of performance and risk, and consider requiring vendors to disclose AI use the agency did not expect.1432
"Winning the Race: America's AI Action Plan"July 2025Among many items: revise the NIST AI RMF; create an AI Information Sharing and Analysis Center (AI-ISAC) led by the Department of Homeland Security; issue guidance on AI-specific vulnerabilities; promote secure-by-design AI; and add AI to CISA's incident and vulnerability response playbooks, with agency CISOs consulting chief AI officers.20
Executive Order 14365, "Ensuring a National Policy Framework for Artificial Intelligence"Dec. 11, 2025Told the Attorney General to form an AI Litigation Task Force to challenge state AI laws (formed Jan. 9, 2026), told Commerce to identify "onerous" state laws within 90 days, and set conditions on some federal broadband funding.933

Congress has not passed a law preempting state AI regulation. The Senate removed a proposed state-law moratorium from the 2025 budget reconciliation bill by a 99 to 1 vote in July 2025, and a bipartisan House discussion draft released June 4, 2026 had not been introduced as a bill as of a September 2026 summary.34 An executive order cannot itself displace a state statute, so organizations remain bound by state AI laws unless a court or Congress says otherwise. Whether the Task Force has won any case against a state law could not be confirmed for this lesson.

For private organizations, the federal documents still matter in three ways. Federal contractors will see M-25-22's terms in their contracts. The NIST frameworks remain the reference regulators and courts look to for reasonable practice. And federal agencies keep enforcing existing law against AI misuse, as the cases below show.

State AI laws

The state picture as of Nov. 15, 2026, for the laws a large organization is most likely to meet:

LawStatusWhat it requires, in brief
Colorado SB 26-189 (repealed and reenacted the 2024 Colorado AI Act, SB 24-205, which had been delayed to June 30, 2026)Signed May 14, 2026; compliance from Jan. 1, 2027Notice when people interact with AI; a plain-language disclosure within 30 days of an adverse outcome from covered automated decisionmaking; a way to correct data and get human review where commercially reasonable; developer documentation and three years of records. The 2024 law's risk management program, impact assessments and duty of reasonable care were dropped. Attorney general enforcement, with a 60-day cure right until Jan. 1, 2030.8
California SB 53, Transparency in Frontier Artificial Intelligence ActSigned Sept. 29, 2025Applies to developers of frontier models (trained with more than 10^26 operations); large frontier developers (over $500 million in annual revenue) must publish a frontier AI framework and assess catastrophic risks. Critical safety incidents go to the state Office of Emergency Services within 15 days, or within 24 hours to the appropriate authority if there is imminent risk of death or serious injury. Whistleblower protections; civil penalties up to $1 million per violation, enforced by the attorney general.35
California CCPA regulations on automated decisionmaking technology, risk assessments and cybersecurity auditsEffective Jan. 1, 2026; ADMT duties from Jan. 1, 2027Pre-use notice, opt-out and access rights for ADMT used for significant decisions; risk assessments; see Lesson 10.36
Texas HB 149, Texas Responsible Artificial Intelligence Governance Act (TRAIGA)Effective Jan. 1, 2026Bans developing or deploying AI with intent to unlawfully discriminate, incite harm or produce certain prohibited content; disclosure duties fall mainly on government agencies and health care providers; attorney general enforcement.37
Utah Artificial Intelligence Policy Act (SB 149 of 2024, amended by SB 226 and SB 332 of 2025)In effect since May 1, 2024; amendments effective May 7, 2025; set to repeal July 2027Disclosure of generative AI use when a consumer clearly asks, and up front in "high-risk" interactions involving sensitive data and personalized advice; a business cannot avoid consumer protection liability by blaming the AI; fines of $2,500 per violation by the Division of Consumer Protection; a state Office of AI Policy with a regulatory sandbox.38
Illinois HB 3773 (amending the Illinois Human Rights Act)Effective Jan. 1, 2026Bars employers from using AI in ways that have the effect of discriminating on protected grounds and requires notice to employees when AI is used in covered decisions.39
New York City Local Law 144Enforced since July 5, 2023Annual independent bias audit, published results and notice before using automated employment decision tools on candidates or employees in the city.40
Idaho SB 1297, Conversational AI Safety ActSigned March 31, 2026; effective July 1, 2027Covers publicly accessible conversational AI services, with exclusions that include customer service and internal business chatbots; see the Idaho section below.41

The pattern across states is that the broad model (Colorado's 2024 law, closely patterned on the EU approach) retreated in 2026, while narrower laws on disclosure, chatbots, employment decisions, deepfakes and frontier model safety kept advancing. For a compliance office, the practical answer is to build the duties that appear in almost every proposal (inventory, notice, a path to human review and correction, records, vendor documentation, testing for discrimination in consequential decisions) and track the rest. Many states also passed laws on election deepfakes and sexual deepfakes; Idaho passed three such laws in 2024.42

Idaho

Idaho has no general AI law, but it has three things a large organization operating in the state should know.

The Conversational AI Safety Act. Senate Bill 1297 passed the Senate 21 to 12 on March 19, 2026 and the House 54 to 12 on March 26, and Gov. Brad Little signed it March 31, 2026 (Session Law Chapter 249). It takes effect July 1, 2027.41 The engrossed bill text covers publicly accessible "conversational AI services" that mainly simulate human conversation and are made available to the public by an "operator." It excludes, among others, tools used only inside a business, services offered only under a commercial agreement, features inside other software, narrow-topic tools, voice assistants in consumer electronics, and customer service, operational, productivity, internal research and technical assistance chatbots. A covered operator must clearly disclose that the service is AI when a reasonable person could think it is human (with a persistent or repeated disclosure for minors); must adopt a protocol that makes reasonable efforts to refer users who express suicidal ideation to crisis services; may not program the service to state that it is designed to provide professional mental or behavioral health care; and, when it knows or is reasonably certain a user is under 18, must take reasonable measures against sexual content, claims of sentience and engagement rewards. The attorney general enforces the law, with civil penalties of $1,000 per violation up to $500,000 per operator, or actual damages if greater, and the law creates no private right of action.4143 For a large organization the exclusions matter most: an ordinary customer service chatbot on a bank, hospital, utility or insurer website appears to fall within the customer service exclusion, while a general-purpose or companion-style assistant open to the public would be covered. Check each public assistant against the exclusions before deciding.

AI in schools. Senate Bill 1227, signed March 19, 2026 (Session Law Chapter 71, effective July 1, 2026; a ceremonial signing followed March 26), directs the Idaho Department of Education to develop a statewide framework for using generative AI in classrooms.44

State agencies. The Idaho Office of Information Technology Services (ITS) issued an AI Governance Policy, Standard and Guideline (P.ITS-01, S.ITS-01 and G.ITS-01, Version 1.0, dated August 2025) for state employees, contractors and third parties handling state data, with an AI Executive Committee under the Idaho Technology Authority. It requires that every AI system, including pilots and free tools, be reviewed and approved through ITS processes and appear on the agency's approved list; that agencies keep an AI inventory and register generative AI tools in the statewide inventory; that restricted data be used only with enterprise-approved generative AI and critical data not at all without explicit high-risk authorization; that personal information, security-sensitive information and login credentials never go into third-party tools; that all generative AI output used for official purposes be reviewed by a person; that AI-generated content be identified, with a disclaimer on public-facing content; and that executives, officials, users and technical staff be trained.2 The document's revision history calls Version 1 an initial draft, so agencies should confirm the currently approved version with ITS. The case for those rules was made in public: in July 2025, reporting on the state's plans noted that a public chatbot on the Division of Human Resources website had told a user Idaho had about 713,015 state employees, then about 23,546, then that it did not know.42

Counties, cities, school districts and public hospitals in Idaho are not bound by the ITS policy, but it is a ready model, and the inventory it requires is a public record that can be requested under Idaho Code Title 74.

Sector regulators apply existing law

Most AI enforcement in the United States so far has come from regulators using laws that existed long before generative AI:

  • Securities. On March 18, 2024, the SEC settled charges against two investment advisers, Delphia (USA) Inc. ($225,000) and Global Predictions Inc. ($175,000), for false and misleading statements about their use of AI, the first "AI washing" cases. Delphia had claimed to use AI and machine learning that incorporated client data when it did not have that capability.11 Every public company that describes its AI in a 10-K, an earnings call or a press release is subject to the same antifraud rules; Module 108, Lesson 4 covers the disclosure machinery.
  • Consumer protection. In December 2023 the FTC banned the pharmacy chain Rite Aid from using facial recognition for surveillance for five years, alleging it had deployed the technology in stores without reasonable safeguards, leading to false matches that fell disproportionately on women and people of color, and without adequate testing, monitoring, vendor oversight or employee training.45 State attorneys general use their own unfair and deceptive practices laws: in September 2024 the Texas attorney general settled with Pieces Technologies, a Dallas health care AI company whose generative AI product summarized patient records at at least four Texas hospitals, over accuracy claims including a "severe hallucination rate" of less than 1 per 100,000. The company agreed to disclose its products' accuracy accurately and to make sure hospital staff understand how far to rely on them.46
  • Banking and insurance. Model risk management guidance (SR 11-7) applies to AI models at banks, and fair lending and fair housing laws apply to AI credit decisions.17
  • Health care. HIPAA applies to any AI vendor that handles protected health information for a covered entity; see Module 109, Lesson 6.
  • Professional services. Courts and bar regulators hold lawyers responsible for AI-generated filings; see Module 110, Lesson 2 and the Mata case below.

Acceptable use of generative AI

Every large organization needs a written rule for how its people may use generative AI, because the alternative is that each employee decides alone. Bans rarely hold: people use the tools on their phones and home computers, and the organization loses both visibility and the chance to provide a safer option. Most large organizations have settled on "approved tools, approved data, human responsibility." The sample policy later in this lesson puts that into words. The decisions behind it are these:

  1. Which tools are approved. Usually enterprise versions with contract terms that bar the provider from training on the organization's data, set retention limits, support single sign-on and produce audit logs. Consumer versions on personal accounts are allowed only for public information, or not at all.
  2. Which data may go in. Tie the rule to the existing data classification scheme (see Module 108, Lesson 7). Idaho's state standard is a good public model: restricted data only in enterprise-approved tools, critical data only with explicit high-risk authorization, and never personal information, security-sensitive information or credentials in third-party tools.2
  3. Who is responsible for the output. The person who uses it. AI output is a draft. It must be checked for accuracy, sources, confidentiality and fairness before it is relied on or sent outside.
  4. What must be disclosed. When content or decisions shown to customers, patients, the public or regulators were produced with AI, and how. Several laws now require disclosure in specific settings; the policy can go further.
  5. What is prohibited. For example: making final decisions about people without human review; generating content that impersonates real people; creating images of real people without consent; uploading others' copyrighted material to create derivative works without rights; evading security controls; and building or connecting new AI tools without going through intake.
  6. Where records go. Prompts and outputs that become part of business records follow the retention schedule and legal holds (see Module 73, Lesson 17).
  7. How to get a new use approved, and who to ask.

The policy only works with training and a usable approved tool. The training should use the organization's own examples: what a confidential document looks like, what a confabulated citation looks like, what a deepfake payment request looks like.

A rack server chassis with GPUs open for testing
A Supermicro server with AMD Epyc processors and Nvidia GPUs during client testing, 2024. AI systems run on hardware like this, in the organization's data center or a provider's cloud. Photo: Lawrence Systems. Wikimedia Commons, CC BY 3.0.

AI vendors and contracts

Most AI in a large organization comes from vendors, either as AI products or as AI features in software already under contract. The general third-party risk process in Module 108, Lesson 8 applies; AI adds specific questions and clauses. Lesson 5 explains how to read the SOC 2 report a vendor offers, and the first question to ask is whether the AI service is inside its scope.

Questions to ask before buying

  • What model or models does the product use, who provides them, and how are model updates rolled out and announced?
  • Will our data, prompts or outputs be used to train or improve the vendor's or any third party's models? Can that be turned off by contract, not just by a setting?
  • Where are prompts and outputs stored, for how long, who at the vendor can see them, and can we delete them?
  • Which subprocessors, including model providers, receive our data, and in which countries?
  • What testing has the vendor done for accuracy, bias, security and misuse, and will it share the results? Has it had independent AI red team testing?
  • Does the product take actions on its own? With what permissions? Can we require approval steps?
  • What logs do we get, and can they go to our security monitoring?
  • Is the AI service within the scope of the vendor's SOC 2 report or ISO/IEC 27001 certificate? Does the vendor hold ISO/IEC 42001 certification, and what is its scope?
  • For high-risk uses: can the vendor give us the documentation we need to meet our own legal duties (notices, impact assessments, EU deployer duties, state disclosures)?
  • What happens to our data and any custom models if we leave?

Contract terms

ClauseWhat it should say
Use of customer dataNo training, fine-tuning or improvement of any model with customer data, prompts or outputs without written consent. This mirrors OMB M-25-22's rule for federal contracts.14
Confidentiality and retentionPrompts and outputs are customer confidential information; retention period; deletion on request and at termination, with certification.
Subprocessors and locationList of subprocessors including model providers; notice and objection right for changes; data location limits.
Model change noticeAdvance notice of material changes to the model or its behavior, and the right to test before the change reaches production for high-risk uses.
SecuritySecurity program requirements, including protections against prompt injection and data leakage appropriate to the product; independent testing; current SOC 2 or ISO reports covering the AI service.
Incident noticeNotice within a set number of hours of any security incident, data exposure or serious AI malfunction affecting the customer, with cooperation in investigation.
Documentation and cooperationInformation needed for the customer's legal duties: impact assessments, transparency notices, records, regulator inquiries.
Accuracy and performanceTruthful descriptions of capabilities and limits; performance measures for high-risk uses; remedies if not met. The Pieces settlement shows why accuracy claims belong in the contract.46
Intellectual propertyWho owns outputs; vendor indemnity for infringement claims from the model or its training data, and its limits.
Audit and monitoringRight to monitor performance and risk, receive logs and audit or receive third-party assurance.
Exit and portabilityReturn of data and any customer-specific models or configurations in a usable format; transition help. M-25-22 asks for knowledge transfer and portability to avoid lock-in.14
Disclosure of AI useVendors and service providers must disclose when they use AI to deliver services to the customer, including through their own subcontractors.

Large vendors will resist some of these. The point of the list is to know which terms the organization did not get, record the risk acceptance, and add compensating controls (for example, keeping sensitive data out of a tool whose retention terms are weak).

Board oversight of AI

AI is now both a strategic question and a risk question for boards, and the Delaware oversight duty discussed in Module 108, Lesson 4 applies to it: where a risk is mission critical, the board needs a reporting system and must pay attention to it. For a health system using AI in clinical documentation, an insurer pricing with models, a bank scoring credit or a software company selling AI products, AI is plainly in that category.

What the board should receive

A short report at least twice a year, and more often for organizations whose products depend on AI, covering:

  1. The inventory: total AI uses by tier, new high-risk uses approved since the last report, and uses retired.
  2. High-risk uses: each one in a line, with its owner, last assessment date and any open issues.
  3. Incidents and near misses involving AI, including AI-enabled fraud attempts against the organization.
  4. Vendor exposure: the main AI vendors, contract gaps accepted, and any vendor incidents.
  5. Legal and regulatory changes and the organization's readiness (for example, EU high-risk duties in December 2027, Colorado in January 2027, Idaho's chatbot law in July 2027).
  6. Public statements: what the company says about its AI in filings, marketing and sales materials, and who checked those claims.
  7. Training completion and policy exceptions.
  8. Investments and expected benefits, so risk is weighed against value.

Lesson 3 shows how to fit these into the board's security dashboard rather than adding a separate report.

Which committee

Boards are placing AI oversight with the audit committee (disclosure and controls), a risk committee, a technology committee, or the full board for strategy. The same rules apply as for cyber: write it into a committee charter, put it on the calendar, and report to the full board. Many boards add AI to the existing cyber and technology agenda because the same executives report on both.

Disclosure and AI washing

Public companies increasingly name AI in their 10-K risk factors, describing both the risk of falling behind competitors and the risks of using AI. The SEC's 2024 cases show the other side: describing AI capabilities a company does not have is fraud.11 The control is simple and often missing: claims about the company's AI in filings, earnings materials, marketing and sales decks should be reviewed against the inventory by someone who knows what the systems actually do, the same way the disclosure committee checks other factual claims.

AI incidents

What counts as an AI incident

An AI incident is any event in which an AI system causes or nearly causes harm, or in which an AI system is attacked or misused. That includes ordinary security incidents involving AI (a data leak through an assistant, a compromised AI vendor, a manipulated agent), but also failures that are not attacks at all: a chatbot that gives customers wrong information, a model that drifts and starts denying claims it should approve, a hiring tool that turns out to screen out a protected group, a deepfake of the chief executive circulating online. Many of those will never reach the security team unless the incident response plan says they should.

Cases that shaped practice

CaseWhat happenedThe governance lesson
Mata v. Avianca, Inc. (S.D.N.Y., June 22, 2023)Lawyers filed a brief citing court decisions that did not exist, generated by ChatGPT, and stood by them when questioned. The court imposed a $5,000 sanction.12The person who uses the output is responsible for checking it. Confabulation is predictable, so verification must be part of the work process.
Moffatt v. Air Canada, 2024 BCCRT 149 (Feb. 14, 2024)The airline's website chatbot told a grieving customer he could apply for a bereavement fare after travel, contrary to the airline's policy. The British Columbia Civil Resolution Tribunal rejected the argument that the chatbot was responsible for its own statements and ordered the airline to pay the difference.13An organization is responsible for what its chatbot tells customers. Public-facing AI needs testing against the organization's actual policies and a correction path.
FTC and Rite Aid (Dec. 2023)Facial recognition deployed in stores without adequate testing, monitoring, vendor oversight or training; five-year ban.45High-risk AI needs pre-deployment testing, ongoing monitoring of errors and their effect on different groups, and vendor oversight.
SEC and Delphia, Global Predictions (March 18, 2024)False claims about AI capabilities; $400,000 in total penalties.11Public claims about AI must match the inventory.
Texas and Pieces Technologies (Sept. 2024)Accuracy claims for a hospital AI product the attorney general found likely inaccurate.46Buyers should test vendor accuracy claims; vendors should be able to support them.
Arup (2024)About US$25.6 million sent after a deepfake video call.28Verification procedures for payments must assume voices and faces can be faked.
Idaho Division of Human Resources chatbot (2025)A public chatbot gave wildly different answers about the number of state employees.42Public chatbots need testing on likely questions before launch and monitoring after.
AI-orchestrated espionage campaign reported by Anthropic (Nov. 13, 2025)A state-sponsored group used an AI coding agent to carry out much of an intrusion campaign against about 30 targets.27Defenders should plan for faster, cheaper attacks; the basic controls in Module 108, Lesson 10 matter more, not less.

Adding AI to incident response

The America's AI Action Plan recommended that CISA's incident and vulnerability response playbooks add AI considerations and that agency CISOs consult chief AI officers and privacy officials.20 A large organization can do the same in its own plan:

  • Add AI incident types to the classification scheme, including non-security failures that cause harm.
  • Name the chief AI officer or AI governance lead as a member of the response team for AI incidents, with the privacy office and the general counsel's office.
  • Make sure logs of prompts, retrieved content, outputs and agent actions exist and are kept long enough to investigate.
  • Have a way to switch off or roll back an AI feature or agent quickly, and test it, as part of the continuity planning in Lesson 12.
  • Know which notices may be required: breach notification laws if personal data was exposed, SEC Item 1.05 if material, EU serious incident reporting for high-risk systems and general-purpose models, California SB 53 for frontier developers, and contractual notices to customers.
  • Prepare holding statements for a public AI failure, because these become news quickly.
A museum exhibition on artificial intelligence and robotics
The artificial intelligence and robotics exhibit at the Heinz Nixdorf MuseumsForum in Paderborn, Germany, 2020. Photo: Sergei Magel/HNF. Wikimedia Commons, CC BY-SA 4.0.

Building the program, step by step

For a large organization starting or rebuilding an AI governance program, a realistic first year looks like this:

  1. Month 1: name the owner and the committee. An executive sponsor (chief AI officer, chief risk officer or chief data officer), with the CISO, chief privacy officer, the general counsel's office, compliance, HR, procurement and major business units on a committee with a written charter.
  2. Month 1: issue an interim acceptable use rule. Short, with approved tools and banned data, while the full policy is written.
  3. Months 1 to 3: build the first inventory using the discovery methods above, with an amnesty for unregistered tools.
  4. Month 2: adopt a framework. NIST AI RMF 1.0 and AI 600-1 for most U.S. organizations; add ISO/IEC 42001 if customers will ask for certification.
  5. Months 2 to 4: set up intake and tiering, connected to procurement and the software release process so nothing launches without a record.
  6. Months 3 to 5: assess high-risk uses first, with impact assessments, security review and testing, and fix or pause what fails.
  7. Months 3 to 6: fix the contracts for the biggest AI vendors at renewal, using the clause list above.
  8. Months 4 to 6: publish the full policy and train everyone, with role-based training for developers, procurement, HR and customer-facing teams.
  9. Months 4 to 8: add AI to security operations: monitoring of AI services, logging, agent identities, AI scenarios in incident response and tabletop exercises.
  10. Month 6: first board report using the format above.
  11. Months 6 to 12: map legal duties by jurisdiction and date (EU, Colorado, California, Idaho, sector rules) into the compliance calendar in Lesson 4.
  12. Month 12: internal audit review of the program, and a decision on whether to pursue ISO/IEC 42001 certification.

Sample: Generative AI Acceptable Use Policy

This sample sits under the organization's top-level information security policy (the sample in Module 108, Lesson 4) and next to its acceptable use and data classification policies.

Sample only. Adapt it to your organization's risks, systems and laws, have qualified people review it, and do not treat it as a substitute for a risk assessment.

Follows: NIST AI Risk Management Framework 1.0 (Jan. 26, 2023), Govern and Map functions; NIST AI 600-1 Generative AI Profile (July 26, 2024); OMB M-25-21 and M-25-22 (April 3, 2025) as public-sector models; Idaho ITS AI Governance Policy, Standard and Guideline, Version 1.0 (August 2025); OWASP Top 10 for LLM Applications 2025; EU AI Act Articles 4, 26 and 50 for organizations within its reach.

1. Purpose

[Organization name] uses generative artificial intelligence to improve its work while protecting the people it serves, its information and its reputation. This policy sets the rules for using generative AI tools and assigns responsibility for them.

2. Scope

This policy applies to all employees, officers, contractors, temporary staff and volunteers, and to vendors acting on the organization's behalf, whenever they use generative AI for the organization's work or with its information, on any device or account. "Generative AI" means software that produces text, images, audio, video, code or other content in response to prompts, including AI features inside other software and AI agents that can take actions.

3. Approved tools

Only tools listed on the Approved AI Tools list, maintained by [the AI Governance Committee] at [location], may be used with non-public information. A tool is approved only after an intake review covering security, privacy, legal terms and the use case, and only under an enterprise agreement that [bars the vendor from training on our data, limits retention to [X days], supports single sign-on and provides audit logs]. Personal accounts on any AI service may not be used for the organization's work except with [Public] information. New AI tools, plug-ins, browser extensions, agents and connections between AI tools and company systems must go through intake before use, including free tools and pilots.

4. Information you may and may not enter

  • [Public] information may be used with any approved tool.
  • [Internal] information may be used only with approved enterprise tools.
  • [Confidential / Restricted] information, including personal information about customers, patients, employees or applicants, may be used only with tools approved for that data class and only for the approved use case.
  • Never enter passwords, keys or other credentials; security-sensitive information such as network diagrams or vulnerability details; [protected health information, payment card numbers, Social Security numbers] except in a tool approved for that data; or information that a contract, court order or law forbids sharing.

5. Human responsibility for output

You are responsible for anything you create, send or decide with the help of AI. Treat AI output as a draft. Before relying on it or sharing it outside your team, check its facts, figures, quotations and citations against original sources; check that it does not reveal confidential information; and check it for unfair or offensive content. AI may not make a final decision about a person's employment, credit, insurance, benefits, health care, housing, education or legal rights without review by a qualified person who has authority to change the outcome and who documents the review.

6. Disclosure

Tell the recipient when content shown to customers, patients, the public, regulators or courts was substantially generated by AI, using the language in [the disclosure standard]. Public-facing chatbots and voice agents must say they are AI at the start of every conversation and must follow [the crisis response protocol] when a user expresses thoughts of self-harm. Do not describe the organization's AI capabilities in marketing, sales, investor or regulatory materials unless [the AI Governance Committee / Disclosure Committee] has confirmed the description against the AI inventory.

7. Prohibited uses

  • Creating content that impersonates a real person or the organization without authorization, or images or audio of real people without their consent.
  • Using AI to evade security controls, monitoring or this policy.
  • Uploading material you do not have rights to use for that purpose.
  • Using AI for any practice prohibited by law, including discrimination on protected grounds.
  • Giving an AI agent authority to make payments, delete records, send external communications, change production systems or grant access without the human approval steps in [the AI agent standard].

8. Records, retention and legal holds

Prompts and outputs that become part of a business record are kept under the records retention schedule. When a legal hold applies, AI prompts and outputs related to the matter must be preserved. Do not use AI tools to summarize or store information subject to a legal hold unless [the general counsel's office] has approved the tool for that purpose.

9. Reporting problems

Report immediately to [the security operations center / report channel] any suspected exposure of confidential information through an AI tool, any AI output that caused or could cause harm, any unexpected AI action, and any suspected deepfake or AI-assisted fraud attempt. Reporting in good faith will not lead to discipline for the report itself.

10. Roles

[The Chief AI Officer / executive sponsor] owns this policy and chairs the AI Governance Committee. The CISO sets security requirements for AI tools and monitors their use. The Chief Privacy Officer reviews uses involving personal data. [The general counsel's office] reviews contract terms and legal requirements. Business owners are accountable for each approved use case and its human review. Internal Audit reviews compliance with this policy [annually].

11. Training, enforcement and review

All workforce members complete AI training before receiving access to approved tools and [annually] after. Violations may lead to loss of access and discipline up to termination. Exceptions require written approval by [the CISO and the AI policy owner], must state compensating controls and expire after no more than [12] months. This policy is reviewed at least every [12] months and whenever a major law or tool changes. Version [X], approved [date] by [body]. Next review due [date].

Pictures, Numbers and Infographics

The numbers and dates a board member, compliance officer or reporter is most likely to need from this lesson, each with its source.

ItemNumber or dateSource
NIST AI RMF 1.0 releasedJan. 26, 2023NIST4
NIST AI 600-1 Generative AI Profile; number of risks it listsJuly 26, 2024; 12NIST AI 600-15
NIST Cyber AI Profile draft; comments receivedDec. 16, 2025; more than 1,400 comments from 144 submissionsNIST2122
Federal AI use cases reported, 2025 and 20243,611 across 56 agencies; 1,757OMB inventory as reported by Nextgov3
OMB deadline for high-impact AI minimum practicesApril 3, 2026 (365 days after M-25-21)OMB M-25-211
EU AI Act entry into force; prohibitions apply; general-purpose AI duties applyAug. 1, 2024; Feb. 2, 2025; Aug. 2, 2025Regulation (EU) 2024/1689; FPF1629
EU high-risk duties after the AI Omnibus (Annex III; Annex I)Dec. 2, 2027; Aug. 2, 2028White & Case; FPF729
AI Omnibus published; in forceJuly 24, 2026; July 27, 2026White & Case7
EU AI Act top fine35 million euros or 7 percent of worldwide annual turnoverArticle 9930
Executive Order 14110 revoked; EO 14179 signed; EO 14365 signedJan. 20, 2025; Jan. 23, 2025; Dec. 11, 2025Federal Register31159
Senate vote removing the state AI law moratorium99 to 1, July 2025CASRAI summary34
Colorado SB 26-189 signed; compliance dateMay 14, 2026; Jan. 1, 2027Law firm summary8
California SB 53 thresholds; maximum penaltyMore than 10^26 operations; over $500 million revenue; up to $1 million per violationJones Day35
California SB 53 incident report deadline15 days (24 hours if imminent risk of death or serious injury)Jones Day35
Utah AI Policy Act fine$2,500 per violationDavis Polk38
Idaho SB 1297 votes; signed; effectiveSenate 21 to 12; House 54 to 12; March 31, 2026; July 1, 2027Idaho Legislature41
SEC AI washing penalties, March 18, 2024$225,000 (Delphia); $175,000 (Global Predictions)SEC press release 2024-3611
Rite Aid facial recognition ban5 years (Dec. 2023)FTC45
Mata v. Avianca sanction$5,000 (June 22, 2023)S.D.N.Y. opinion12
Arup deepfake lossHK$200 million, about US$25.6 million, in 15 transactionsCNN, May 202428
Targets in the AI-orchestrated espionage campaignAbout 30Anthropic, Nov. 13, 202527
2023 2024 2025 2026 2027 2028 NIST AI RMF, Jan. 26 NYC LL 144, July 5 Utah AI Act, May 1 NIST AI 600-1, July 26 EU AI Act in force, Aug. 1 EO 14110 revoked, Jan. 20 EU bans apply, Feb. 2 OMB M-25-21, -22, Apr. 3 EU GPAI duties, Aug. 2 Calif. SB 53, Sept. 29 EO 14365, Dec. 11 Texas, Illinois laws, Jan. 1 OMB practices due, Apr. 3 Colorado reset, May 14 AI Omnibus, July 27 EU transparency, Aug. 2 Colorado, Calif. ADMT, Jan. 1 Idaho chatbot law, July 1 EU Annex III high-risk, Dec. 2 EU Annex I, Aug. 2
Key AI rule dates for a large U.S. organization, 2023 to 2028. Original graphic for this module; dates from the sources in the table above, read Nov. 15, 2026.

Common mistakes

  • Deploying AI that makes or shapes decisions about people without testing it first. The FTC's Rite Aid case turned on a retailer that used facial recognition in stores without adequate testing of accuracy, monitoring of false matches, oversight of its vendors or training of the employees who acted on the alerts.45 The cure is the pre-deployment assessment and ongoing monitoring that NIST, OMB and the EU all describe.
  • Treating AI output as finished work. In Mata v. Avianca, fabricated case citations went to a federal court and the people who filed them were sanctioned.12 Similar episodes have since involved reports, filings and published articles. Verification has to be a required step in the process, not a matter of individual care.
  • Assuming the organization is not responsible for its chatbot. Air Canada argued that its chatbot was responsible for its own statements; the tribunal disagreed.13 Utah's law says the same thing in statute: the use of generative AI is not a defense to a consumer protection violation.38
  • Launching a public chatbot before testing it on ordinary questions. Idaho's HR chatbot gave three different answers about the size of the state workforce.42 A short list of likely questions, checked by a person before launch and monitored after, would have caught it.
  • Overstating AI in public statements. The SEC's 2024 AI washing cases and the Texas Pieces settlement both involved claims about AI that the companies could not support.1146
  • Building the program around one statute. Organizations that built their Colorado program around the 2024 law's impact assessments and duty of care saw the law delayed and then replaced in 2026, and the EU high-risk deadline moved by 16 months.87 A program built on a framework and an inventory survives those changes.
  • Turning on an enterprise assistant before fixing access rights. An assistant that can search everything a user can reach will surface every over-shared folder. The fix is permission cleanup first, which is a production change to plan, test and be able to reverse.
  • Giving agents broad standing access. OWASP lists excessive agency among the top risks for good reason: an agent with an employee's full permissions and no approval step turns one manipulated instruction into an incident.10
  • Leaving AI out of payment verification. The Arup loss happened because a video call was accepted as verification.28

Questions people ask

Do we need a chief AI officer?

Federal agencies must have one under OMB M-25-21.1 No law requires a private company to create the title. What every large organization needs is one named executive who owns the AI inventory, the review process and the board report, with authority to stop unreviewed uses. That can be a chief AI officer, a chief data officer, the chief risk officer or the CISO, as long as the role is written down and resourced and the person is not the only check on their own projects.

Should we block public AI tools?

Many organizations block consumer AI sites on company networks and devices while providing an approved enterprise tool. Blocking alone tends to push use to personal devices where the organization cannot see it. Blocking plus a good approved alternative, a clear data rule and training works better. Highly sensitive environments (classified work, some research and clinical settings) may justify stricter limits.

Does the EU AI Act apply to a U.S. company?

It can, if the company places AI systems or general-purpose models on the EU market, deploys AI in the EU, or uses AI whose output is used in the EU.16 A U.S. company that only uses AI for its U.S. operations, with no EU customers, staff or output, is generally outside it. The inventory's "people affected" and "location" fields are how a compliance office answers the question system by system.

Is ISO/IEC 42001 certification worth it?

It is worth it when customers, especially in Europe, Asia or regulated industries, ask for it, or when the organization sells AI products and needs a recognized way to show a management system. For an organization that mainly uses AI internally, aligning with the NIST AI RMF and having internal audit test the program gives most of the benefit at lower cost. Lesson 6 covers the certification process and costs.

How do we handle AI features a vendor turns on in software we already use?

Treat a new AI feature as a change to the vendor relationship. Require vendors by contract to give notice before enabling AI features that process the organization's data, review the feature through intake, and turn it off until reviewed if the vendor allows that. Read the vendor's release notes and subprocessor change notices as part of vendor management.

What does Idaho's chatbot law mean for an organization with a website assistant?

First decide whether it is covered. SB 1297 excludes customer service, operational, productivity and technical assistance chatbots, tools used only inside a business and services offered only under a commercial agreement, among others. If the assistant is publicly accessible, conversational and not excluded, plan now for the July 1, 2027 effective date: clear AI disclosure, a protocol that refers users who express suicidal ideation to crisis services, no statement that it is designed to provide professional mental or behavioral health care, and added protections when the operator knows a user is a minor.4143 Check the enrolled bill text for the exact definitions and exemptions.

How does internal audit review an AI program?

The same way it reviews any risk program: test whether the inventory is complete (by comparing it with procurement, single sign-on and network records), whether high-risk uses received the required assessments before launch, whether human oversight actually happens, whether contracts include the required terms, whether incidents were handled under the plan, and whether board reporting matches the facts. The NIST AI RMF and, where adopted, ISO/IEC 42001 give the criteria. Lesson 8 covers the audit process.

Hard questions

Should AI rules come from the states or from Washington?

The case for a national standard: companies that operate in every state face a growing set of different and sometimes conflicting laws; AI products are sold across state lines; and a single rule would be easier to follow and enforce. Executive Order 14365 rests on this view and directs federal challenges to state laws seen as burdensome.9 The case for state action: Congress has not passed a general AI law; states have historically protected consumers and workers where federal law is silent; and the Senate's 99 to 1 vote to strip a moratorium from the 2025 budget bill showed broad reluctance to bar state laws without a federal replacement.34 What the evidence shows: as of November 2026 no federal statute preempts state AI laws, the state laws that took effect are narrower than the EU model, and Colorado narrowed its own law in 2026 after federal pressure and industry objections.8 For a compliance office, the practical consequence is the same either way: build to the common duties and watch the courts.

Should AI agents be allowed to act without a person approving each step?

For more autonomy: requiring approval for every step removes much of the value of agents, and people approving dozens of routine actions a day stop reading them, so the approval becomes theater. Well-scoped agents with narrow permissions, logging and spending limits can be safer than overloaded staff. For tight limits: models can be misled by text they read, their errors are not predictable in the way software bugs are, and the Anthropic report showed what an agent can do at machine speed when pointed at the wrong goal.27 OWASP's list puts excessive agency among the top risks.10 What the evidence shows: there is not yet good public data comparing error rates of supervised and unsupervised agents in business settings. The defensible position today is risk-based: autonomy for low-consequence, reversible actions; human approval for actions that move money, delete data, communicate externally, change production systems or affect people's rights.

Do frameworks and certifications make AI safer, or just produce paperwork?

For: a management system forces an inventory, assessments, assigned owners and review, which are exactly the steps missing in the Rite Aid and Air Canada cases; certification gives customers a way to compare vendors. Against: a certificate says a process exists, not that any particular system is accurate or fair, and the same criticism has long been made of security certifications held by companies that were later breached (see Lesson 5). What the evidence shows: ISO/IEC 42001 is too new for studies of outcomes. The enforcement record so far punishes missing basics (no testing, no review, no oversight, false claims), which a working management system would catch. Whether a certified program catches them depends on whether the organization tests its systems, not on the certificate.

Should employees be told when AI monitors or evaluates them?

For notice: Illinois, New York City and the EU AI Act all require some form of notice before AI is used in employment decisions, and notice lets workers correct errors.394016 Against broad notice: employers argue that security monitoring, including detection of data sent to AI services, loses value if every method is disclosed, and that general monitoring notices already cover it. What the evidence shows: the laws so far focus on decisions about employment (hiring, promotion, discipline, pay), not on security monitoring. A reasonable line is a general monitoring notice for security tools and specific notice, human review and a correction path for any AI used to evaluate people.

For reporters and advocates

Public records that exist:

  • Federal AI use case inventories. Each federal agency posts one under OMB M-25-21, and OMB publishes a consolidated set. They list use cases, which are high-impact, and in many cases the vendor.13 Independent agencies such as the Federal Reserve post their own.18
  • Agency AI strategies and chief AI officer names under M-25-21, and waivers of the high-impact minimum practices, which agencies must report publicly with their reasons.1
  • SEC filings on EDGAR. Search 10-K risk factors and Item 1C for "artificial intelligence"; compare what companies tell investors with what they tell customers. SEC enforcement releases cover AI washing.11
  • FTC and state attorney general orders on AI claims and AI tools, including Rite Aid and the Texas Pieces settlement.4546
  • New York City bias audit summaries, which employers using automated employment decision tools must post publicly.40
  • California frontier AI frameworks, which large frontier developers must publish under SB 53.35
  • Court dockets on PACER and CourtListener for AI sanctions orders, discrimination suits involving AI hiring tools, and chatbot liability cases.
  • Idaho records. State agency AI inventories and approved tool lists kept under the ITS AI Governance Standard, AI contracts, AI Executive Committee materials, and Legislature committee records on SB 1297 and SB 1227 are public records that can be requested under Idaho Code Title 74, subject to its exemptions for security-sensitive information.241

Questions to ask an organization:

  • Do you have an inventory of AI uses? How many are high-risk, and who reviews them before they launch?
  • Which AI systems make or shape decisions about people, and how can a person get a human review or correct the data?
  • Does your AI vendor contract let the vendor train on your data or the public's data?
  • What testing was done for accuracy and fairness before launch, and can we see the results?
  • Has any AI system been paused or withdrawn after problems? What happened?
  • For a public agency: is this use on the published inventory, was it classified as high-impact, and was a waiver granted?
  • For a public company: does what you say about AI in your 10-K match what your marketing says?
  • For an Idaho organization with a public chatbot: is it covered by SB 1297 or excluded, and if covered, what is your plan for the July 1, 2027 requirements?

The Idaho angle. Idaho's state AI governance rules were written by ITS and presented to the Legislature's AI working group in 2025, with pilots at the Division of Human Resources and the Division of Motor Vehicles.42 Reporters can ask how many tools are on the statewide inventory, which agencies run public chatbots, how the high-risk review works in practice, and whether counties, cities and school districts have adopted similar rules. The Conversational AI Safety Act passed with significant opposition in the Senate (21 to 12), and its rules, effect on businesses and enforcement after July 1, 2027 are a story to follow.41

When writing about AI incidents, describe what happened at the level of the government report, court record or company statement, distinguish confirmed facts from claims, and give the date of each source; AI rules and products change month to month.

Where to go next

  • NIST AI Risk Management Framework and Playbook: The free U.S. framework for AI risk, with a Playbook of suggested actions. For AI governance leads, CISOs, privacy and risk staff. Free. Use it as the structure of the AI program and check the page for the revision NIST says is under way.
  • NIST AI 600-1, Generative AI Profile: NIST's list of 12 generative AI risks and suggested actions. For review committees and security teams. Free. Use the 12 risks as a checklist for each high-risk generative AI use.
  • NIST Cyber AI Profile (IR 8596 draft): A draft CSF 2.0 profile for securing AI, using AI for defense and countering AI-enabled attacks. For CISOs. Free. Use it to add AI to the security program's existing CSF profile; check for newer drafts.
  • OWASP Top 10 for LLM Applications: Community list of the main security risks in LLM applications with mitigations. For developers, security architects and vendor reviewers. Free. Use it as requirements for internal builds and as questions for vendors.
  • MITRE ATLAS: Knowledge base of adversary tactics and real case studies against AI systems. For security teams planning threat models and tests. Free. Use it alongside MITRE ATT&CK in threat modeling.
  • OMB M-25-21 and M-25-22: Federal rules for agency AI governance and buying. Required for federal agencies; a free, practical model for state and local governments and private organizations. Use the high-impact test and the procurement terms as templates.
  • Idaho ITS AI Governance Policy, Standard and Guideline: Idaho's rules for state agency AI use: approval, inventory, data limits, human review, disclosure and training. For Idaho agencies, local governments and contractors. Free. Use it as a model policy and confirm the current approved version with ITS.
  • EU AI Act Explorer: An independent, searchable copy of the EU AI Act with amendments and summaries, kept by a nonprofit research institute. For compliance teams at organizations with EU exposure. Free. Use it to read articles quickly, and the Official Journal text for anything you rely on.
Still being checked (10 items)

These points could not be fully confirmed against a primary source when this lesson was written and fact checked in November 2026. The text above states each one as uncertain.

  • ISO/IEC 42001, 23894, 42005 and 42006 details: the ISO catalog refused automated access on Nov. 15, 2026, so the publication dates of 42005 and 42006 (2025) and the description of 42001's structure rest on earlier public summaries.
  • The EU AI Omnibus regulation number (2026/1744) and signing date (July 8, 2026) come from one law firm alert (White & Case, Aug. 4, 2026); the Official Journal entry was not opened directly.
  • The Mata v. Avianca, Moffatt v. Air Canada and FTC Rite Aid sources refused automated access on Nov. 15, 2026; the facts stated are from the published decisions and order as widely reported and were not re-read today.
  • Illinois HB 3773 text was not opened (site refused automated access); the description follows the Module 108 fact check of Nov. 13, 2026.
  • Whether the DOJ AI Litigation Task Force has filed or won any case against a state AI law by Nov. 15, 2026 could not be confirmed; the CASRAI summary of Sept. 20, 2026 reported none decided.
  • A 2026 revision of the OWASP Top 10 for LLM Applications is described in an Oct. 21, 2026 Cloud Security Alliance note but did not appear on the OWASP project site on Nov. 15, 2026.
  • SR 11-7's page refused automated access; its date (April 4, 2011) and contents are stated from the guidance as previously published.
  • The web search budget for this run ran out before every item could be searched; no photos were placed if IMAGES.md was not available (see the build report).
  • Federal AI inventory: OMB's 2024 total is reported as 1,757 by Nextgov and Global Government Forum but as about 2,133 across 41 agencies by ExecutiveGov; the OMB data set itself was not opened. The lesson uses 1,757.
  • The Texas Pieces settlement date (Sept. 18, 2024) is not shown on the Texas attorney general page; the page confirms the facts stated.

Last verified November 15, 2026. General education, not legal, compliance, audit or other professional advice; rule versions and dates are stated in the text, and the sources used are listed in the Notes and Sources below. Check the regulator's current text before relying on any point.

Notes

  1. Office of Management and Budget, Memorandum M-25-21, Accelerating Federal Use of AI through Innovation, Governance, and Public Trust, April 3, 2025: chief AI officers, AI strategies, annual use case inventories, high-impact AI definition and minimum practices within 365 days; rescinds M-24-10 (read Nov. 15, 2026). Read it online. ↩
  2. Idaho Office of Information Technology Services, Artificial Intelligence (AI) Governance Policy, Standard, and Guideline (P.ITS-01, S.ITS-01, G.ITS-01), Version 1.0, August 2025 (read Nov. 15, 2026). Read it online. ↩
  3. Nextgov/FCW, Agencies report over 3,000 AI use cases in 2025, April 2026: OMB documented "3,611 individual use cases across 56 submitting agencies," a 105 percent increase from 1,757 in 2024; 445 high-impact uses, 215 of them at the Department of Veterans Affairs. ExecutiveGov's report of the same release gave a different 2024 figure (about 2,133 across 41 agencies); the lesson uses OMB's 1,757, also reported by Global Government Forum (read Nov. 16, 2026). Read it online. ↩
  4. National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, Jan. 26, 2023: trustworthiness characteristics and the Govern, Map, Measure and Manage functions (read Nov. 15, 2026). Read it online. ↩
  5. NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 26, 2024: the 12 risks unique to or exacerbated by generative AI (read Nov. 15, 2026). Read it online. ↩
  6. International Organization for Standardization, catalog page for ISO/IEC 42001:2023, Information technology: Artificial intelligence: Management system (published December 2023). The ISO site refused automated access on Nov. 15, 2026; the description of the standard's structure and of ISO/IEC 23894, 42005 and 42006 is from ISO's public summaries as previously published and should be checked against the catalog (read Nov. 15, 2026). Read it online. ↩
  7. White & Case, EU AI Omnibus enters into force, amending the AI Act, Aug. 4, 2026: signed July 8, 2026; published July 24, 2026 as Regulation (EU) 2026/1744; in force July 27, 2026; new high-risk dates; new prohibitions; other Digital Omnibus parts still under negotiation (read Nov. 15, 2026). Read it online. ↩
  8. Mondaq (republishing a law firm summary), Colorado Hits Reset On AI Regulation: SB 26-189 Repeals And Reenacts The Colorado AI Act, 2026: signed May 14, 2026; compliance Jan. 1, 2027; cure right until Jan. 1, 2030 (fact-checked for Module 108 on Nov. 13, 2026) (read Nov. 15, 2026). Read it online. ↩
  9. Executive Order 14365 of Dec. 11, 2025, Ensuring a National Policy Framework for Artificial Intelligence, Federal Register, Dec. 16, 2025: sections 3 (AI Litigation Task Force), 4 (evaluation of state AI laws) and 5 (funding conditions) (read Nov. 15, 2026). Read it online. ↩
  10. OWASP GenAI Security Project, OWASP Top 10 for LLM Applications 2025 (LLM01 to LLM10), as listed on the project site (read Nov. 15, 2026). Read it online. ↩
  11. U.S. Securities and Exchange Commission, press release 2024-36, SEC Charges Two Investment Advisers with Making False and Misleading Statements About Their Use of Artificial Intelligence, March 18, 2024 (read Nov. 15, 2026). Read it online. ↩
  12. Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), opinion and order on sanctions (S.D.N.Y. June 22, 2023), 678 F. Supp. 3d 443. The court's site and the linked copy refused automated access on Nov. 15, 2026; details are from the published opinion as widely reported (read Nov. 15, 2026). Read it online. ↩
  13. Moffatt v. Air Canada, 2024 BCCRT 149, British Columbia Civil Resolution Tribunal, Feb. 14, 2024. CanLII refused automated access on Nov. 15, 2026; details are from the published decision as widely reported (read Nov. 15, 2026). Read it online. ↩
  14. OMB, Memorandum M-25-22, Driving Efficient Acquisition of Artificial Intelligence in Government, April 3, 2025 (read Nov. 15, 2026). Read it online. ↩
  15. Executive Order 14179 of Jan. 23, 2025, Removing Barriers to American Leadership in Artificial Intelligence, 90 Fed. Reg. 8741 (Jan. 31, 2025), adopting the definition in 15 U.S.C. 9401(3) (read Nov. 15, 2026). Read it online. ↩
  16. Regulation (EU) 2024/1689 of the European Parliament and of the Council (Artificial Intelligence Act), Official Journal of the European Union, July 12, 2024, in force Aug. 1, 2024: Articles 2, 3(1), 4, 5, 6, 26, 27, 50 and Annex III (read Nov. 15, 2026). Read it online. ↩
  17. Board of Governors of the Federal Reserve System, SR 11-7, Guidance on Model Risk Management, April 4, 2011 (issued with OCC Bulletin 2011-12) (read Nov. 15, 2026). Read it online. ↩
  18. Board of Governors of the Federal Reserve System, AI Use Case Inventory (read Nov. 15, 2026). Read it online. ↩
  19. NIST, AI Risk Management Framework page: AI RMF 1.0 "is being revised as part of the White House AI Action Plan"; Trustworthy and Responsible AI Resource Center launched March 30, 2023; critical infrastructure profile concept note April 7, 2026 (read Nov. 15, 2026). Read it online. ↩
  20. The White House, Winning the Race: America's AI Action Plan, July 2025: recommended actions on revising the NIST AI RMF, an AI-ISAC led by DHS, secure-by-design AI and AI incident response (pages 4, 18 and 19) (read Nov. 15, 2026). Read it online. ↩
  21. NIST, Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile): NIST Community Profile, NIST IR 8596, initial preliminary draft, Dec. 16, 2025, comments closed Jan. 30, 2026 (read Nov. 15, 2026). Read it online. ↩
  22. NIST, Cyber AI Profile Project Overview, presentation, May 2026: Secure, Defend and Thwart focus areas; more than 1,400 comments from 144 submissions; roadmap; COSAiS listed among NIST projects (read Nov. 15, 2026). Read it online. ↩
  23. Vassilev, Oprea, Fordyce, Anderson, Davies and Hamin, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2 E2025, March 24, 2025 (read Nov. 15, 2026). Read it online. ↩
  24. MITRE, ATLAS knowledge base of adversary tactics and techniques against AI-enabled systems (read Nov. 15, 2026). Read it online. ↩
  25. Cloud Security Alliance, The OWASP Top 10 for LLM Applications 2026: What Changed and Why It Matters, Oct. 21, 2026 (describing a 2026 list not yet shown on the OWASP project site on Nov. 15, 2026) (read Nov. 15, 2026). Read it online. ↩
  26. NIST National Vulnerability Database, CVE-2025-32711 (Microsoft 365 Copilot information disclosure), published June 2025; see also Microsoft's Security Update Guide entry (read Nov. 15, 2026). Read it online. ↩
  27. Anthropic, Disrupting the first reported AI-orchestrated cyber espionage campaign, Nov. 13, 2025 (read Nov. 15, 2026). Read it online. ↩
  28. CNN Business, Arup revealed as victim of $25 million deepfake scam involving Hong Kong employee, May 2024: HK$200 million (about US$25.6 million) in 15 transactions (read Nov. 15, 2026). Read it online. ↩
  29. Future of Privacy Forum, The AI Act implementation timeline: What changes under the AI Omnibus?, July 28, 2026 (updated Aug. 5, 2026): dates before and after the amendment, including general-purpose AI dates (read Nov. 15, 2026). Read it online. ↩
  30. EU Artificial Intelligence Act, Article 99 (Penalties), consolidated text with amendments, artificialintelligenceact.eu (read Nov. 15, 2026). Read it online. ↩
  31. Executive Order 14148 of Jan. 20, 2025, Initial Rescissions of Harmful Executive Orders and Actions, 90 Fed. Reg. 8237 (Jan. 28, 2025), revoking Executive Order 14110 of Oct. 30, 2023 (read Nov. 15, 2026). Read it online. ↩
  32. Hunton Andrews Kurth, OMB Issues Revised Policies on AI Use and Procurement by Federal Agencies, April 2025: deadlines and procurement provisions of M-25-21 and M-25-22 (read Nov. 15, 2026). Read it online. ↩
  33. Proskauer Rose, What President Trump's AI Executive Order 14365 Means for Employers, April 6, 2026: AI Litigation Task Force established Jan. 9, 2026 (read Nov. 15, 2026). Read it online. ↩
  34. CASRAI Editorial Board, Federal AI Preemption Fight: Where It Stands, Sept. 20, 2026: the 99-1 Senate vote of July 2025, the June 4, 2026 House discussion draft, no enacted preemption statute (read Nov. 15, 2026). Read it online. ↩
  35. Jones Day, California Enacts SB 53, Setting New Standards for Frontier AI Safety Disclosures, October 2025: signed Sept. 29, 2025; thresholds, incident reporting, whistleblower protections and penalties (read Nov. 15, 2026). Read it online. ↩
  36. California Privacy Protection Agency, California Finalizes Regulations to Strengthen Consumers' Privacy, Sept. 23, 2025, with compliance dates for risk assessments, cybersecurity audits and ADMT (read Nov. 15, 2026). Read it online. ↩
  37. Texas Legislature Online, 89th Legislature, HB 149 (Texas Responsible Artificial Intelligence Governance Act), bill history, effective Jan. 1, 2026 (read Nov. 15, 2026). Read it online. ↩
  38. Davis Polk, Utah scales back reach of generative AI consumer protection law, 2025: SB 149 (effective May 1, 2024), SB 226 and SB 332 (effective May 7, 2025), repeal date July 2027, $2,500 fines (read Nov. 15, 2026). Read it online. ↩
  39. Illinois Public Act 103-0804 (HB 3773), amending the Illinois Human Rights Act on employer use of artificial intelligence, effective Jan. 1, 2026. The Illinois General Assembly site refused automated access on Nov. 15, 2026 (read Nov. 15, 2026). Read it online. ↩
  40. New York City Department of Consumer and Worker Protection, Automated Employment Decision Tools (AEDT), Local Law 144 of 2021 and rules, enforced from July 5, 2023 (read Nov. 15, 2026). Read it online. ↩
  41. Idaho Legislature, 2026 session, Senate Bill 1297, Conversational AI Safety Act: bill history (Senate 21-12-2 on March 19; House 54-12-4 on March 26; signed March 31, 2026; Session Law Chapter 249; effective July 1, 2027); engrossed bill text S1297E1 (definitions and exclusions, disclosure, suicidal ideation protocol, mental health statements, minors, attorney general enforcement, penalties, no private right of action) (bill history read Nov. 15, 2026; bill text read Nov. 16, 2026). Read it online. ↩
  42. Route Fifty (from the Idaho Capital Sun), Idaho preps to roll out artificial intelligence in state government, July 2025: draft AI framework, DMV and HR chatbot pilots, the HR chatbot's answers, and Idaho's three 2024 AI laws (read Nov. 15, 2026). Read it online. ↩
  43. Enterprise DNA, Idaho's Chatbot Safety Law (summary of SB 1297), 2026: scope, disclosure, crisis protocols, minors, $1,000 per violation up to $500,000. A secondary summary; check the enrolled bill text (read Nov. 15, 2026). Read it online. ↩
  44. Idaho Legislature, 2026 session, Senate Bill 1227, bill history (signed by the governor March 19, 2026; Session Law Chapter 71; effective July 1, 2026), read Nov. 16, 2026; Idaho Education News, 'The genie's out of the bottle:' Little signs artificial intelligence education bill, March 26, 2026, on the ceremonial signing and the statewide framework (read Nov. 15, 2026). Read it online. ↩
  45. Federal Trade Commission, Rite Aid Banned from Using AI Facial Recognition After FTC Says Retailer Deployed Technology without Reasonable Safeguards, press release, Dec. 19, 2023. The FTC site refused automated access on Nov. 15, 2026 (read Nov. 15, 2026). Read it online. ↩
  46. Office of the Texas Attorney General, Attorney General Ken Paxton Reaches Settlement in First-of-its-Kind Healthcare Generative AI Investigation (Pieces Technologies), September 2024 (read Nov. 15, 2026). Read it online. ↩

Check yourself

1. Under OMB Memorandum M-25-21, when is a federal AI use case "high-impact"?

2. After the AI Omnibus took effect July 27, 2026, when do the EU AI Act's high-risk duties apply to stand-alone Annex III systems?

3. What are the four functions of the NIST AI Risk Management Framework 1.0?

4. How many risks "unique to or exacerbated by" generative AI does NIST AI 600-1 list?

5. Which entry is first on the OWASP Top 10 for LLM Applications 2025?

6. In March 2024 the SEC settled the first "AI washing" cases. What was the conduct?

7. What did the tribunal decide in Moffatt v. Air Canada (2024)?

8. When does Idaho's Conversational AI Safety Act (SB 1297 of 2026) take effect?

9. Which contract term does OMB M-25-22 tell federal agencies to include when buying AI?

10. As of November 2026, what is the legal effect of Executive Order 14365 on state AI laws?

Terms in this lesson

AI inventory
A register of every AI system or use in an organization, with its owner, purpose, people affected, data, model, actions, oversight, risk tier and review dates.
High-impact AI
OMB M-25-21's term for AI whose output is a principal basis for decisions or actions with a legal, material, binding or significant effect on rights or safety; it triggers minimum risk practices for federal agencies.
High-risk AI system
Under the EU AI Act, AI used as a safety component of regulated products (Annex I) or in listed areas such as employment, credit, insurance, education and critical infrastructure (Annex III), subject to the act's strictest duties.
Generative AI
AI that produces text, images, audio, video or code in response to prompts, such as large language models.
Large language model (LLM)
A generative AI model trained on large amounts of text that produces language by predicting what comes next; fluent but capable of stating false things confidently.
Confabulation
NIST's term for confidently stated but false AI output, commonly called hallucination.
Prompt injection
Text, typed by a user or hidden in content an AI system reads, that changes the system's behavior against its owner's intent; indirect injection hides the text in emails, documents or web pages.
AI agent
An AI system given tools and permissions to take actions, such as sending messages, changing records or running code, with limited human involvement.
Excessive agency
OWASP's term for an AI application having more permissions, functions or autonomy than its job requires.
Retrieval-augmented generation (RAG)
A design in which an AI system searches a document store and uses what it finds to answer, which makes access control on the store a security issue.
AI washing
Overstating or misdescribing an organization's use of AI in public statements; treated by the SEC as securities fraud in 2024 cases.
AI management system
The policies, roles, processes and reviews an organization uses to govern AI; ISO/IEC 42001:2023 sets certifiable requirements for one.
Shadow AI
AI tools employees use for work without approval or registration, often on personal accounts.
Deployer
Under the EU AI Act, an organization that uses an AI system under its authority in the course of its work, as distinct from the provider that developed it.

Full glossary

Sources