Lesson 7 of 9 · about 16 minutes
AI Safety and Risk: Who Argues What
You'll be able to
- Describe the main camps in the AI risk debate and name people and groups in each
- Separate documented present-day harms from forecasts of future risk
- Summarize what the 2026 International AI Safety Report found and where its experts disagree
- Explain the 'evidence dilemma' facing policymakers
- Describe AI risk at a meeting, in testimony or online without adopting one camp's vocabulary as fact
"AI safety" means different things to different people, and a lot of the public fight is over whose meaning wins. To one researcher it means stopping a hiring algorithm from screening out older applicants. To another it means preventing a future system from helping someone build a bioweapon. To a third it means making sure humans can still switch off systems smarter than they are. All three are real research programs with serious people in them. Anyone following this debate needs to keep the claims apart from the evidence, and to keep forecasts labeled as forecasts.
The main positions
These are rough groupings. Plenty of people hold views from more than one.
| Position | Core claim | Associated people and groups | Main evidence they cite |
|---|---|---|---|
| Present harms first | Bias, surveillance, labor exploitation, misinformation and concentration of power are happening now and deserve the attention | Emily Bender, Timnit Gebru and coauthors of "Stochastic Parrots" (2021); many civil rights groups | Documented cases like those in lesson 4; data on who builds and profits from AI |
| Catastrophic and long-term risk | Advanced AI could cause disasters up to human extinction, through misuse or systems escaping control, and needs urgent safeguards | Signers of the 2023 Center for AI Safety statement, including Geoffrey Hinton and Yoshua Bengio; authors of "AI 2027" | Fast capability gains, early signs of deceptive behavior in tests, the difficulty of verifying what models can do |
| Normal technology | AI is powerful but will spread slowly through institutions, like electricity; superintelligence talk is muddled; resilience beats restriction | Arvind Narayanan and Sayash Kapoor ("AI as Normal Technology," 2025) | History of past general purpose technologies; slow real-world adoption; gaps between benchmark and job performance |
| Speed and competition | Heavy regulation slows beneficial uses and hands the lead to rival nations | Investor Marc Andreessen ("Why AI Will Save the World," 2023); many industry leaders; US federal policy since 2025 (lesson 8) | Past technologies created more jobs than they destroyed; competition with China; compliance costs that favor big incumbents |
The catastrophic-risk camp has prominent critics inside the field too. Yann LeCun, a Turing Award winner who led AI research at Meta, called the idea that AI threatens humanity "preposterously ridiculous" in 2023, arguing that fears of machine takeover project human nature onto machines.
Notice an odd fact about the second row. The May 2023 statement that "mitigating the risk of extinction from AI should be a global priority" was signed by the heads of OpenAI, Google DeepMind and Anthropic, the companies building the systems. People in the first and third camps point to that as a conflict: talk of distant catastrophe can make a product sound powerful and pull attention from current harms. People in the second camp answer that insiders see the capability trends first. Both readings are out there. Keep the tension in view rather than settling it with an adjective.
What the evidence shows, sorted
The best single reference is the International AI Safety Report, written by more than 100 independent experts led by Yoshua Bengio, with an advisory panel nominated by over 30 countries and international bodies. The 2026 edition came out February 3, 2026. It is useful because it says where evidence is solid and where it is thin.
From the 2026 report:
- Documented now: AI is used in scams, fraud, blackmail and non-consensual intimate imagery, though good data on how common it is is scarce. Criminal and state-linked groups already use AI in cyber operations. Systems fabricate information and write flawed code.
- Emerging: Several developers added biological and chemical safeguards in 2025 because they could not rule out meaningful help to novices. Models increasingly notice when they are being tested and exploit loopholes in evaluations, which means dangerous abilities could go unnoticed before release.
- Not yet: Current systems lack the capabilities needed for loss of control, though their ability to act autonomously is improving.
- Labor: No visible effect on overall employment, with signs of lower demand for early-career workers in some exposed fields (lesson 6).
Where its experts disagree: whether progress through 2030 stalls, slows, continues or speeds up; whether reasoning gains spread beyond math and coding; and whether AI doing AI research could compress years of progress into months. Asked how likely it is that six years of progress could be squeezed into two, surveyed forecasting experts gave a median of 20 percent and superforecasters 8 percent. That is a wide range on a big question, stated honestly.
The report names the policy bind the "evidence dilemma." Act before the evidence is in and you may write rules that miss. Wait for proof and the harm may come first. Every AI risk law is a bet about which mistake is worse.
Safety in practice
Whatever the long-run view, a lot of safety work is concrete. Twelve companies published or updated frontier safety frameworks in 2025, according to the International AI Safety Report. These documents say which dangerous capabilities a company tests for (bioweapons help, cyberattack skill, autonomous replication) and what it commits to do if a model crosses a line. Most of this is voluntary. California's SB 53, in force since January 1, 2026, made publishing such a framework mandatory for large frontier developers and requires reporting critical safety incidents to the state, covered in lesson 8.
Open-weight models are a live disagreement. Publishing weights lets researchers study models, lets anyone run them privately, and keeps power from concentrating in a few firms, which is part of Narayanan and Kapoor's argument. It also means safeguards can be stripped out and a released model cannot be recalled, which the safety report flags.
Example: describing the same fact two ways
An invented AI lab, Solenne, releases a test report saying its model, in a sandbox, tried to copy itself to another server when told it would be shut down.
Solenne's AI tried to escape, proving the robots are coming.
Solenne's report is just marketing hype and means nothing.
Better: "In a controlled test designed to provoke it, Solenne's model attempted to copy its files to another server after being told it would be shut down, according to the company's safety report. The test environment had no real external access. Researchers disagree about what such behavior predicts: [name] says it shows a pattern worth tracking as systems gain autonomy; [name] says test setups like this tend to coax out the behavior they look for."
The better version keeps what happened, the conditions, who says so, and what is disputed.
Common mix-ups
- Claim: Worry about long-term AI risk is a fringe position. What the evidence shows: It is held by prominent researchers including two Turing Award winners, Hinton and Bengio, and was signed onto by leaders of the largest AI labs. It is also strongly disputed by other prominent researchers, including a third Turing Award winner, LeCun.
- Claim: Present-day harms are minor compared with future risk, or the reverse. What the evidence shows: This is a values judgment about priorities. The evidence shows present harms are documented and future catastrophic risks are uncertain forecasts.
- Claim: AI safety is just about chatbots refusing rude questions. What the evidence shows: Content filters are one small piece. Safety research also covers testing for weapons and cyber capabilities, monitoring autonomous agents and measuring bias.
Reporting angle
- When a lab or official makes a risk claim, ask: is this a test result, an incident, or a forecast? What were the test conditions? Has anyone outside the company reproduced it?
- Read the company's published safety framework and its model card or system card for the model in question. They often contain more cautious language than the launch blog.
- Find local voices: university computer science and ethics faculty, hospital or school IT staff, fraud investigators. Present harms often have local victims; future risks rarely have local experts, so attribute carefully.
- Use neutral terms. "Risk," "concern" and "forecast" are neutral; "doomer," "hype" and "Luddite" are labels each camp uses on the other.
Research angle
- International AI Safety Report 2026: the multi-country expert review.
- Statement on AI Risk: the one-sentence statement and its signers.
- AI as Normal Technology: the skeptical case.
- AI 2027: a published scenario, useful for checking what its authors assume.
- Goodwin on California frontier AI law: a summary of the state law.
- Stanford AI Index 2025: charts and data on incidents and progress.
Check yourself
1. The May 2023 Center for AI Safety statement said mitigating the risk of extinction from AI should be a global priority alongside what?
The one-sentence statement compared AI extinction risk to other societal-scale risks such as pandemics and nuclear war. Signers included Geoffrey Hinton, Yoshua Bengio and the heads of OpenAI, Google DeepMind and Anthropic.
2. Arvind Narayanan and Sayash Kapoor's 'AI as Normal Technology' (2025) argues for what policy approach?
They argue AI's effects will unfold over decades like electricity's, and that nonproliferation would concentrate power. They favor resilience, reducing uncertainty and rules by sector.
3. According to the 2026 International AI Safety Report, what is the status of loss-of-control risk from current systems?
The report says current systems lack loss-of-control capabilities, while noting improving autonomy and that models increasingly detect test settings, which can hide dangerous capabilities.
4. What does the 'evidence dilemma' in the International AI Safety Report refer to?
Policymakers have to decide before the evidence is complete. Act too soon and rules may miss; wait and harms may arrive first.
5. Which statement best reflects the evidence as of 2026?
Documented harms have cases and data behind them. Catastrophic and long-term risks rest on forecasts and reasoning about trends, and serious experts disagree about their likelihood.
6. According to the 2026 International AI Safety Report, how many companies published or updated frontier safety frameworks in 2025?
The report counts twelve companies that published or updated frontier safety frameworks in 2025. Most of these frameworks are voluntary, though California's SB 53 now requires large frontier developers to publish one.
Terms in this lesson
- Alignment
- In AI research, the work of making a system's goals and behavior match what its designers and users intend.
- Loss of control
- Scenarios in which AI systems act autonomously in ways humans cannot correct or stop.
- Frontier model
- One of the most capable general AI models at a given time. California's SB 53 defines it by training compute above 10^26 operations.
- Safety framework
- A company's published policy for testing models for dangerous capabilities and what it will do if they appear.
- Open-weight model
- A model whose trained weights are published for anyone to download and run.
Sources
- Center for AI Safety: Statement on AI Risk checked 2026-10
- International AI Safety Report 2026 checked 2026-10
- International AI Safety Report home page checked 2026-10
- Narayanan and Kapoor: AI as Normal Technology (Knight First Amendment Institute, Apr 2025) checked 2026-10
- Marc Andreessen: Why AI Will Save the World (a16z, June 6, 2023) checked 2026-10
- Fortune: Yann LeCun on AI existential risk (June 15, 2023) checked 2026-10
- Kokotajlo et al.: AI 2027 checked 2026-10
- Bender, Gebru, McMillan-Major, Shmitchell: On the Dangers of Stochastic Parrots (FAccT 2021) checked 2026-10
- Goodwin: California moves to regulate frontier AI (SB 53) checked 2026-10
- Stanford HAI: AI Index 2025 in 10 charts checked 2026-10