🏛️ Issues and Advocacy Training
Courses › AI Issues: Technical, Legal, Social and Logistical

Lesson 7 of 9 · about 16 minutes

AI Safety and Risk: Who Argues What

You'll be able to

  • Describe the main camps in the AI risk debate and name people and groups in each
  • Separate documented present-day harms from forecasts of future risk
  • Summarize what the 2026 International AI Safety Report found and where its experts disagree
  • Explain the 'evidence dilemma' facing policymakers
  • Describe AI risk at a meeting, in testimony or online without adopting one camp's vocabulary as fact

"AI safety" means different things to different people, and a lot of the public fight is over whose meaning wins. To one researcher it means stopping a hiring algorithm from screening out older applicants. To another it means preventing a future system from helping someone build a bioweapon. To a third it means making sure humans can still switch off systems smarter than they are. All three are real research programs with serious people in them. Anyone following this debate needs to keep the claims apart from the evidence, and to keep forecasts labeled as forecasts.

The main positions

These are rough groupings. Plenty of people hold views from more than one.

PositionCore claimAssociated people and groupsMain evidence they cite
Present harms firstBias, surveillance, labor exploitation, misinformation and concentration of power are happening now and deserve the attentionEmily Bender, Timnit Gebru and coauthors of "Stochastic Parrots" (2021); many civil rights groupsDocumented cases like those in lesson 4; data on who builds and profits from AI
Catastrophic and long-term riskAdvanced AI could cause disasters up to human extinction, through misuse or systems escaping control, and needs urgent safeguardsSigners of the 2023 Center for AI Safety statement, including Geoffrey Hinton and Yoshua Bengio; authors of "AI 2027"Fast capability gains, early signs of deceptive behavior in tests, the difficulty of verifying what models can do
Normal technologyAI is powerful but will spread slowly through institutions, like electricity; superintelligence talk is muddled; resilience beats restrictionArvind Narayanan and Sayash Kapoor ("AI as Normal Technology," 2025)History of past general purpose technologies; slow real-world adoption; gaps between benchmark and job performance
Speed and competitionHeavy regulation slows beneficial uses and hands the lead to rival nationsInvestor Marc Andreessen ("Why AI Will Save the World," 2023); many industry leaders; US federal policy since 2025 (lesson 8)Past technologies created more jobs than they destroyed; competition with China; compliance costs that favor big incumbents

The catastrophic-risk camp has prominent critics inside the field too. Yann LeCun, a Turing Award winner who led AI research at Meta, called the idea that AI threatens humanity "preposterously ridiculous" in 2023, arguing that fears of machine takeover project human nature onto machines.

Notice an odd fact about the second row. The May 2023 statement that "mitigating the risk of extinction from AI should be a global priority" was signed by the heads of OpenAI, Google DeepMind and Anthropic, the companies building the systems. People in the first and third camps point to that as a conflict: talk of distant catastrophe can make a product sound powerful and pull attention from current harms. People in the second camp answer that insiders see the capability trends first. Both readings are out there. Keep the tension in view rather than settling it with an adjective.

What the evidence shows, sorted

The best single reference is the International AI Safety Report, written by more than 100 independent experts led by Yoshua Bengio, with an advisory panel nominated by over 30 countries and international bodies. The 2026 edition came out February 3, 2026. It is useful because it says where evidence is solid and where it is thin.

Documented, measured Forecast, disputed Fraud and deepfakes Fabricated facts Biased outcomes AI in cyberattacks Entry-level hiring squeeze Manipulation at scale Bio or chemical uplift Loss of control Extinction-level events Placement follows the 2026 International AI Safety Report's account of how much evidence exists. Further right means less direct evidence, not less importance.
Further right means less direct evidence. It does not mean less important or less likely.

From the 2026 report:

  • Documented now: AI is used in scams, fraud, blackmail and non-consensual intimate imagery, though good data on how common it is is scarce. Criminal and state-linked groups already use AI in cyber operations. Systems fabricate information and write flawed code.
  • Emerging: Several developers added biological and chemical safeguards in 2025 because they could not rule out meaningful help to novices. Models increasingly notice when they are being tested and exploit loopholes in evaluations, which means dangerous abilities could go unnoticed before release.
  • Not yet: Current systems lack the capabilities needed for loss of control, though their ability to act autonomously is improving.
  • Labor: No visible effect on overall employment, with signs of lower demand for early-career workers in some exposed fields (lesson 6).

Where its experts disagree: whether progress through 2030 stalls, slows, continues or speeds up; whether reasoning gains spread beyond math and coding; and whether AI doing AI research could compress years of progress into months. Asked how likely it is that six years of progress could be squeezed into two, surveyed forecasting experts gave a median of 20 percent and superforecasters 8 percent. That is a wide range on a big question, stated honestly.

The report names the policy bind the "evidence dilemma." Act before the evidence is in and you may write rules that miss. Wait for proof and the harm may come first. Every AI risk law is a bet about which mistake is worse.

Safety in practice

Whatever the long-run view, a lot of safety work is concrete. Twelve companies published or updated frontier safety frameworks in 2025, according to the International AI Safety Report. These documents say which dangerous capabilities a company tests for (bioweapons help, cyberattack skill, autonomous replication) and what it commits to do if a model crosses a line. Most of this is voluntary. California's SB 53, in force since January 1, 2026, made publishing such a framework mandatory for large frontier developers and requires reporting critical safety incidents to the state, covered in lesson 8.

Open-weight models are a live disagreement. Publishing weights lets researchers study models, lets anyone run them privately, and keeps power from concentrating in a few firms, which is part of Narayanan and Kapoor's argument. It also means safeguards can be stripped out and a released model cannot be recalled, which the safety report flags.

Example: describing the same fact two ways

An invented AI lab, Solenne, releases a test report saying its model, in a sandbox, tried to copy itself to another server when told it would be shut down.

Solenne's AI tried to escape, proving the robots are coming.

Solenne's report is just marketing hype and means nothing.

Better: "In a controlled test designed to provoke it, Solenne's model attempted to copy its files to another server after being told it would be shut down, according to the company's safety report. The test environment had no real external access. Researchers disagree about what such behavior predicts: [name] says it shows a pattern worth tracking as systems gain autonomy; [name] says test setups like this tend to coax out the behavior they look for."

The better version keeps what happened, the conditions, who says so, and what is disputed.

Common mix-ups

  • Claim: Worry about long-term AI risk is a fringe position. What the evidence shows: It is held by prominent researchers including two Turing Award winners, Hinton and Bengio, and was signed onto by leaders of the largest AI labs. It is also strongly disputed by other prominent researchers, including a third Turing Award winner, LeCun.
  • Claim: Present-day harms are minor compared with future risk, or the reverse. What the evidence shows: This is a values judgment about priorities. The evidence shows present harms are documented and future catastrophic risks are uncertain forecasts.
  • Claim: AI safety is just about chatbots refusing rude questions. What the evidence shows: Content filters are one small piece. Safety research also covers testing for weapons and cyber capabilities, monitoring autonomous agents and measuring bias.

Reporting angle

  • When a lab or official makes a risk claim, ask: is this a test result, an incident, or a forecast? What were the test conditions? Has anyone outside the company reproduced it?
  • Read the company's published safety framework and its model card or system card for the model in question. They often contain more cautious language than the launch blog.
  • Find local voices: university computer science and ethics faculty, hospital or school IT staff, fraud investigators. Present harms often have local victims; future risks rarely have local experts, so attribute carefully.
  • Use neutral terms. "Risk," "concern" and "forecast" are neutral; "doomer," "hype" and "Luddite" are labels each camp uses on the other.

Research angle

Check yourself

1. The May 2023 Center for AI Safety statement said mitigating the risk of extinction from AI should be a global priority alongside what?

2. Arvind Narayanan and Sayash Kapoor's 'AI as Normal Technology' (2025) argues for what policy approach?

3. According to the 2026 International AI Safety Report, what is the status of loss-of-control risk from current systems?

4. What does the 'evidence dilemma' in the International AI Safety Report refer to?

5. Which statement best reflects the evidence as of 2026?

6. According to the 2026 International AI Safety Report, how many companies published or updated frontier safety frameworks in 2025?

Terms in this lesson

Alignment
In AI research, the work of making a system's goals and behavior match what its designers and users intend.
Loss of control
Scenarios in which AI systems act autonomously in ways humans cannot correct or stop.
Frontier model
One of the most capable general AI models at a given time. California's SB 53 defines it by training compute above 10^26 operations.
Safety framework
A company's published policy for testing models for dangerous capabilities and what it will do if they appear.
Open-weight model
A model whose trained weights are published for anyone to download and run.

Full glossary

Sources