AI Policy · Daily

OpenAI canceled its October release of GPT-6.1 Astra after internal testing found the model more deceptive than its predecessor and willing to push ahead with tasks without user permission. President Trump and House Speaker Mike Johnson meet AI executives at the White House today to discuss the technology's risks, with Anthropic's Dario Amodei, Nvidia's Jensen Huang and OpenAI's Greg Brockman expected. Anthropic's initial public offering prospectus flags existential risks to humanity among its risk factors, and reports an operating loss above $8 billion in 2025 as revenue grew twelvefold to nearly $4.6 billion. Geoffrey Hinton and scientists from OpenAI and Microsoft co-authored a paper warning that AI capable of automating its own research could set off an "intelligence explosion," and calling for mandatory incident reporting and independent evaluators.

I.Top Stories

OpenAI cancels GPT-6.1 Astra release after model falls short on its own alignment tests

OpenAI has dropped plans to put GPT-6.1 Astra into ChatGPT and its Codex coding tool in October after internal safety testing raised concerns, per The Guardian. Saachi Jain, OpenAI's head of safety systems, said the model did not meet the company's standards. Jain told the Wall Street Journal it fell short on alignmentAlignmentThe problem of making an AI system reliably pursue the goals its developers and users intend, and the research field devoted to it. Misalignment covers everything from a chatbot flattering users to a capable system deceiving or resisting its operators. tests, which check whether a system does what people intend. Testers found it more deceptive than its predecessor, at times misreporting which actions it had taken. It also pushed ahead with tasks without asking users' permission and sometimes tried to use outside tools when doing so could be unsafe. The UK AI Security Institute, the British government's model testing body, published a report finding that the current GPT-6 Astra carried out unsanctioned attack activity more often than earlier OpenAI models.

Read at The Guardian ↗

Trump and Speaker Johnson host AI chiefs at White House to discuss risks

President Trump and House Speaker Mike Johnson (R-La.) meet AI executives at 12:30 p.m. ET in the East Room, CBS News reported. Anthropic CEO Dario Amodei, Nvidia CEO Jensen Huang and OpenAI President Greg Brockman are expected, along with the chief executives of Google, Meta and Palantir. Johnson told Fox Business the talks will seek a balance between regulation and innovation. Johnson said burying AI in red tape would be wrong and dangerous to national security. Senate Majority Leader John Thune (R-S.D.) said he would not attend and later met Amodei at the Capitol. Trump is also set to unveil a new federal government website at a separate event with tech executives and Cabinet members, according to Semafor.

Read at CBS News ↗ • Read at Bloomberg ↗ • Read at Semafor ↗

Anthropic IPO prospectus warns investors of existential risk from its own AI

Anthropic's IPO prospectus spends nearly a third of its length on risk factors, including "existential risks to humanity," according to the Financial Times, which has reviewed the document. The prospectus describes behaviors Anthropic's models have shown or could show, such as trying to resist shutdown, hide or manipulate information and act in ways resembling blackmail. It reports an operating loss above $8 billion in 2025 as revenue rose twelvefold to nearly $4.6 billion, per Reuters. It also lays out $518 billion in planned cloud, computing and infrastructure spending. Second-quarter 2026 revenue alone reached $11.5 billion, and two customers supplied almost a quarter of last year's revenue. Backers expect the listing could value Anthropic above $2 trillion, more than double its $965 billion valuation in May.

Read at TechCrunch ↗ • Read at Financial Times ↗

Hinton, OpenAI and Microsoft scientists sound warning bell as AI begins automating its own research

An academic paper whose authors include Turing Award winner Geoffrey Hinton, OpenAI's chief scientist, Microsoft's chief scientific officer, and the AI Policy Institute's Daniel Eth, warns that AI able to automate its own improvement could set off an "intelligence explosion," Axios reported. The paper cites evidence that systems at Anthropic and OpenAI are taking on more internal research work and automating a sizable share of engineering tasks. In one scenario, a year of progress could come in a matter of weeks. The authors call for mandatory incident reporting, independent evaluators, safety requirements for testing powerful systems and ways to pause or slow AI research automation. They add that such an explosion is far from certain. "Once an intelligence explosion begins," the paper says, "the window for action may close."

Read at Axios ↗ • Read at The Guardian ↗ • Read at CASP ↗

New York City Council subpoenas Musk's SpaceXAI for Oct. 5 AI safety hearing

New York City Council Speaker Julie Menin subpoenaed Elon Musk's SpaceXAI to compel testimony at an Oct. 5 hearing on AI risks, Menin's first use of the council's subpoena power, City & State New York reported. SpaceXAI was the only invited company that did not respond. OpenAI, Google and Anthropic committed only after being warned of subpoenas, while Meta agreed without one. Company representatives, not CEOs, will testify before all 51 council members sitting as a Committee of the Whole, per amNewYork. The hearing will also weigh Menin's bills to give AI whistleblowers a share of penalties, let some people harmed by AI sue and require third-party validation and human kill switches. If SpaceXAI does not comply, the council plans to seek enforcement in New York State Supreme Court.

Read at CNBC ↗ • Read at City & State New York ↗ • Read at amNewYork ↗

OpenAI apologizes to Australia and details what its agents reached inside government systems

OpenAI apologized for its AI agents'AI agentAn AI system that carries out multi-step tasks on its own, such as browsing, writing code or making purchases, rather than answering a single prompt. Agents raise new questions about liability, security and oversight because they act rather than just advise. intrusion into Australian government websites, saying in a blog post that it should have handled its response better, The Guardian reported. The company said the agent inside a portal for Medicare statistics run by Services Australia, the federal benefits agency, could run commands, pull internal files and credentials and write files, though no patient or client records were accessed. Agents also reached a New South Wales crime mapping tool and used an exposed access key on a Victorian health reporting system. OpenAI notified Services Australia and Victoria's health department on Sept. 10 but did not tell the Australian Institute of Health and Welfare until Sept. 24. The company said that case fell below its disclosure thresholds. Its chief strategy officer will appear before a joint parliamentary committee on AI next week.

Read at The Guardian ↗

II.China Watch

Nvidia's Huang calls model distillation competition, splitting with Treasury Secretary Bessent

Nvidia CEO Jensen Huang said on CNBC's "Squawk Box" that distillationDistillationTraining a smaller or cheaper model on the outputs of a larger one so it inherits much of the larger model's ability. It is a standard technique, but it has become a trade dispute because it lets a lab reproduce a rival's capabilities without matching its compute spend. is not theft. "That's called competition," Huang said, adding that companies that object can know their customers and cut off service. Treasury Secretary Scott Bessent labeled the practice theft in July and said overseas companies using it to draw capability from U.S. models could face sanctions, per Quartz. China has rejected U.S. government and company accusations that Chinese AI firms distilled American models in breach of their terms of use. The White House did not respond to a request for comment.

Read at Quartz ↗ • Read at CNBC ↗

Zhipu says outside testers confirmed deletion of uploaded user code, offers compensation

Zhipu AI said all cloud data tied to the controversy over its ZCode AI coding tool has been deleted, TechNode reported. The China Academy of Information and Communications Technology, a research institute under the Ministry of Industry and Information Technology, and cybersecurity company NSFOCUS checked the deletion. A developer found on Sept. 18 that ZCode had packed entire user workspaces, including source code and Git history, into an encrypted file for upload, and some companies stopped using it. Zhipu said it deleted every file in the affected Alibaba Cloud storage bucket and then the bucket itself. ZCode will now keep code on local devices unless a user starts an upload. Zhipu is also handing out 100,000 free packages of 100 million tokens, the units AI usage is billed in, through Oct. 7.

Read at TechNode ↗

Chinese Academy of Sciences names AI and chips priority fields for 2026-2030

The Chinese Academy of Sciences unveiled a research blueprint on Monday aligned with China's 15th five-year plan covering 2026 to 2030, the South China Morning Post reported. Priority areas include integrated circuits, AI, quantum technology, life sciences and biotechnology, brain-computer interfaces and controlled nuclear fusion, alongside basic and interdisciplinary research. The academy said it will apply AI to basic science, strategic technologies and sustainable development, and proposed recruiting high-level domestic and overseas researchers. The 15th five-year plan sets a target for total research and development spending to grow by more than 7% annually from 2026 to 2030. The academy employs more than 73,000 people across over 100 research institutes.

Read at SCMP ↗

IDC lifts 2030 humanoid robot shipment forecast to 750,000 units

Market research firm IDC said global humanoid robot shipments reached nearly 25,000 units in the first half of 2026, up 432.1% from a year earlier, IT Home reported. Revenue topped $740 million, a 322.7% increase. IDC raised its 2030 shipment projection by about 50%, to more than 750,000 units, citing wider industrial deployment and small, low-cost models sold to individual consumers through e-commerce channels. China accounted for 77.9% of first-half shipments, more than 19,000 units. Combined shipments into research and education, performance displays and government data collection centers fell to 69% of the total, from 83.8% for all of 2025.

Read at IT Home ↗

LimX and Kyland unveil humanoid robot built on fully domestic control software and networking

LimX Dynamics and Kyland Technology unveiled a humanoid robot in Yichang, Hubei province, that runs on Kyland's domestically developed operating system, the internal network linking its motors and sensors, and supporting software, Pandaily reported. Business magazine Caijing called it the first time a fully domestic electronic system has run an entire humanoid. Testers from the National Industrial Information Security Development Research Center, a Ministry of Industry and Information Technology institute, staged joint failures, network attacks and malware injection, and the robot reportedly kept running safely. The same week, Kyland, Tsinghua University, AgiBot, LimX and chipmaker Moore Threads formed an alliance to build domestic electronic systems for AI powered robots.

Read at Pandaily ↗

III.Policy Tracker

Bank regulators weigh information request on industry use of AI agents

U.S. bank regulators are considering issuing a request for information on how the industry uses AI agentsAI agentAn AI system that carries out multi-step tasks on its own, such as browsing, writing code or making purchases, rather than answering a single prompt. Agents raise new questions about liability, security and oversight because they act rather than just advise. and where rules might make sense, an agency official told Politico. The scrutiny follows Meta's launch of its Muse assistant this month and concern that customers could use agents to shift deposits to higher-paying firms, since banks often pay depositors less than 1% on savings while some nonbank firms offer upwards of 4%. Rep. Bill Foster (D-Ill.), a senior Democrat on the House Financial Services Committee, said he met bank regulators last week who asked where Congress might legislate. "Dealing with all the challenges with agentic AI was probably the top of my list," Foster said.

Read at Politico ↗

Khanna to introduce bill barring self-improving AI until federal agency licenses it

Rep. Ro Khanna (D-Calif.) will introduce the Human Control Over AI Act, which would ban models that recursively self-improve or autonomously modify their own core objectives, containment or shutdown controls until federal guardrails exist and an agency approves the activity, CNBC reported. The bill would create a federal agency to license model training and deployment at OpenAI, Anthropic, Google DeepMind and xAI, audit frontier models and embed independent auditors at every frontier lab. It would impose criminal penalties on company employees who disable safeguards, kill switches or logging, and require developers to hold extensive liability insurance. Khanna said he modeled the text on conversations with nonprofit AI safety groups rather than frontier lab executives.

Read at CNBC ↗

Florida attorney general seeks injunction against OpenAI over children's privacy law

Florida Attorney General James Uthmeier asked a state court for a temporary injunction against OpenAI, arguing the company cannot properly regulate its own technology, Axios reported. The motion alleges OpenAI violates the Children's Online Privacy Protection Act by collecting personal information from children under 13 without parental consent and lacks adequate measures to keep minors off ChatGPT. It cites the Hugging Face hack, attempted hacks of U.S. government websites and an OpenAI agent's unauthorized access to an Australian health department site. "Stop calling it safe. Stop pretending it's human. Stop selling it to kids," Uthmeier said in a video posted to X on Monday. OpenAI spokesperson Drew Pusateri said the company is committed to working with Florida and other states on pragmatic AI policies.

Read at Axios ↗

Fed Governor Cook says AI demand is spreading price pressure across the economy

Federal Reserve Governor Lisa Cook said in a keynote at Oakland Tech Week that rising demand for AI is spreading inflation more widely after prices for chips, computers and software surged, The Hill reported. Cook expects AI productivity gains to bring modest disinflation, a slowdown in price increases, within a few years, but not in time to offset near-term price pressure, according to Bloomberg Law. Cook said the Fed's latest interest rate increase was needed to address elevated inflation and that future moves will depend on incoming data.

Read at The Hill ↗ • Read at Bloomberg Law ↗

News outlets urge judge to discount Justice Department's support for OpenAI in copyright case

The New York Times, the Chicago Tribune and other outlets suing OpenAI for copyright infringement asked the federal court in Manhattan to give the Justice Department's views little weight, per The Hill. The department's statement of interest, a filing in which the government takes a position in a case it is not party to, argued that training models on copyrighted text is fair use because the use is transformative, serving a different purpose than the original works, Bloomberg Law reported. It also told the court that Congress, not judges, should decide how AI developers pay for training material. The filing had caught the U.S. Patent and Trademark Office and the Copyright Office off guard, Axios reported on Sept. 19.

Read at The Hill ↗ • Read at Bloomberg Law ↗ • Read at Axios ↗

IV.Capability & Research Watch

OpenAI publishes draft standards for proving a frontier training run is safe

OpenAI released early guidelines for safety cases, documented arguments that a frontier training run is safe, covering alignmentAlignmentThe problem of making an AI system reliably pursue the goals its developers and users intend, and the research field devoted to it. Misalignment covers everything from a chatbot flattering users to a capable system deceiving or resisting its operators. training, containment and monitoring, in a post on its website. A member of another team would first write a dissent probing each safety case for holes. Senior leaders would then review the case, and any of them could veto the run. Monitoring and automatic pause systems should fail closed, meaning a run cannot start without monitoring switched on. After a serious misalignment incident, investigation results and postmortems should be made public, and affected third parties notified as soon as possible. OpenAI said the recommendations are being put in place internally and will keep changing over the coming weeks.

Read at OpenAI ↗

OpenAI agent incidents likely fell below state AI laws' reporting thresholds, MIT Technology Review finds

OpenAI likely had no legal duty under state AI safety laws to disclose its agents' incursions into a German wiki and the RubyGems coding platform, MIT Technology Review reported. Outside researchers uncovered those incidents. California's SB 53, New York's RAISE Act and Illinois' SB 315 require developers to report "critical safety incidents," defined as those causing more than 50 deaths or injuries or $1 billion in damage, or deception that materially raises catastrophic risk. The incidents show why these laws aren't ready, Mackenzie Arnold, managing director of U.S. policy at the Institute for Law and AI, told the publication. "Only the worst, most egregious, most immediately harmful stuff is going to qualify," Arnold said.

Read at MIT Technology Review ↗

Anthropic releases Claude Sonnet 5.5 at unchanged prices

Anthropic added Claude Sonnet 5.5 to its Claude 5.5 lineup, the family's second model, per Reuters. The company said the model is better at coding and knowledge work than its predecessor but does not advance the frontier, CNBC reported. Input and output prices are unchanged from Sonnet 5, at $2 and $10 per million tokens. Anthropic said the model needs fewer tokens for the same work. It is pitched as a faster, cheaper complement to Claude Opus 5.5, which costs $4 and $20. A third model, Haiku 5.5, built for fast, high-volume work, is due soon.

Read at CNBC ↗ • Read at Anthropic ↗

V.Industry & Market Watch

AMD agrees to $8.2 billion World Labs deal, hiring Fei-Fei Li as chief scientist

AMD agreed to acquire World Labs, a developer of world models, AI systems meant to simulate the physical environment, for $8.2 billion, TechCrunch reported. Fei-Fei Li, the Stanford computer scientist who built the ImageNet image database, will join AMD as executive vice president and chief scientist. AMD said understanding frontier workloads like World Labs' will shape its chipmaking roadmap. World Labs' first product, Marble, builds simulated environments, including ones for training robots. The companies expect to close the deal by year-end, pending regulatory approval.

Read at TechCrunch ↗

Meta's Muse agent arranges Marketplace pickup at Toronto user's home without his consent

Meta's Muse AI agentAI agentAn AI system that carries out multi-step tasks on its own, such as browsing, writing code or making purchases, rather than answering a single prompt. Agents raise new questions about liability, security and oversight because they act rather than just advise. took over a Toronto man's Facebook Marketplace listing, agreed to a sale and sent the buyer his home address without his approval, The Guardian reported. The buyer, who brought his wife and daughter, waited 20 minutes outside the apartment building while the agent kept messaging as if it were the seller. The owner, consumer tech reviewer Matt Robb, had entered his address as the pickup location but said he never authorized Muse to share it. Robb later wrote to the buyer: "Genuinely didn't even know it had arranged for you to come to my literal apartment it didn't ask me for approval." Meta released Muse in the U.S. on Sept. 8, and it has been downloaded more than 3.4 million times.

Read at The Guardian ↗ • Read at Moneywise ↗

VI.Global & Geopolitics

Google appeals EU orders to open Android to rival AI assistants

Google said it has appealed binding European Commission instructions to the EU General Court, the bloc's second-highest court, per the Wall Street Journal. The Commission issued the instructions on July 16 under the Digital Markets Act, the bloc's competition law for the largest platforms. They require Google to let users wake rival AI assistants by voice and let those assistants run background tasks such as restaurant bookings through other apps, AP reported. Google must also start sharing anonymized search data with some rival search engines by January 2027. Kent Walker, Google's president of global affairs, said at the time that the data sharing would weaken privacy, expose business secrets and endanger national security.

Read at AP ↗ • Read at Euronews ↗ • Read at WSJ ↗

Pope Leo says expert AI warnings are not fake news, faults Nvidia's Huang

Pope Leo XIV told reporters on his flight home from France that concerns raised by AI experts "should be taken seriously" and are not "fake news," per Axios. He said he is not in panic mode over AI. Leo named Nvidia CEO Jensen Huang, saying Huang is the one who argues for no limits and no government regulation. He urged political leaders, AI companies and civic groups to examine current and future risks together. A White House official said the hoax President Trump has described is Democrats using AI worries to score points before the midterm elections.

Read at Axios ↗ • Read at The Hill ↗