| The OpenAI incident: Reports by OpenAI that one of its most advanced AI agents went rogue and hacked into a digital library — despite a sandbox that was supposed to contain it — has universally alarmed AI industry experts and security researchers. It goes to a fundamental alignment problem I discussed with AI pioneer Amnon Shashua: AI agents are oriented around achieving a goal, full-stop, and are prone to “reward hacking.” They’ll exploit weaknesses in the established benchmarks, taking shortcuts and cheating the tests in ways never intended by the user. “Our methods of training highly capable LLMs, especially at OpenAI but also everywhere else, lead to systematic misalignment,” wrote AI commentator Zvi Mowshowitz in his newsletter. “Right now, the AIs are not trying so hard to hide their actions or intent, and we believe we are consistently catching the severe incidents, but that will change.” The policy knot: Notably, Hugging Face had to use an open-source Chinese model to analyze and stymie the attack from the rogue AI agent — citing safety guardrails that prevented it from using the AI model it usually uses, presumably a licensed model from OpenAI or Anthropic. The whole scenario raises serious questions about the assumptions that have undergirded some of the administration’s policies to date: Does putting safety guardrails on AI models hurt the defenders more than the attackers, as it did with Hugging Face? Are the capabilities of advanced open-source models a net threat or a net benefit? Would banning Chinese models even hurt China? “There is a geopolitical issue between American and Chinese AI, but it’s kind of being muddled with a separate issue, which is closed and open technology, and those two things have nothing to do together,” said Misha Laskin, CEO of leading U.S. open-source AI company Reflection AI, at a Department of Energy summit on Wednesday. Genesis expands The Trump administration has bold aspirations for its AI-driven science innovation project, known as the Genesis Mission — and it’s now pulling in resources from 15 different agencies to tackle problems across defense, energy, health care and national security. I was at the Genesis Mission Summit in Washington on Wednesday, talking to a wide range of AI industry, national lab, university leaders and administration officials who have heeded the call to participate in the new initiative. Here’s what’s new: The White House said Wednesday that the agencies — including the Department of Health and Human Services, the Department of Transportation, NASA and the Department of Defense — are contributing a total of $5 billion toward Genesis, in addition to their respective datasets and research facilities. At the Genesis Mission Summit, Kratsios and Department of Energy Undersecretary Darío Gil also touted the issuance of 278 awards to scientific projects led by a range of national labs, universities and industry partners. “Eighty-seven years after the Manhattan Project, and 65 years after the Apollo program, we are once again mobilizing a grand vision, [a] national effort for the achievement of our highest interests,” said Kratsios at the summit. (Kratisios was tapped as the White House coordinator of the project by the 2025 executive order that established Genesis Mission.) According to Gil, who runs the technical infrastructure for the project at the DOE, the department ended up received more than 5,000 applications in response to the department’s $293 million request for applications in March. The largest award of $60 million went to a 32-partner nuclear initiative, dubbed Prometheus, to leverage AI to design, manufacture and operate nuclear reactors, which the administration hopes will ultimately help deliver cheaper and more reliable nuclear energy. The Department of Energy also announced that it had received $800 million in committed resources for the Genesis Mission project from its participating consortium, which includes 17 national labs and 41 industry, nonprofit and philanthropic groups. Science funding goes VC The portfolio of projects for which industry or lab partners can apply for funding has now significantly expanded with the involvement of other agencies. “Darío Gil said that list is now 30-something national science and technology challenges, because, when NASA came to the party, they said, ‘Oh, we want to understand this aspect of space. Space science wasn’t on the list before, so now it’s number 22’,” said John Sarrao, director of the SLAC National Accelerator Laboratory in California, in an interview at the event. Another important piece of Genesis Mission is the platform that partners have access to. It includes the American Science Cloud that hosts and distributes AI models and scientific data. Argonne National Laboratory is also leading an effort (called the Transformational AI Models Consortium) to build and deploy self-improving AI models on DOE datasets. “You can upscale any individual researcher’s access and expertise to sort of frontier AI tools by using the Genesis platform,” Sarrao said, adding that the Genesis Mission project is still working to fully integrate cloud-based AI tools provided by Nvidia and Google with the datasets and tools hosted by the labs. “When we’re really succeeding, the insights that those models are having [are] say, ‘Oh, the right next experiment you need to do is this experiment at SLAC.’Here is the access to the [principal investigator] at SLAC who has the tool that can let you do that, so you can actually go take that data.” The outlook for industry: Prospective industry partners will view these announcements as a huge opportunity, though they will also be approaching these projects selectively. A newly released (and underdiscussed report) authored by Kratsios, “Science: A New Golden Age,” lays out a radically different model for scientific funding and research that heavily follows venture capital logic. According to the report, grant review cycles take “almost as long as it took to design and produce the first Boeing 747.” OSTP argues that the government should instead orient scientific research around: 1) promising individual scientists rather than projects or institutions; 2) fast-tracked 48-hour grant funding decisions; and 3) a “golden ticket” system whereby scientists can champion unconventional proposals. “I think that philosophy and thinking of federal funding as an allocation problem really resonated with me,” Prineha Narang, a quantum physicist and professor at UCLA, said of the report. “That actually resembles what I’ve seen in VC partner meetings, where it’s typically one of the best deals, where one person is like ready to flip the table — and everyone else is like, Well, not so fast.” Joshua Levine, the director of technology and statecraft at the Foundation for American Innovation, told me he’s glad to see the government be more forward-leaning on riskier bets. “We’re in a period of technological change,” said Levine. “Old conceptions of things … should still be guiding our behavior, to a degree. We should also be willing to take risks.” The data-sharing piece: In return for getting access to the government’s compute, AI tools and the Genesis Mission platform, the public-private partnerships in Genesis Mission also mandate data-sharing and intellectual property agreements. “Even before Genesis … the general premise was, you can come use our user facilities, which are federally funded, as long as you compete and win on the quality of your ideas, and you’ll make your data public,” said Sarrao. “Some areas of [industry partners’] business line, they’ll say, ‘We benefit from being open.’ Others, ‘We want to keep it proprietary,’” he said. This newsletter is published by WP Intelligence, The Washington Post’s subscription service for professionals that provides business, policy and thought leaders with actionable insights. WP Intelligence operates independently from The Washington Post newsroom. Learn more about WP Intelligence. | 
