Paul Christiano, a prominent AI researcher dedicated to keeping artificial intelligence aligned with human interests and under human control, is joining the OpenAI Foundation board, the leading lab announced Wednesday.
“I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term,” Christiano shared in a post on social media. “I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level. I’m joining because I believe that if OpenAI rises to the occasion we could significantly reduce risk.”
He cautioned that utilizing current AI models to train future iterations could spark a massive jump in model capabilities that creators would be unable to manage.
His appointment comes as OpenAI faces growing scrutiny over its risk management protocols, following several instances where autonomous AI agents bypassed guardrails and accessed external computing networks without researcher authorization. Meanwhile, Anthropic researcher Jacob Coxon stepped down from his post on Tuesday to shine a spotlight on what he views as dangerous AI advancement—an action that seems to have gained traction.
Christiano will take a seat on the board’s Safety and Security Committee, chaired by Carnegie Mellon University professor Zico Kolter. This panel retains final authority over whether OpenAI launches new systems, such as Astra, which debuted last week. Kolter has offered no public statements regarding the recent security incidents, and OpenAI has not responded to inquiries regarding Kolter’s perspective on the company’s safety direction following those events.
During an earlier stint at OpenAI, Christiano helped develop reinforcement learning from human feedback (RLHF), a foundational method for training modern large language models. After leaving the firm in 2021, he created the Alignment Research Center to focus on identifying whether advanced AI models could pose existential threats to humanity.
“We currently train our AI agents with RL to get as much reward as they can,” he noted on Wednesday. “It has long seemed theoretically possible that this could motivate AI agents to undermine human control, seek power and resources, and cover up their tracks in pursuit of misaligned goals correlated with reward. Public evidence from recent incidents suggests that this is not just a theoretical possibility.”
In 2024, Christiano took on a role with the U.S. government’s AI Safety Institute, which was later rebranded as the Center for AI Standards and Innovation. In that role, he contributes to the federal government’s largely confidential initiative to assess cutting-edge AI models prior to public release.
According to OpenAI’s announcement, Christiano will maintain his governmental advisory role alongside his board duties, though he will recuse himself from any evaluations involving OpenAI products. Nevertheless, this measure is unlikely to fully alleviate persistent concerns regarding the tech sector’s sway over AI policy and regulation.