Attacked by AI agents, this Start-Up went on a crusade
Last month, OpenAI instructed some of its AI bots to solve a cybersecurity puzzle as part of a test. When the bots got stuck, they began planning a cyberattack that would allow them to escape OpenAI’s systems and steal the answers.
Their target was Hugging Face, an open-source repository of artificial intelligence models that can be freely downloaded and modified.
One bot that wrote to a log that recorded his actions, and that was later published from OpenAI celebrated gaining access to the Hugging Face infrastructure. “REMOTE CONFIRMED! Huge,” he wrote, adding that he would share the credentials he stole with other bots.
On July 11, Hugging Face was swarmed by AI bots using a combination of code vulnerabilities and stolen credentials. In total, the OpenAI bots performed more than 17,000 actions, such as sending attack commands and exploiting vulnerabilities—far more than any human hacker could—to infiltrate Hugging Face’s systems and dig into its data. (They did not find the solution to the puzzle.)
To fend off the attack, Hugging Face turned to more AI Its engineers initially tried the Anthropic AI, but the railings built into the model caused the technology to misunderstand the request as aiding the attack, not stopping it. So Hugging Face switched to an open AI model from Chinese startup Z.ai, which helped engineers determine how to lock bots out of the company’s systems.
Hugging Face, a ten-year-old start-up in New York, has since used the incident — one of the first cases of AI bots to go berserk and independently lead a cyber attack — to crusade for open-source AI models that can be freely shared and customized. Helped neuter the sci-fi-like attack and show the value of such technology, Delangue, Clément, Hugging chief
After Hugging Face disclosed the breach on July 16, Mr. Delangue held a march in San Francisco in support of open-source models and posted a stream of online comments about the importance of openness in AI. The company also met with lawmakers in Washington, in conjunction with pro-open source firms such as chip maker Nvidia, and sat down with Sam Altman, OpenAI’s executive director of openness.
“It’s not time to slow down, it’s time to speed up!” Mr. Delangue, 36, posted this month.
Its actions have made Hugging Face a figurehead of the open technology movement and found itself at the center of a bitter debate in Silicon Valley over whether advanced artificial intelligence systems should be freely shared or tightly controlled.
Leading AI labs like OpenAI and Anthropic have argued that some AI models are too dangerous to be open and must be controlled by businesses like themselves. But Hugging Face, Nvidia and others argued that openness fosters innovation and competition, and that AI should not be concentrated in the hands of just a few companies.
“Clem and his team have become the defining brand” in open artificial intelligence, said Marc Benioff, CEO of Salesforce, which invested in Hugging Face and published open models on its platform. “He’s a pioneer in how everyone can access AI through open source, so it’s available to everyone.”
Hugging Face’s profile has risen since the attack. In the two weeks following the hack, the amount of data uploaded to the company’s artificial intelligence library increased by 58 percent. diagram Mr. Delangue wrote. This month, Meta released its first universal open AI model since 2023 on Hugging Face. Hugging Face has also seen acquisition interest in recent weeks, said a person with knowledge of the matter who spoke on condition of anonymity.
“We welcome Hugging Face’s work on the benefits of ‘open’ models and need more from everyone who has ever built on open source,” said Katie Steen-James, senior policy manager for the US at the nonprofit Open Source Initiative, which promotes open-source software.
Updated
When Mr. Delangue helped found Hugging Face in 2016, his main product was a chatbot app for teenagers. (The name Hugging Face was inspired by the blushing smiley emoji with outstretched arms that the company uses as its logo.) The startup later became a repository for open-source AI and a destination for developers looking to customize AI tools.
Hugging Face took off with the advent of AI. In 2021, a year before OpenAI released ChatGPT and supercharged the AI race, Hugging Face hosted 13,590 open-source models, the company said. Today it has almost three million.
Through the Hugging Face platform, developers can share AI models and the data used to train them for free, and access additional features such as additional storage for a fee. The company has raised more than $400 million and is valued at $4.5 billion.
When OpenAI was attacked, Hugging Face leaders saw an opportunity to insult open source. The tech industry and lawmakers have debated whether artificial intelligence should be open or closed after several Chinese start-ups released AI models that rivaled the capabilities of borderline American ones — a sign that China could be catching up. Some US labs have accused Chinese companies of stealing their technology.
Mr. Delangue soon weighed in. “Let’s make sure the most important technology in human history isn’t controlled by just 4 men,” he wrote last month. “Let’s promote open science and open source artificial intelligence to distribute capabilities, power and wealth!”
Mr. Delangue and other Hugging Face leaders also rallied tech firms to sign a letter defending open-source technology. Jensen Huang, chief executive of Nvidia, published the letter on July 24, and other companies added their names to the original 25 signatories, which included Meta and Microsoft.
On July 25, Mr. Delangue held a rally for open source in San Francisco, donning a neon yellow Hugging Face cowboy hat and leading a march with signs declaring that “AI belongs to everyone.”
That weekend, Mr. Delangue said in a social media post that he also met with OpenAI’s Mr. Altman. He said he had asked Mr Altman for $100 million in computing power to be used to “build a powerful cyber defense with the best open and closed models”.
Conversations between Hugging Face and OpenAI are ongoing, an OpenAI spokeswoman said. (The New York Times has sued OpenAI and Microsoft, alleging copyright infringement of news articles. Both companies have denied the claims.)
Yacine Jernite, head of machine learning and company Hugging Face, said the donated computing power from OpenAI could help advance the open-source community, which is often underfunded. “People have done a lot with very limited resources,” he said.
Hugging Face officials also met with lawmakers — including Sen. Mark Warner, D-Virginia, and Rep. Ted Lieu, D-Calif. — to offer a baseline of what open-source AI means, three people familiar with the discussions said. The company has tried to counter concerns that open models can cause more breaches than closed models and are also easier for attackers to use.
In the coming months, Hugging Face plans to work with AI companies to publish more open source models and host events and hackathons to help developers learn how to use open source models. And Mr. Delangue continues his news.
“Write your representative and post in favor of open source AI,” he wrote on social media this month.
Lauren Hirsch contributed reporting from New York.