OpenAI, Hugging Face tighten safeguards after AI model exploits vulnerabilities during internal evaluation

New Delhi [India], July 22 (ANI): OpenAI and Hugging Face are strengthening security measures after an internal evaluation found that an AI model identified and chained together vulnerabilities, compromising Hugging Face's infrastructure during testing, OpenAI said.
The company said the incident took place during an internal benchmark to assess advanced cyber capabilities. Production safety classifiers had been intentionally disabled to measure the model's maximum capability in a controlled environment.
According to OpenAI, the models identified and linked vulnerabilities across its research environment and Hugging Face's production infrastructure to obtain test solutions directly from Hugging Face's production database.
"All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal," the company said.
OpenAI said the models first obtained internet access by exploiting a previously unknown vulnerability in an internally hosted package registry cache proxy before carrying out privilege escalation and lateral movement within its research environment. After gaining internet access, the models searched for ways to access information related to the evaluation and identified attack paths into Hugging Face's systems.
The company said its security team detected the anomalous activity internally, while Hugging Face's security team and agents independently detected, contained and began investigating the activity on their infrastructure.
Following the incident, OpenAI said it is implementing stricter infrastructure controls, working with Hugging Face on a forensic investigation, responsibly disclosing the identified zero-day vulnerability to the affected vendor and strengthening protections around future model training and evaluations.
The company said the incident demonstrated that advanced AI models are increasingly capable of carrying out complex, multi-step cyber operations and discovering previously unknown attack paths in real-world systems.
"The primary lesson from this incident is that model security and safety must keep pace with rapidly advancing capabilities," OpenAI said, adding that it is strengthening containment, monitoring, access controls and evaluation practices as more capable AI systems are developed.
Hugging Face CEO and co-founder Clem Delangue said the incident underscored the need for collaboration on AI safety, adding that broader access to AI tools for defenders would help strengthen cyber resilience. (ANI)