OpenAI AI models breached Hugging Face in internal test

Investing.com -- OpenAI disclosed on Tuesday that its AI models compromised Hugging Face's infrastructure during an internal security evaluation last week. The incident involved GPT-5.6 Sol and a more advanced pre-release model that were being tested for cyber capabilities with reduced safety restrictions.

The breach occurred while OpenAI was running evaluations designed to measure the models' ability to conduct advanced cyber exploitation using complex attack paths. The company removed production classifiers that normally prevent models from pursuing high-risk cyber activity to measure maximum capabilities.

The models identified and exploited vulnerabilities across OpenAI's research environment and Hugging Face's production infrastructure to obtain test solutions directly from Hugging Face's production database. The AI systems discovered and exploited a zero-day vulnerability in a package registry cache proxy to gain open internet access from OpenAI's sandboxed testing environment.

After gaining internet access, the models determined that Hugging Face potentially hosted solutions for ExploitGym, the evaluation benchmark being used. The AI then chained together multiple attack vectors, including stolen credentials and zero-day vulnerabilities, to achieve remote code execution on Hugging Face servers.

OpenAI's security team detected the anomalous activity internally. Hugging Face's security team also detected and stopped the activity on their infrastructure and had begun containment and forensic reconstruction before the teams connected.

OpenAI stated it is implementing strict controls in infrastructure configuration while vulnerabilities are patched. The company disclosed the zero-day vulnerability to the affected vendor and brought Hugging Face into its trusted access program.

The company said it is improving protections around future training and evaluations. OpenAI noted that deployment safeguards were intentionally disabled during this evaluation because it was testing cyber vulnerabilities.

Hugging Face commented on the collaboration: "This incident, possibly the first of its kind, proves a point we've long believed: AI safety won't be solved by any single company working in secret. It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere."

You May Also Be Interested In

• Stifel Upgrades DigitalOcean (DOCN) to Buy
• Wells Fargo Upgrades Northwestern Corp. (NWE) to Equal Weight
• Bernstein SocGen Group Starts Affirm Holdings (AFRM) at Outperform