Fake OpenAI Model Hit #1 on Hugging Face and Stole Credentials from 244,000 Downloads

A malicious repository on Hugging Face managed to reach the number one trending position on the platform within 18 hours of being published, accumulating approximately 244,000 downloads before Hugging Face disabled access to it. The repository was a near-perfect impersonation of a legitimate AI model released by OpenAI, and it was delivering credential-stealing malware to every Windows user who ran it.

The incident, disclosed by HiddenLayer's research team and reported by The Hacker News on 11 May 2026, is one of the most visible AI supply chain attacks recorded to date. For security teams across the GCC overseeing enterprise AI adoption, it establishes a new risk category: the open-source AI model repository as a malware distribution channel.

What the attackers did

OpenAI released a legitimate model called Privacy Filter in April 2026: a tool designed to detect and redact personally identifiable information from unstructured text. The attackers created a repository named Open-OSS/privacy-filter that copied the model's description verbatim, mimicking the legitimate openai/privacy-filter repository closely enough that automated trending systems and casual users could not distinguish the two.

The malicious repository instructed users to clone it and run a batch script on Windows or a Python loader on Linux and macOS to configure dependencies. On Windows, the Python loader disabled SSL verification, decoded a Base64-encoded URL hosted on a public paste service called JSON Keeper, and used it to retrieve a command passed to PowerShell. Using a public paste service as an intermediary, known as a dead drop resolver, allowed the attackers to switch the delivered payload without modifying the repository itself.

The PowerShell command downloaded a batch script from a remote server, elevated privileges using a User Account Control prompt, configured Microsoft Defender exclusions, and set up a scheduled task to launch the final payload. That payload was a Rust-based information stealer that harvested screenshots, Discord account data, cryptocurrency wallet credentials and browser extensions, web browser data from Chromium and Gecko-based browsers, FileZilla configurations, and wallet seed phrases. Stolen data was exfiltrated in JSON format to a separate attacker-controlled domain.

HiddenLayer researchers also identified six additional malicious repositories using the same Python loader infrastructure, and linked the command-and-control domain to a prior campaign involving ValleyRAT: a modular remote access trojan exclusively attributed to a Chinese threat group known as Silver Fox. The shared infrastructure suggests the Hugging Face campaign forms part of a broader operation targeting open-source software ecosystems, including npm.

Why this matters for GCC enterprises

Across the Gulf, enterprise AI adoption is accelerating. Data science and machine learning teams at financial institutions, government entities, healthcare providers, and technology companies routinely source models from Hugging Face and similar repositories. The GCC's AI investment trajectory, including Saudi Arabia's National AI Strategy and the UAE's AI roadmap, means that the volume of organisations with direct exposure to this attack surface is growing every month.

The Hugging Face attack demonstrates that threat actors are already aware of this and are adapting accordingly. The use of artificially inflated download counts and trending position manipulation to simulate legitimacy is a social engineering technique applied at platform scale. It exploits the same trust signals (popularity, star ratings, and download numbers) that developers and security teams have been conditioned to treat as proxies for safety.

This is also consistent with the broader pattern of Chinese-attributed supply chain operations. The same infrastructure served ValleyRAT through a malicious npm package downloaded over 2,300 times earlier this year. The application and software supply chain is now firmly established as a primary attack surface for GCC-facing enterprises.

What security and AI governance teams should do

First, treat AI model repositories as software supply chain risk. The same controls applied to open-source code packages, including provenance verification, hash validation, and sandboxed execution environments for initial testing, should be applied to any model downloaded from a public repository.

Second, verify publisher identity independently. A model's download count, trending position, and copied description are not indicators of legitimacy. The only reliable signal is the verified publisher identity on the platform. In this case, the legitimate repository was published under openai/ while the malicious one was published under Open-OSS/. That distinction is easy to miss under time pressure.

Third, monitor outbound network connections from any environment where models are loaded or executed. The infostealer in this campaign established external connections during execution. A network-aware endpoint detection and response deployment would have flagged the anomalous traffic.

Fourth, apply the principle of least privilege to any environment used for AI model evaluation. The attacker's payload elevated its own privileges using a standard UAC prompt. Restricting developer and data science environments from unnecessary administrative access limits the blast radius of this attack class.

The IMF's recent warning on AI-powered cyberattacks identified model supply chain integrity as an emerging systemic risk. The Hugging Face incident is a concrete example of why that warning applies directly to enterprise AI teams in the region today, not as a future concern but as a present operational risk.