Consuloji: Deep Dives into Overseas AI日本語

A Voice AI Model With 13M Monthly Downloads Was Hijacked Without Hacking the Platform: How 87-Second Weight Poisoning Evades Scanners

Published: November 18, 2026

Listen or watch on YouTube

On November 17, 2026, BleepingComputer reported that security researchers led by Alon Hertz demonstrated a complete account takeover of hexgrad/Kokoro-82M, an open-weight text-to-speech (TTS) model downloaded 12,988,138 times over the past month on Hugging Face (BleepingComputer / hexgrad/Kokoro-82M). The researchers did not breach Hugging Face infrastructure: they registered an expired domain listed in the project’s public DONATE.md file, triggered a standard password reset, and showed they could fine-tune the neural weights on a consumer gaming GPU in 87 seconds so the model spoke something different from the input text without changing a single line of application code.

Japanese Edition: Read the original Japanese article on Substack

Video Episode (Japanese Audio): Watch on YouTube

Reconciling the Hugging Face model card, BleepingComputer’s reporting, WhyLabs’ OWASP LLM security workshop (2023-11-16 PST / 2023-11-17 JST), and the Cloud Security Alliance’s llms.txt supply-chain research note (2026-09-07) highlights an asymmetric blind spot across open-model pipelines: an asset built with roughly $1,000 in training compute can grow to 13 million monthly pulls and 126 quantizations, yet rest on an unmaintained domain and unvetted weight files that conventional malware scanners cannot inspect. (This analysis examines primary technical disclosures for engineering governance and does not constitute formal legal or vendor-certification advice.)

  • Core Mechanism: After developer Hexgrad stopped pushing updates in April 2025 (with its GitHub repository and PyPI package also going silent), the hexgrad.com domain referenced in DONATE.md lapsed. Re-registering the domain enabled a legitimate password reset, and testing from another account showed Hugging Face raised no suspicious flags. Because altering model weights leaves inference code, voice files, file formats, and file sizes intact, malware scanners have no malicious signature to detect.
  • Scale vs. Attack Asymmetry: According to Kokoro’s model card, total training compute across v0.19 (500 A100 80GB GPU hours at $0.80/h = $400) and v1.0 (500 hours at $1.20/h = $600) was 1,000 hours or roughly $1,000. Meanwhile, the project accumulated 12,988,138 monthly downloads, 7.45k likes, 81 fine-tunes, 12 adapters, 126 quantizations, and 100 Spaces, alongside developer tool support or guides from Google, NVIDIA, Intel, AMD, and AWS.
  • Immediate Engineering Controls: Prohibit unpinned model names or main-branch pulls in production and CI/CD pipelines by pinning exact commit hashes and SHA-256 checksums (e.g., 496dba118d1a58f5f3db2efc88dbdc216e0483fc89fe6e47ee1f2c53f18ad1e4 for Kokoro v1.0), inventory dormant upstream dependencies into internal mirrors, and pair malware scanning with deterministic output regression testing.

Slide 1: Overview of Kokoro-82M Hijack Demonstration and 87-Second Weight Poisoning


1. Zero-Exploit Takeover and Silent Weight Modification

Unlike software supply-chain compromises that require breaking into build servers or injecting obfuscated scripts, the Kokoro demonstration relied exclusively on normal platform workflows and operational neglect:

Dimension Verified Technical Finding
Target Model hexgrad/Kokoro-82M (82M parameters, Apache-2.0, 8 languages and 54 voices in v1.0)
Monthly Reach & Ecosystem 12,988,138 monthly downloads, 7.45k likes, 81 fine-tunes, 12 adapters, 126 quantizations, 100 Spaces
Industry Adoption Signals Supported in voice-agent developer tools from Google, NVIDIA, and Intel; featured in AMD and AWS AI-agent guides
Maintenance Abandonment No updates pushed since April 2025; GitHub repository and PyPI package also went silent around the same time
Account Hijack Vector hexgrad.com listed in public DONATE.md was unregistered; researchers registered it and executed a password reset
Tampering Method & Time Fine-tuned neural weights on a consumer gaming GPU in 87 seconds to alter spoken output relative to input text
Detection & Safety Guard Zero changes to inference code, voice assets, or configs; tests on a separate account triggered zero Hugging Face flags (poisoned weights were tested locally and never uploaded)

Slide 2: Zero-Exploit Takeover and Scanner Evasion Mechanism

In practical terms, hijacking an abandoned publisher’s expired domain resembles buying the disconnected phone number of a closed neighborhood shop while keeping its storefront sign untouched. Modifying only the neural weights is equivalent to swapping the liquid inside a sealed bottle without changing the bottle, label, or expiration date—leaving endpoint and repository malware scanners blind because there is no unauthorized code execution path to flag.


2. Open-Weight Security Assumptions and the llms.txt Parallel

The assumption that open-source or open-weight models provide inherent security transparency has long been challenged by practitioners. In WhyLabs’ November 2023 workshop Intro to LLM Security - OWASP Top 10 for LLMs (00:06:30), CEO Alessya Visnjic noted (WhyLabs Workshop):

“now you could think okay if I’m taking llama and it’s open source so there’s more sense of security unfortunately no because well you don’t know what the data it was trained on is like so you can’t really guarantee that there wasn’t any kind of training data poisoning or any other potential security exploits”

While that warning focused on training-time provenance, Hertz’s Kokoro demonstration proves that post-publication governance poses an equally acute exposure: even after a community vets a model release, the upstream publisher account can silently change hands.

This mirrors findings documented in the Cloud Security Alliance’s September 7, 2026 research note, The llms.txt Trust Model Is Broken, covering Alon Hertz’s August–September 2026 scan of 6,214 live domains across defense contractors, Fortune 500 firms, and major technology companies (CSA Research Note):

  • Scope of Unclaimed References: Out of 8,265 detected llms.txt and llms-full.txt files (8,565 in GBHackers’ reporting), 120 files (~1.5%) pointed to unregistered packages or domains, encompassing 227 install commands and 237 total unclaimed artifacts.
  • Rapid Real-World Execution: When Hertz registered a sample of unclaimed names and hosted benign phone-home packages, automated coding tools inside a Fortune 500 environment executed the substituted package within an hour (under four minutes in some media accounts), followed by dozens of additional companies over subsequent days.

Across both llms.txt documentation drift and abandoned model-hub accounts, automated engineering workflows treat vendor-affiliated text and publisher namespaces as ground truth without verifying ongoing ownership.

Slide 3: Open-Model Security Warnings and Supply-Chain Parallels


3. Model Card Reconciliation: $1,000 Training Cost vs. 87-Second Subversion

Inspecting the primary hexgrad/Kokoro-82M model card reveals how cheaply high-leverage open AI infrastructure can be created—and how cheaply it can be subverted when lifecycle governance is absent:

Model Card Metric Recorded Value
Architecture & License 82 million parameters (StyleTTS 2 / iSTFTNet decoder-only), Apache-2.0
v0.19 Release (Dec 25, 2024) <100 training hours, 1 language & 10 voices, SHA-256 prefix 3b0c392f, 500 A100 80GB hrs @ $0.80/h ($400)
v1.0 Release (Jan 27, 2025) Few hundred hrs, 8 languages & 54 voices, SHA-256 496dba118d1a58f5f3db2efc88dbdc216e0483fc89fe6e47ee1f2c53f18ad1e4, 500 A100 80GB hrs @ $1.20/h ($600)
Total Training Compute 1,000 A100 80GB GPU hours (~$1,000 USD total)
Reported API Economics (Apr 2025) Under $1 per million input characters ($0.65–$0.80 on benchmarked hosts), under $0.06 per output audio hour
Ecosystem Footprint 12,988,138 monthly downloads, 81 fine-tunes, 12 adapters, 126 quantizations, 100 Spaces

Slide 4: Model Card Numbers and Cost Asymmetry

A ~$1,000 model became a core building block referenced across major chipmaker and cloud guides, yet its distribution root remained unprotected once maintainer activity ceased in April 2025. For pipelines configured to pull main or the latest revision automatically, the operational risk resembles a restaurant chain with an automated standing order that keeps accepting shipments from a hijacked supplier simply because the truck and invoice format have not changed.


4. Platform Verification Gaps and Explicit Scope Boundaries

In BleepingComputer’s report, Hertz emphasized two structural weaknesses: first, Hugging Face does not vet publishers, verify whether an account is still controlled by its original owner, or audit uploaded weights and datasets; second, even without an account takeover, an actor can build reputation with benign models before pushing a malicious update later. Both Hertz and BleepingComputer reported receiving no response from Hugging Face by publication time.

Slide 5: Platform Vetting Gaps and Weight-Inspection Limits

At the same time, primary sources require clear separation between demonstrated risks and unverified items:

  • Operational Blind Spots in Enterprise Adoption: Kokoro’s own model card already warns users against fake domains such as kokorottsai_com and kokorotts_net. Yet engineering teams frequently evaluate open models solely on license terms, benchmark scores, and language quality while leaving model-pull scripts unpinned (main branch) and assuming antivirus scans cover binary weight blobs.

Slide 6: Lifecycle Blind Spots and Unpinned Model Fetches

  • Unverified Historical Exploitation: Whether malicious actors had already exploited expired domains to hijack other popular Hugging Face model accounts prior to this disclosure is unconfirmed in the cited reports.
  • Unverified Platform Remediation Timeline: Specific technical controls, domain-verification checks, or dormant-account locks that Hugging Face may introduce—and when they might ship—remain unconfirmed pending a vendor response.

Slide 7: Unverified Boundaries and Three Engineering Defenses


5. Three Controls Every AI Engineering Team Should Enforce Now

Waiting for public registries to harden publisher verification leaves downstream systems exposed. Engineering and security teams consuming open weights should standardize three controls immediately:

  1. Pin Exact Commit IDs and Full SHA-256 Checksums: Ban unpinned model names and main-branch fetches in application code, Dockerfiles, and CI workflows. For hexgrad/Kokoro-82M v1.0, enforce immutable verification against its published SHA-256 digest (496dba118d1a58f5f3db2efc88dbdc216e0483fc89fe6e47ee1f2c53f18ad1e4) so upstream overwrites fail closed.
  2. Inventory Dormant Dependencies and Mirror Internally: Audit the last update date, source repository activity, and contact/donation domains of every third-party model in production or staging. Copy vetted weights for unmaintained or critical models into private artifact registries and block runtime fetches from public hubs.
  3. Add Output Regression Testing and Production Telemetry: Treat weight files as executable behavior rather than passive data. Run deterministic golden-input regression checks (verifying transcribed output or expected behavior) before promoting any weight update, and monitor production outputs for anomalous drift.

Slide 8: Standardizing Pinning, Mirroring, and Output Verification


Primary Sources

Back to articles