What DeepSeek Actually Open-Sourced: Dissecting the DeepInfer Offline Engine, the $38/GPU-Hour Metric, and 3 Missing Production Components
A forensic engineering audit of DeepSeek DeepInfer GitHub repositories, separating the offline engine from the unreleased 100PB 3FS KV-cache cluster, P/D disaggregation, and load balancing.
Japanese Edition: Read the original Japanese article on Substack
Video Episode (Japanese Audio): Watch on YouTube
Executive Summary: Separating the Offline Engine from the Production Infrastructure
When artificial intelligence research lab DeepSeek released its inference engine DeepInfer on GitHub in late October 2026, technology commentators immediately declared that DeepSeek had open-sourced its entire commercial cloud serving stack. Headlines circulated asserting that any enterprise could now deploy DeepInfer to generate “$38 per GPU-hour in revenue” while achieving “96% cache hit rates” and outperforming standard open-source engines like vLLM.

A forensic engineering audit of the primary GitHub repositories, documentation commits, model cards, official API pricing schedules, and external vulnerability databases establishes four critical technical truths:
- What Was Actually Open-Sourced: The public repository (
deepseek-ai/DeepInfer, initial commit October 20, 2026, Apache 2.0 license) is an offline inference engine tailored exclusively to one model architecture (DeepSeek-V4.1-Flash) running on two specific hardware targets: NVIDIA SM100-generation GPUs (Blackwell) and Huawei Ascend 950. - Three Missing Production Components: The repository README explicitly acknowledges that three critical components necessary to run a live commercial serving API were not included in this release:
- The Key-Value (KV) Cache Management System
- Vision / Prefill / Decode Disaggregation
- Cross-Replica Cluster Load Balancing DeepSeek states that these components remain unreleased because they are tightly coupled with internal proprietary infrastructure.
- The Reality of the “$38 per GPU-Hour” Metric: In the accompanying deployment documentation (
doc/inference_deployment_qa.md), DeepSeek reports that its internal production cluster processes 155 trillion (155T) tokens per day, generating $38 per GPU-hour overall at off-peak pricing (approximately 5,984 JPY, or ~6,000 JPY per GPU-hour). This figure reflects DeepSeek’s massive multi-cluster scale, extreme prefix reuse, and multi-petabyte storage tiers—it cannot be replicated simply by executing the standalone offline script. - Parity with vLLM: On October 23, 2026, the vLLM engineering team published benchmark results showing that when evaluated on identical settings (128K context, batch size 8 to 256), vLLM decode performance was within 6% of DeepInfer. Because DeepInfer was primarily constructed on top of vLLM, core engine execution speed remains largely equivalent.
The strategic takeaway for machine learning platform teams: DeepInfer provides an invaluable architectural reference for running DeepSeek-V4.1-Flash on Blackwell and Ascend silicon. However, it is not a drop-in production serving stack. Realizing DeepSeek’s enterprise economics requires engineering a sophisticated external caching and scheduling fabric.

1. Research Protocol: Establishing Baseline Facts Against Market Claims
To separate primary source evidence from viral rumors, this investigation cross-references six primary documentation streams retrieved in early November 2026:
- The
deepseek-ai/DeepInferrepository (README andinference_deployment_qa.mdcommits from October 20 to November 3, 2026). - The
deepseek-ai/deepseek-reciperepository (commit history from October 19, 2026). - Hugging Face model cards for
DeepSeek-V4.1-FlashandDeepSeek-V4.1-Flash-1017. - DeepSeek’s official API pricing schedule (
pricing.md). - DeepSeek’s February 2025 Open Source Week Day 6 infrastructure overview (
day_6_one_more_thing_deepseekV3R1_inference_system_overview.md). - The vLLM project official blog (DeepSeek V4.1 Flash on vLLM, October 23, 2026).
- National Vulnerability Database (NVD) records for related distributed KV-cache libraries (CVE-2026-103761).
Reconciling Viral Claims Against Primary Source Text
A meticulous line-by-line comparison between secondary analyst claims and primary git commits reveals significant distortions:
| Secondary Analyst Claim | Primary Git / Documentation Audit | Forensic Determination |
|---|---|---|
| Cache Hit Rate of 96% | Absent from DeepInfer README, QA docs, model cards, and vLLM blogs. February 2025 disclosure cited 56.3% | Unverified Rumor (Excluded) |
| GPU Revenue Jumped 3x from $12.91 to $38 | The Feb 2025 $12.91 was a theoretical calculation assuming all tokens billed at R1 rates. The $38 figure reflects off-peak blended pricing | Apples-to-Oranges Comparison |
| SWA KV-Cache Saved Every 128 Tokens as 1.42MB | Text states SWA cache is retained once per sequence in host DRAM (~1.4MB/seq); 128 tokens refers to full-attention block chunking | Conflated Metrics Corrected |
2. Chapter 1: The Public Release Scope ── Model Specs and Hardware Targets

DeepSeek’s initial public commit for DeepInfer was recorded on October 20, 2026, under the Apache 2.0 license. The opening section of the repository README establishes an unambiguous boundary:
“DeepInfer is the inference engine that powers DeepSeek’s large-scale production services. This open-source release provides offline inference.”
“Offline inference” denotes batch execution from local scripts rather than an online HTTP server handling dynamic traffic routing, connection pooling, and multi-tenant isolation.
Hardware and Architecture Boundary
DeepInfer’s operational scope is tightly constrained:
- Supported Architecture: Exclusively
DeepSeek-V4.1-Flash(and its minor updateDeepSeek-V4.1-Flash-1017). - Supported Accelerators: NVIDIA SM100 architecture (Blackwell generation) and Huawei Ascend 950. Older NVIDIA architectures (such as Hopper SM90 or Ampere SM80) are not natively targeted in this release.
- Upstream Lineage: The README explicitly states that DeepInfer was primarily based on vLLM, customized with dedicated high-performance kernels.
- Reference Benchmark Setup: Evaluated using 4 GPUs for Prefill and 64 GPUs for Decode in a Wide Expert Parallelism (Wide EP) configuration.
Concurrently, DeepSeek open-sourced deepseek-recipe (October 19, 2026), a specialized Rust and Python library designed to handle prompt formatting, chat template normalization, and token serialization.
DeepSeek-V4.1-Flash Technical Architecture
The model card on Hugging Face outlines the computational parameters governing the engine:
| Architecture Dimension | Specification in Official Model Card | Engineering Significance |
|---|---|---|
| Total Parameter Count | 552B (552 billion) | Large-scale Mixture-of-Experts (MoE) foundation |
| Active Parameters per Token | 8B (Prefill) / 16B (Decode) | Asymmetric MoE routing reducing computational load |
| Maximum Context Window | 1,000,000 (1M) tokens | Native support for extreme context processing |
| Weight License | MIT License | Broad commercial permissibility |
| Total KV-Cache Footprint | 890 bytes / token | Reduced by ~4x compared to previous generation Flash |
| Attention Pattern (5+1 Hybrid) | 5 SWA layers + 1 Full Attention layer | 5 Sliding Window Attention layers (512-token window) per 1 full layer |
| Persistent KV-Cache Footprint | ~8x reduction vs. prior Flash | Only the full attention layer is saved to long-term storage |
3. Chapter 2: Deconstructing the “$38 per GPU-Hour” Metric

In doc/inference_deployment_qa.md, DeepSeek outlines the economics of its internal production deployment:
- Aggregate processing volume: 155 trillion tokens per day on a single large production cluster.
- Revenue generation: $38 per GPU-hour overall (at off-peak pricing).
Converted to Japanese Yen using the Federal Reserve H.10 statistical benchmark (October 30, 2026 rate: 1 USD = 157.47 JPY), $38 corresponds to approximately 5,984 JPY (or ~6,000 JPY) per GPU-hour.
The 50x Spread in DeepSeek’s Official API Pricing Schedule
To understand what “$38 at off-peak pricing” represents, we inspect DeepSeek’s official commercial API pricing schedule (pricing.md):
| Token Category (per 1M Tokens) | Standard Hours Pricing | Off-Peak Hours Pricing | Cache Miss vs. Hit Multiplier |
|---|---|---|---|
| Input (Cache Hit) | $0.006 | $0.003 | 1x (Baseline) |
| Input (Cache Miss) | $0.300 | $0.150 | 50x ($0.150 / $0.003) |
| Output (Decode Generation) | $1.500 | $0.750 | — |
The critical economic mechanism is the 50x price divergence between cache hits and cache misses. When a prompt misses the cache, the user pays $0.150 per million tokens off-peak; when it hits the cache, the charge drops to $0.003.
Generating $38 per GPU-hour while billing at off-peak rates requires extraordinarily high throughput and dense token multiplexing across thousands of concurrent streams.
Why Direct Comparison to February 2025 Is Invalid
Tech commentators claimed DeepSeek’s GPU monetization expanded roughly 3x from ~$12.91 in early 2025 to $38 in late 2026. However, in its February 2025 “Open Source Week Day 6” paper, DeepSeek’s $12.91 metric was calculated by taking hypothetical theoretical revenue ($560,658 over 43,432 GPU-hours) assuming every single processed token was billed at full DeepSeek-R1 standard rates. DeepSeek explicitly noted in the text that actual revenue was substantially lower due to free promotional quotas and off-peak discounts.
Comparing a hypothetical high-tier pricing ceiling from 2025 against an off-peak blended average from late 2026 compares two entirely different accounting constructs.
4. Chapter 3: The 3 Unreleased Production Components ── 100PB 3FS Storage and Schedulers

In the DeepInfer README, DeepSeek explicitly lists the three missing infrastructure components:
- The KV-Cache Management System
- Vision / Prefill / Decode Disaggregation
- Cross-Replica Cluster Load Balancing
The documentation explains:
“Because KV-cache management and cluster load balancing are tightly coupled with our internal infrastructure, we have not yet released the full code.”
DeepSeek’s 3-Tier Storage Hierarchy
In doc/inference_deployment_qa.md, DeepSeek details the proprietary storage architecture powering its production services:
| Storage Tier | Data Resident | Physical Sizing and Sizing Invariants |
|---|---|---|
| Tier 1: GPU HBM | Active batch KV-cache tensors | Onboard GPU memory |
| Tier 2: Host CPU DRAM | SWA attention buffers across 128-token boundaries | Retained once per sequence (~1.4MB / sequence) |
| Tier 3: 3FS NVMe SSD Cluster | Full-attention prefix KV-cache blocks (128 tokens/chunk) | ~100 Petabytes of NVMe SSDs (~14.4TB logical capacity per GPU; recommended 20x 30.73TB SSDs per storage node) |
Cache Retention Lifecycles
DeepSeek sizes its ~100PB SSD storage tier based on empirical prefix reuse distribution:
- Autonomous Agent Workloads: Retained for 3 days (72 hours).
- Interactive Chat Workloads: Retained for 7 days.
By pruning model KV-cache sizes by 8x (saving only the full attention layer) and persisting caches across 100PB of distributed 3FS storage for 3 to 7 days, DeepSeek avoids recomputing prefill attention when agents resume tasks days later.
Asymmetric Scheduling: Pull vs. Push
DeepSeek decouples scheduling across computational phases:
- Prefill Workers (Pull-Based): Individual prefill workers calculate their estimated remaining processing time and independently pull incoming requests when capacity becomes available.
- Decode Instances (Centralized Push-Based): A centralized scheduler routes tokens to decode replicas to balance the sum of active requests and total KV-cache length.
- Minimum Efficient Cluster Footprint: A balanced production unit requires 64 Prefill GPUs + 64 Decode GPUs = 128 GPUs minimum.
5. Chapter 4: Comparison with vLLM and Open-Source Alternatives

The vLLM Benchmark Audit (October 23, 2026)
Following DeepInfer’s release, the maintainers of vLLM benchmarked DeepSeek-V4.1-Flash to evaluate whether DeepInfer introduced a radical execution performance leap:
- Decode Performance: Under identical settings (128K context, batch sizes 8 to 256), vLLM decode throughput matched DeepInfer within 6%.
- Architectural Rationale: The vLLM blog notes that both engines share extensive architectural cross-pollination. Because DeepInfer was built using vLLM as its primary foundation, single-node execution performance is nearly identical.
- Prefill Caveats: The vLLM team emphasized that prefill measurements could not be evaluated on an apples-to-apples basis due to differing context length chunking and pipeline configurations.
Open-Source Ecosystem Replacements
To bridge the three missing components, enterprise infrastructure teams look to open-source orchestration projects:
- Mooncake (
kvcache-ai/Mooncake): A KV-cache-centric disaggregated architecture developed for the Kimi chat platform. - LMCache (
LMCache/LMCache): A vendor-agnostic distributed caching layer managing KV transfers across GPU, host memory, and remote storage. - vLLM Production Stack (
vllm-project/production-stack): An enterprise reference deployment combining prefix-aware routing and distributed cache offloading.
Security Hygiene: Auditing Open-Source Dependencies
Integrating third-party caching frameworks requires active vulnerability governance. For example, Mooncake’s transfer engine was subject to CVE-2026-103761 (CVSS 7.5 High), where unauthenticated attackers could trigger an Out-of-Memory (OOM) denial-of-service in RPC servers. Mooncake maintainers released v0.3.14 on November 3, 2026, introducing mandatory batch-size bounds checks to mitigate the flaw.
6. Synthesis: What Can and Cannot Be Formally Asserted

Verified Facts:
- DeepInfer is an offline inference engine supporting only DeepSeek-V4.1-Flash on NVIDIA SM100 (Blackwell) and Huawei Ascend 950.
- DeepInfer omits the production KV-cache system, P/D disaggregation infrastructure, and cluster load balancing.
- DeepSeek’s internal production deployment achieves $38/GPU-hour at off-peak rates on a 155T token/day cluster backed by ~100PB of 3FS SSD storage.
- vLLM decode throughput is within 6% of DeepInfer under identical model and batch configurations.
Unverified Claims:
- That deploying DeepInfer standalone yields $38/GPU-hour in an enterprise environment.
- That DeepSeek operates at an unverified 96% cache hit rate.
- That DeepInfer provides a general-purpose engine replacement for existing Hopper or Ada GPU workstation clusters.
7. Actionable Enterprise Implementation Playbook

For platform engineering teams evaluating DeepInfer, three disciplined actions should guide planning:
- Audit Hardware and Model Boundaries: Ensure your cluster operates NVIDIA Blackwell (SM100) or Huawei Ascend 950 silicon and targets DeepSeek-V4.1-Flash. If operating Hopper (H100/H200) or generic open models, maintain standard vLLM or SGLang deployments.
- Profile Prompt Reuse and Temporal Intervals: Analyze production API logs to determine prompt prefix commonality and calculate reuse distributions across 24-to-72-hour windows before investing in multi-terabyte SSD caching layers.
- Map the Three Missing Layers: If deploying V4.1-Flash in production, construct a formal architecture matrix pairing DeepInfer or vLLM with verified caching (LMCache, Mooncake) and routing solutions, verifying CVE security patch statuses prior to deployment.
8. Primary Source Documentation
- DeepSeek DeepInfer Repository: deepseek-ai/DeepInfer on GitHub
- DeepSeek Inference Deployment Q&A: doc/inference_deployment_qa.md
- DeepSeek Initial Commit Record (October 20, 2026): GitHub API Commit d585e65
- DeepSeek Recipe Repository: deepseek-ai/deepseek-recipe on GitHub
- DeepSeek-V4.1-Flash Model Card: Hugging Face Model Repository
- DeepSeek Commercial API Pricing: Official DeepSeek API Documentation
- DeepSeek Infrastructure Overview (February 2025): Open Source Week Day 6
- vLLM Engineering Blog (October 23, 2026): DeepSeek V4.1 Flash on vLLM: Prefill-Decode Disaggregation and Wide EP: vLLM Official Blog
- National Vulnerability Database: CVE-2026-103761 Detail
- Federal Reserve Statistical Release H.10 (November 2, 2026): Foreign Exchange Rates: Federal Reserve Board