Consuloji: Deep Dives into Overseas AI日本語

Why Ripple-65B's Benchmark Spread Swings from 0.4 to 9.7 Points: Deconstructing Self-Reported Tables, 129.6GB Weights, and the $32 Million Cost Narrative

Published: November 5, 2026

Listen or watch on YouTube

A forensic architecture audit on how evaluation harnesses, sliding-window attention, and 16k provider context caps separate open-weight dense models from frontier MoE claims.

Japanese Edition: https://consuloji.substack.com/p/airipple-65b0497

YouTube (Japanese audio): https://www.youtube.com/watch?v=cMzzU1mqrUQ


Executive Summary & Key Technical Takeaways

  • The Release: On November 4, 2026, Ricursive Intelligence released weights for Ripple-65B Preview, an open-weight dense model under Apache 2.0. Derived from Ricursive’s internal AI chip-design tooling, it was pre-trained on 12 trillion tokens across 80 layers (d_model = 5,120) with a 1,024-token sliding-window attention mechanism.
  • The Dual-Row Benchmark Anomaly: On Terminal-Bench 2.1, Ricursive’s model card reported an initial score of 80.5, edging out Reflection Beam 501B (80.1, +0.4 margin). However, a secondary evaluation in the same table using the Terminus 2 harness kept Ripple at 80.5 while Reflection Beam plunged 9.3 points to 70.8—widening Ripple’s margin from +0.4 to +9.7 points, while Meta Muse Spark Open rose to 78.7.
  • Active Parameter Physics vs. Marketing Narratives: Headlines highlighting a “compact 65B model beating 500B-to-900B models” conflate storage footprint with active forward-pass compute. While MoE competitors possess higher total parameter counts (Beam at 501B, Muse at 816B, Inkling at 975B, Nemotron at 550B), their sparse routing activates only 23B to 55B parameters per token. Ripple-65B executes all 64.8 billion parameters on every token, making its per-step memory and FLOP load heavier than any MoE competitor in the table.
  • The Context Window Compression Air-Pocket: While marketing highlights “up to 1M context” and config.json records an internal max_position_embeddings of 2,097,152, commercial hosted endpoints tell a different story. On OpenRouter (via GMI Cloud), the live context limit is strictly capped at 16,384 tokens—a 61-fold compression from the 1M marketing claim.
  • Unverified Economic Assertions: Ricursive claims a total R&D cost of $32 million (“one-tenth the cost of prior open-weight frontier models”), citing 1.4 million reinforcement learning rollouts compared to Reflection Beam’s reported >100 million rollouts (~71x fewer). However, these comparative figures represent vendor estimates without third-party audit, and the official Terminal-Bench 2.1 repository contained zero verified submissions for any of the five evaluated models as of November 5, 2026.

Slide 1: Deconstructing the 80.5 Benchmark Score and Evaluation Source Discrepancies


1. Deconstructing the Headline Benchmark: The Dual-Row Anomaly on Terminal-Bench 2.1

In the release documentation for Ripple-65B Preview, Ricursive claimed state-of-the-art agentic coding among US open-weight models, anchored by a score of 80.5 on Terminal-Bench 2.1 across 89 discrete bash task directories. A granular examination discloses two conflicting evaluation rows:

Evaluation Row Ripple-65B Reflection Beam 501B (A23B) Meta Muse Spark Open 816B (A42B) Thinking Machines Inkling 975B (A41B) NVIDIA Nemotron 3 Ultra 550B (A55B) Evaluation Source Notes
Self-Reported Baseline 80.5 80.1 76.4 63.8 56.4 Compiled from competitor cards; unstandardized
Terminus 2 Harness (Ricursive Re-run) 80.5 70.8 78.7 56.2 55.1 Italicized internal run; long context with summarization
Calculated Spread Variance 0.0 -9.3 +2.3 -7.6 -1.3 Beam delta widens from +0.4 to +9.7; rank inverts

Slide 2: Structural Dimensions of the Release ── Weights, Context Limits, and Harnesses

In the self-reported row, the gap between Ripple-65B (80.5) and Reflection Beam (80.1) is 0.4 points. Across 89 tasks, each task represents approximately 1.12 points. A 0.4-point differential represents less than half the resolution of a single task outcome. Without explicit rollout counts, pass@k distributions, or confidence intervals, a 0.4-point difference is indistinguishable from stochastic evaluation noise.

Furthermore, when Ricursive re-evaluated competitor models using Terminus 2 (configured with long context and automated summarization), Reflection Beam collapsed from 80.1 to 70.8, while Meta’s Muse Spark Open gained 2.3 points to 78.7. Crucially, Ripple-65B remained pinned at 80.5 across both rows. The model card provides no architectural explanation for why Ripple exhibited zero variance across harness transitions while competitor baselines shifted by up to 9.3 points.

Verification against Harbor Framework’s official Terminal-Bench 2.1 repository reveals that community submissions were closed, with the leaderboard tracking 20 official maintainer-run evaluations. None of the five models appeared on the official maintainer leaderboard as of November 5, 2026.

Slide 3: Anatomy of the Release ── Apache 2.0 Licensing, Sliding-Window Attention, and Weight Footprint


2. Model Architecture, Storage Footprint, and Hardware Sizing Physics

Enterprise deployment requires separating parameter marketing from VRAM capacity, quantization, and kernel execution.

2-1. Dense Parameter Execution vs. Sparse MoE Forward Passes

Ricursive chose a dense topology over MoE to eliminate distributed routing overhead and all-to-all communication latency:

Model Architecture Parameter Topology Total Parameters Active Parameters / Token BF16 Weight Footprint Minimum GPU Memory Required
Ripple-65B Preview Dense 64.8B 64.8B ~129.6 GB 2x 80GB H100 / A100 or 1x DGX Spark
NVIDIA Nemotron 3 Ultra Sparse MoE 550B 55B ~1,100 GB Multi-node cluster (8x 80GB min)
Meta Muse Spark Open Sparse MoE 816B 42B ~1,632 GB Multi-node cluster (16x 80GB min)
Thinking Machines Inkling Sparse MoE 975B 41B ~1,950 GB Multi-node cluster (24x 80GB min)
Reflection Beam 501B Sparse MoE 501B 23B ~1,002 GB Multi-node cluster (8x 80GB min)

Every dense forward pass touches all 64,782,689,360 parameters. Consequently, while Reflection Beam requires massive aggregate VRAM to house its 501 billion weights in memory, its compute engine executes only 23 billion active parameters per token. Ripple-65B executes nearly 2.8 times more parameters per forward token step than Reflection Beam. Claims that a “65B model defeats models 8 to 15 times its size” are true for raw weight storage, but invert for per-token compute intensity.

Slide 4: Evaluating the Table ── Who Measured What, and With Which Tools?

2-2. Precision Quantization and Weight Arithmetic

Ricursive released model weights across three distinct variants on Hugging Face:

  • BF16 Instruct Variant: 16 sharded safetensors files totaling 129,565,519,472 bytes (~129.57 GB), equaling exactly 2.00 bytes per parameter.
  • NVFP4 Instruct Variant: 6 sharded safetensors files totaling 57,213,852,024 bytes (~57.21 GB), equaling approximately 0.883 bytes per parameter.

While NVFP4 compression reduces disk and memory footprint by 55.8%, Ricursive’s model card omits benchmark evaluation scores for the NVFP4 release. In contrast, Reflection Beam’s model card discloses precise quantization degradation curves: BF16 scored 80.1, FP8 scored 78.9, and NVFP4 dropped to 78.4. Enterprise teams adopting Ripple’s 57.2GB NVFP4 checkpoint cannot assume retention of the 80.5 score without empirical in-house validation.

Additionally, running Ripple-65B requires loading custom execution code via modeling_ricursive.py. In regulated environments, loading external Python definitions (trust_remote_code=True) introduces code-execution risks requiring security review prior to deployment.

Slide 5: Why Benchmark Gaps Widen from 0.4 to 9.7 Points Across Identical Suites


3. The 1M Context Mirage vs. 16K Operational Provider Reality

A persistent friction point in foundation model integration is the gap between theoretical architecture limits and production API service parameters.

In marketing releases, Ricursive emphasizes an “expanded context window of up to 1M tokens,” and config.json specifies max_position_embeddings: 2097152 (2M tokens). The architecture incorporates sliding-window attention with a local window size of 1,024 tokens to manage memory scaling across deep layers.

However, commercial API endpoints available on OpenRouter reveal a severe deployment constraint:

Provider / Interface Context Window Limit Maximum Completion Tokens Input Token Pricing (Per 1M) Output Token Pricing (Per 1M) Operational Status
Ricursive Technical Blog 1,000,000 tokens Unspecified N/A N/A Vendor architectural claim
Model Repository config.json 2,097,152 tokens Unspecified N/A N/A Maximum rotary embedding position
OpenRouter (via GMI Cloud) 16,384 tokens 14,745 tokens $0.20 $0.60 Active live inference endpoint
Together AI Endpoint 1,000,000 tokens Unspecified Pending Pending Marked “Serverless coming soon”

The active hosted endpoint on OpenRouter enforces a context ceiling of 16,384 tokens—a 61-fold reduction from the 1M marketing claim.

For engineering teams designing agentic workflows—such as repository-wide refactoring or multi-turn debugging—a 16k context window is exhausted rapidly. Passing a moderate codebase of 25 source files easily exceeds 20,000 tokens, rendering the hosted API incapable of executing tasks that the theoretical 1M specification promises.

Slide 6: Unpacking the $32M R&D Narrative and Parameter Accounting


4. Deconstructing the Broader Benchmark Suite: SWE-bench, DeepSWE, and General Reasoning

Beyond Terminal-Bench 2.1, Ricursive’s model card presents evaluations across coding and reasoning benchmarks with notable methodological heterogeneity:

Benchmark Suite Ripple-65B Score Nemotron 3 Ultra Thinking Machines Inkling Meta Muse Spark Open Reflection Beam 501B Evaluation Harness Configuration
SWE-bench Verified 76.5 71.9 77.6 78.7 81.6 OpenCode default; Beam holds +5.1 point lead
SWE-bench Pro v1.0 61.42 46.9 54.3 58.2 66.1 OpenCode default; Beam holds +4.68 point lead
SWE-bench Pro v2.0 Hard 79.1 30.7 2.0 52.3 76.5 Ripple tested with Terminus 2; competitors unstandardized
DeepSWE 1.1 44.3 0.2 1.8 60.0 43.3 Ripple tested with Terminus 2; Muse leads by +15.7
GPQA Diamond 85.2 N/A N/A N/A 90.6 Zero-shot CoT; Beam leads by +5.4
MMLU 89.6 N/A N/A 92.6 N/A Muse Spark Open leads by +3.0 points
AIME 2025 98.3 N/A N/A 100.0 N/A Muse Spark Open achieves perfect 100.0
IFBench 79.2 81.7 79.8 N/A 79.6 Nemotron 3 Ultra leads across open-weight models

(Note: Italicized scores denote internal evaluations conducted by Ricursive where external vendor data was unavailable. Un-italicized scores reflect vendor-reported metrics.)

Three critical takeaways emerge:

  1. Harness Asymmetry in SWE-bench Pro v2.0 Hard: On SWE-bench Pro v2.0 Hard, Ripple reports 79.1. However, the model card footnote clarifies that Ripple was evaluated using Terminus 2, whereas competitor scores reflect unstandardized external disclosures or internal re-runs.
  2. Standardized Benchmarks Favor Competitors: On standardized software engineering benchmarks using OpenCode uniformly, Reflection Beam maintains a consistent lead: +5.1 points on SWE-bench Verified (81.6 vs 76.5) and +4.68 points on SWE-bench Pro v1.0 (66.1 vs 61.42).
  3. Reasoning and Instruction-Following Ceilings: Across non-coding domains, Ripple-65B trails its peers: Beam leads GPQA Diamond by 5.4 points (90.6 vs 85.2), Muse Spark Open outscores Ripple on MMLU and AIME 2025, and Nemotron 3 Ultra leads IFBench at 81.7.

Ricursive’s claim of achieving “U.S. Open-Weight SOTA” is strictly bounded: it applies narrowly to self-reported agentic coding under proprietary harnesses, not across generalized reasoning.

Slide 7: Strategic Assessment ── Open Weights as Progress, Benchmark Rank Reserved


5. The $32 Million Cost Narrative and Training Efficiency Claims

Ricursive highlights achieving frontier capability at a claimed R&D cost of $32 million (“approximately one-tenth the cost of preceding frontier open models”). Examining the substantiation reveals significant gaps:

  • Scope of the $32M Claim: Ricursive states that the $32 million figure encompasses all R&D expenses, including pre-training compute, data synthesis, and alignment.
  • Competitor Baseline Is Vendor Estimation: To justify the “one-tenth” comparison, Ricursive cites press reports of competitor compute commitments (such as $150 million monthly commitments at SpaceX) and estimates derived from Reflection Beam’s reported four weeks of pre-training and four weeks of post-training.
  • Rollout Disparity in Reinforcement Learning: Ricursive reports 1.4 million RL rollouts during post-training, compared to Reflection’s reported >100 million rollouts (~71x fewer). However, raw rollout counts do not account for step lengths, environment complexity, or token depths.

Without third-party audited financial records or standardized compute accounting, comparing internal R&D expenditure against competitor rumors cannot be treated as verified engineering fact.

Slide 8: The Enterprise Implementation Blueprint ── 3 Steps to Verify Before Deployment


6. Enterprise Governance Blueprint: 3 Steps Before On-Premises Deployment

While Apache 2.0 weights represent tangible progress, relying solely on self-reported vendor tables risks operational friction. Enterprise teams should execute a three-step deterministic protocol:

Step 1: Map Physical Memory Capacity to Quantization Realities

Audit target deployment nodes against exact weight requirements:

  • If deploying in BF16 (~129.6 GB), allocate at least two 80GB GPUs (NVIDIA A100/H100) or an NVIDIA DGX Spark station, plus 20GB–30GB host VRAM for KV-cache allocation.
  • If evaluating the NVFP4 variant (~57.2 GB), ensure target hardware natively supports 4-bit tensor cores (NVIDIA Blackwell architecture) and benchmark task accuracy internally before production release.
  • Audit corporate security policies regarding trust_remote_code=True execution for modeling_ricursive.py.

Step 2: Audit Provider Context Constraints vs. Workflow Requirements

Do not build architecture pipelines assuming 1,000,000 tokens of context. If utilizing hosted APIs:

  • Query provider metadata directly (via OpenRouter endpoint APIs) to establish hard token caps.
  • If task workflows involve multi-file code review exceeding 16,384 tokens, deploy private self-hosted inference servers utilizing vLLM or SGLang with flash-attention enabled rather than relying on commercial aggregators.

Step 3: Execute a Deterministic 3-Task In-House Evaluation Suite

Replace generic industry benchmarks with an internal evaluation matrix comprising three representative corporate engineering tasks:

  1. Repository Bug Localization and Fix: Resolving an issue across multiple internal modules.
  2. Log File Forensic Diagnosis: Identifying root cause failures from a 5,000-line structured telemetry log.
  3. Configuration / Infrastructure Generation: Generating production Kubernetes manifests or Terraform modules adhering to internal compliance standards.

Run these three tasks across five trials under identical environmental conditions (same harness, identical prompt templates, zero temperature, fixed token limits) comparing Ripple-65B against your current production baseline to measure pass rates, wall-clock latency, and token expenditure.


Primary Sources

Note: This article is for informational purposes only and does not constitute investment advice or a price forecast. Podcast audio is synthesized from the host’s voice.

Back to articles