Lattice Compute × Vast Data × NVIDIA Ecosystem

Chapter 1

Three Technologies, Explained in One Sentence

All three tackle the same problem — feeding AI data to the GPU — but via completely different paths.

LATTICE

The NVMe SSD pool becomes the GPU's "expanded memory". GPUs attach directly to the SSDs through a PCIe switching fabric (Broadcom PEX89144); weights are streamed in layer by layer.

VAST

The SSD pool becomes a rack-wide "shared storage pool". GPU servers access remote storage over Ethernet (NVMe-oF / RoCE).

NVIDIA NVIDIAEcosystem Compute Servers

Model weights are packed into the GPU's own HBM; GPUs interconnect at high speed via NVLink / NVSwitch; if memory runs out, add more cards.

Bottom Line All three solve the same problem — feeding AI data to the GPU — but take completely different paths: one usesPCIe Direct Attach(Lattice), one usesEthernet(Vast), and one relies onGPU Memory + Private Interconnect(NVIDIA).

Chapter 2

How It Works

Core chapter: the 'GPU directly attached to SSD' path running on Broadcom's PEX89144 switch chip.

Traditional Path (Slow)

Data path:SSD → System Memory → NIC → Network → NIC → System Memory → GPU. Data hops via CPU Root Complex / IOMMU and multi-layer protocol stacks — long path, many hops,Millisecond-Level。

Lattice Path (Fast)

GPUs and the NVMe SSD pool hang on thethe same PCIe Fabric(PEX89144 switch chip),KFNative™letting the GPU issue NVMe read commands directly to the SSDs,Microsecond-Level。

Animation 1 | Traditional vs. Lattice: Data Path Comparison

Left: particles hop slowly (milliseconds); right: particles flash straight through (microseconds)

Animation 2 | PEX89144 Topology: GPU and SSD Woven into a Low-Latency Fabric

PEX89144 weaves multiple GPU servers + SSD pooling modules into one low-latency switching fabric

Key Numbers

144PEX89144 Lanes

8×NT2.0Multi-Host Ports

128 GB/sBandwidth per x16 Port

<115nscut-through Forwarding Latency

ms → μsLattice Interconnect: from milliseconds down to microseconds

Tens of TimesEffective Bandwidth Boost

1/64 GPUGPU scale needed to train LLMs

35msGPT-3 175B Single-Layer Weight Loading

180msGPT-3 175B Single-Layer Computation

≈100 GB/sGen6 x16 SSD

Bottom Line KFNative™

essence:put the NVMe SSD pool and GPUs on the same PCIe Fabric, using KFNative™

letting the GPU issue NVMe commands directly, bypassing the CPU, NIC and protocol stack.

Three-Layer Technology Stack

KFNative™'s Lattice computing is built on three layers that work together to feed AI data to GPUs at microsecond speed:

LAYER 1

Interconnect Layer — Lattice PCIe Fabric. A PCIe switching fabric (Broadcom PEX89xxx) hangs GPUs and the NVMe SSD pool on the same exchange domain; data moves with no CPU / NIC hops.

LAYER 2

Compute Layer — KFNative™ runtime & driver. Lets the GPU issue NVMe read / write commands directly to the SSDs, with KV-PUT / GET / DEL primitives for storage semantics.

LAYER 3

Service Layer — Tier0 NFS. High-throughput near-storage service for distributed training; scale-out across racks while keeping microsecond-level local access.

Four Pillars of GPU↔SSD Tight Coupling

Same PCIe domain:GPUs and SSDs share one PCIe switching fabric; no PCIe root-complex detour.

NVMe passthrough:The GPU issues NVMe commands directly to the SSD, bypassing CPU, NIC and the multi-layer protocol stack.

Paged KV-Cache:KV-Cache is managed in pages; when GPU memory is short, pages are swapped within the extended PCIe domain in microseconds.

Streaming weights:Model weights are streamed layer by layer into the GPU, overlapping load and compute across layers.

KV-Cache Paging & Weight Streaming

Instead of holding the whole KV-Cache in HBM, KFNative™ pages it onto the NVMe pool and keeps only the working set resident. When attention needs a page that is not in GPU memory, it is fetched inside the extended PCIe domain — the swap cost drops from milliseconds to microseconds.

Model weights are streamed per layer: for GPT-3 175B, loading one layer's weights takes about 35 ms while computing that layer takes about 180 ms, so load and compute overlap naturally and the GPU never waits for storage.

Ring Extension: All/All Ring Reduce

With Ring Extension, commodity GPUs (no NVLink / HBM dependency) form a ring topology over the Lattice Fabric. All/All Ring Reduce performs collective reduction across the ring, which is the basis of the 1/64 GPU claim for training LLMs — far fewer GPUs are needed for the same model scale.

PCIe Gen5 → Gen6 Bandwidth Doubling

32 GT/sPCIe Gen5 per-lane rate

64 GB/sGen5 x16 per direction

64 GT/sPCIe Gen6 per-lane rate (2× Gen5)

128 GB/sGen6 x16 per direction

Moving from Gen5 to Gen6 doubles per-direction bandwidth of an x16 link from 64 GB/s to 128 GB/s, keeping the Lattice Fabric ahead of SSD bandwidth demand.

KFNative™ Accelerator Card Specifications

Switch chip	Broadcom PEX89xxx series PCIe switch
Lanes / topology	80 lanes · 4× GPU + 8× NVMe SSD
Power	121–189 W
GPU port	x16 per GPU port
SSD bandwidth	≈ 100 GB/s per Gen6 x16 SSD

Hardware Deployment Models

1:1

One GPU card pairs with one SSD — the tightest coupling for latency-sensitive inference.

1:N

One GPU card drives multiple SSDs for large working sets and high bandwidth.

N:1

Multiple GPUs share one SSD pool for rack-scale capacity with balanced access.

A single node starts at 32–64 TB of NVMe capacity; a full rack stays in a 2000–2700 W power envelope with 32–64 TB+ per node.

Driver / Firmware Five-Layer Software Stack

KFNative™'s driver and firmware follow a five-layer software stack — hardware abstraction, BAR management (BAR0 / BAR2 / BAR4), transport & queues, storage service, and application interface — exposing KV-PUT / KV-GET / KV-DEL / KV-SYNC primitives to the upper layer.

Chapter 3

How It Works

Reference: a rack-wide shared pool built around storage.

DASE Architecture (Disaggregated Shared Everything)

Capacity nodesEBox / DBox(holding SSDs) and performance / service nodesCNodeare separated, allowingcapacity and performance scale asymmetrically。

Inside a storage node:SCM(storage-class memory, e.g., KIOXIA) as write buffer +QLC SSDfor bulk capacity.

Access method:NVMe over Fabrics (RoCE / RDMA Ethernet); controllers useNVIDIA BlueField DPU, dual 100GbE (single node totals800Gb/sbandwidth).

Key trait: storage-centric, rack-wide shared pool; clients (including GPU servers) access it via NICs over network protocols.

Animation 3 | Vast Topology: GPU Servers Access Remote Storage over Ethernet

GPU servers go via NIC → Ethernet switch → Vast storage node (SCM + QLC)

Vast vs. Lattice: Similarities & Differences

Similarities

Both pool SSDs, decouple storage from compute, and let GPUs leverage massive SSD capacity instead of relying on HBM alone.

Differences

Vast usesEthernet + DPU + network protocol stack(NVMe-oF / RoCE); latency is bounded by network RTT; Lattice usesPCIe Fabric Physical Direct Attach, with no NIC, no network protocol stack; latency is an order of magnitude lower, and GPUs and SSDs are more tightly coupled (same PCIe domain).

Bottom Line VAST

turning storage into rack-wide shared service, essentially 'network storage + DPU acceleration'; Lattice, by contrast,soldered into the PCIe domain where the GPUs live。

Chapter 4

vs. NVIDIAEcosystem Compute Servers: The Differences

Key comparison: closed & proprietary vs. open standards.

NVIDIAEcosystem Features

HBM High-Bandwidth Memory: H100 80GB, B200 192GB (per GPU)
NVLink GPU Interconnect: H100 900 GB/s, B200 1.8 TB/s (bidirectional); NVSwitch provides full mesh across 8 cards
Cross-node via InfiniBand / Ethernet; SSDs only store datasets and checkpoints;weights must reside permanently in HBM
Extremely powerful butClosed & Proprietary(NVLink / NVSwitch / HBM are all NVIDIA-proprietary); HBM capacity is a hard constraint — if the model doesn't fit, add cards / upgrade to larger HBM,Cost scales linearly with GPU count

Lattice Ecosystem Features

No reliance on NVLink / HBM: weights in the SSD pool areLayer-by-Layer Streaminginto the GPU; HBM only serves as a compute buffer
GPU demand drops dramatically (e.g., training LLMs needs only1/64 GPUscale)
Built onopen PCIe standardsand standard NVMe SSDs; the card + software stack is offered for server / storage vendors to integrate

Animation 4 | Same GPT-3 175B Training Scenario: NVIDIA vs. Lattice

Left: weights fill every HBM; right: weights stream from the SSD pool into the GPU layer by layer

Core Comparison Table

Dimension	NVIDIA Ecosystem Compute Servers		Vast Data	NVIDIA GB200 NVL72 (SuperNode)
Where Data Lives	GPU HBM Memory (Resident)	SSD Pool (Streaming Load)	SSD Pool (Shared Storage)	HBM3e Unified Pool (13.4 TB / 72 GPUs)
Interconnect	NVLink / NVSwitch (Proprietary)	PCIe Fabric (PEX89144, Open Standard)	Ethernet NVMe-oF / RoCE	NVLink 5th Gen: 1.8 TB/s per GPU, 130 TB/s fabric
GPU↔Storage Latency	ns-level in HBM, but capacity-limited	Microsecond-Level (PCIe Direct Attach)	Microsecond-Level+ (Network RTT)	ns-level HBM inside one NVLink domain
Scaling Method	Add GPUs / upgrade to larger HBM	Add SSD pools / expand the fabric	Add EBox / CNode	Add NVL72 racks (domain scales to 576 GPUs)
Proprietary Dependencies	NVLink、NVSwitch、HBM	None (standard PCIe / NVMe)	BlueField DPU (optional)	NVLink / NVSwitch / HBM / Liquid cooling
Cost Structure	Cost scales linearly with GPU count	Far fewer GPUs, cheap SSDs	Storage pooling, billed by capacity	~$3M per rack, 120+ kW power
Ecosystem Openness	Closed	Open PCIe / NVMe	Open storage ecosystem	Closed (proprietary NVIDIA stack)

Bottom Line NVIDIA

NVIDIAstrength isextreme bandwidth inside GPU memory, at the cost ofa proprietary, closed ecosystem and a hard HBM capacity limit; Lattice usesopen PCIe / NVMeshifting the cost curve from "GPU count" to "SSD capacity".; the NVIDIA GB200 NVL72 supernode answers the HBM capacity wall with a rack-scale 13.4 TB HBM pool at NVLink speed — closed, liquid-cooled, and ~$3M per rack.

Chapter 5

What Changes Are Needed on the Server

What changes is the "connectivity" and the software — the CPU / GPU architecture stays untouched.

Hardware Changes

GPU servers install theLattice Interconnect Card(KF-6600 Mezzanine-class, PCIe Gen5 x16)
DeployPEX89144switch card / fabric module: GPU / CPU upstream ports connect to the switch, SSD pooling modules hang on the downstream
NVMe SSDs are regrouped intopooling modules(shared across multiple drives, no longer dedicated to a single server)
Rack-level scaling:PEX89144 Cascading(RDK slim-line cascade solution)

Software Changes

InstallKFNative™ Software Stack(user-space driver that lets the GPU issue NVMe commands directly)
BIOS / firmware: enablePCIe resource allocation, SR-IOV VF mapping
Training / inference frameworks adapt toweight streaming load

Not Needed

NVLink / NVSwitch
Huge HBM
InfiniBand Network

Keep (No Replacement Needed)

Standard x86 Servers
Standard NVMe SSDs
Standard PCIe Gen5 Slots

Bottom Line The changes center on "add one switch card + pool the drives + install one software stack, without touching the CPU / GPU architecture.

Chapter 6

Summary: Everything on One Page

Three paths, one goal: feed AI data to the GPU.

LATTICE

The SSD pool acts as the GPU's expanded memory, directly attached via PCIe Fabric; weights are streamed in layer by layer.

PCIe Direct AttachMicrosecond-Level1/64 GPU

Best for: LLM training / inference where weights don't fit in HBM, targeting a dramatic reduction in GPU cost.

VAST

The SSD pool becomes a rack-wide shared storage pool; GPU servers access it over Ethernet (NVMe-oF / RoCE).

Shared StorageEthernetDPU

Best for: enterprise storage foundation with multi-server sharing and independently scalable capacity and performance.

NVIDIA NVIDIAEcosystem

Weights are crammed into GPU HBM; NVLink / NVSwitch provide high-speed interconnect; if memory runs out, add more cards.

HBMNVLinkClosed & Proprietary

Best for: flagship compute pursuing extreme per-node performance, with ample budget and acceptance of a closed ecosystem.

Scenario — Large-Model Training (example)

HBM and the lattice are complementary layers, like cache and main memory: HBM keeps the active layer / hot weights at TB/s-scale bandwidth (HBM3E ~1.2TB/s per stack, HBM4 targeting >2TB/s), while the NVMe SSD pool provides TB-level capacity.
Bandwidth trade-off: HBM wins on speed (TB/s scale); the lattice's extended PCIe path is ~128GB/s (PCIe 5.0 x16, ~100GB/s scale) — the streaming pipeline (load-while-compute) and the direct path hide most of the gap.
Below: particles show where the data travels, and how fast it moves on each path.

HBM Server Cluster

TB/s-scale compute-layer bandwidth (HBM3E ~1.2TB/s per stack), but capacity-limited & costly

HIGH BW · CAPACITY-LIMITED

KFNative™ Lattice Cluster

Dimension	HBM Server Cluster	KFNative™ Lattice Server Cluster
Memory bandwidth	TB/s scale — HBM3E ~1.2TB/s per stack, HBM4 targeting >2TB/s	~128GB/s (PCIe 5.0 x16) — ~100GB/s scale, an order of magnitude lower
Capacity	Limited by on-GPU HBM; full model does not fit → repeated re-load / re-sync	NVMe SSD pool — TB-level capacity, scales with storage
Role in the hierarchy	High-speed compute layer (cache-like): holds active layer / hot weights	Extended storage layer (main-memory-like): large-capacity complement, not a replacement
Data path	GPU → HBM → NIC → network → sys memory → remote node (multi-hop)	GPU ↔ NVMe SSD pool via extended PCIe lattice (direct); reduces double-port SSD dependency
Streaming	Weights must fit in HBM / re-load on miss — ms-level cross-node sync	Load-while-compute streaming pipeline hides the lower bandwidth
GPU requirement	Premium high-HBM GPUs needed — costly	Standard GPUs + SSD memory pool suffice; ~64X fewer GPUs via Ring Extension
Limitations	Capacity ceiling, high cost, cross-node sync bottleneck	Lower bandwidth (~100GB/s scale; mitigated by streaming); requires KFNative™ lattice HW/SW stack
Relationship	Complementary — cache-like hot-data layer	Complementary — main-memory-like capacity layer; together they form a layered memory hierarchy

Worked Example — Training GPT-3 175B (~350GB of weights)

The same workload on the two clusters: HBM loads the whole model first (serial wait), the KFNative™ lattice streams layer-by-layer and overlaps load with compute.

HBM cluster — load-then-compute (serial)

T ≈ T_load + T_compute

Load — weights → HBM

ms-level · GPU idle

Compute

starts after load

0sT_loadT_total = T_load + T_compute

All ~350GB of weights are loaded into HBM at once (cross-node network sync, millisecond-level); GPUs wait, compute capacity sits idle.
Compute starts only after the load finishes — total time ≈ load + compute (serial wait, high latency).

KFNative™ lattice — streaming load-while-compute (pipeline)

T ≈ T_compute + first layer

Load — layer-by-layer

~35ms / layer

Compute

~180ms / layer

0sstreaming…T ≈ T_compute + first layer

Bandwidth is limited (~128GB/s), so the model cannot be loaded all at once — weights stream layer-by-layer, load-while-compute.
Load and compute overlap in a pipeline; the GPU keeps working — total time ≈ compute + first layer (lower latency, higher utilization), even though bandwidth is lower than HBM.

Chapter 7

One GPT-3 175B, Any Training Window — Hardware & Cost

Same model, same 3.14×10²³ FLOPs. Pick a training window, and the table below shows how much hardware each path must stack and what it costs — the fastest way to see who wins on cost-performance. Estimated from public baselines: GPT-3 paper, NVIDIA H100 / GB200 NVL72 official specs, market list prices.

PURE HBM

Weights live in GPU HBM. To hold 700 GB of weights plus gradients and optimizer states, GPT-3 175B demands hundreds of HBM-equipped GPUs.

512×H100~9 days~$17M

Best for: performance-first teams with deep pockets.

PURE LATTICE

Weights stream from a shared NVMe pool over PCIe, so 8 GPUs can train the same model — no 700 GB crammed into VRAM.

8×H100+SSD~13 months~$0.5M

Note: the 1/64 GPU ratio applies to memory/bandwidth-bound scenarios, not to dense pretraining — see the fixed-window table below.

SUPERNODE

72 Blackwell GPUs in one NVLink domain act like a single giant GPU with 13.4 TB HBM3e — NVIDIA's rack-scale answer to the HBM capacity wall.

72×B200~9 days~$3.2M

Best for: frontier labs that can afford a closed, liquid-cooled rack.

Same Training Window → Hardware Stacked & Cost (GPT-3 175B)

Training Window	Pure HBM (H100)	Pure Lattice (H100 + NVMe pool)	SuperNode (GB200 NVL72)
7 days	656×H100 · ~$21.6M	438×H100 · ~$13.4M	2×NVL72 · ~$6.4M
30 days	154×H100 · ~$5.1M	103×H100 · ~$3.4M	1×NVL72 · ~$3.2M
60 days	77×H100 · ~$2.5M	52×H100 · ~$1.8M	1×NVL72 · ~$3.2M
90 days	52×H100 · ~$1.7M	35×H100 · ~$1.3M	1×NVL72 · ~$3.2M

GPT-3 175B: 3.14×10²³ FLOPs to train, 700 GB of weights. Peak compute (corrected): H100 SXM FP8 dense = 1979 TFLOPS; GB200 NVL72 = 720 PFLOPS/rack.
Effective efficiency: HBM ~40%, lattice ~60%, supernode ~55%. List price: H100 ~$30K/GPU, NVL72 ~$3.2M/rack, lattice adds ~$0.26M NVMe pool + PCIe fabric.
The 1/64 GPU ratio is a memory/bandwidth-bound scenario claim, not dense pretraining; order-of-magnitude comparison only — real numbers shift with electricity, depreciation and cloud discounts.

× Vast Data × NVIDIA Ecosystem · Technical Advantage · Source: Broadcom PEX89144 datasheet and public materials