Recorded on real hardware · replayed in real time

Watch a 125B model answer on one RTX 5070

RTX 5070 (12 GB) · AMD EPYC 7B13 64-Core Processor · Qwen3.8-Flash-Next IQ2_XS, recorded in a rented cloud box with ~31 CPU cores. One consumer graphics card holds the busiest experts; the rest of the model sits in ordinary RAM. A typical gaming PC with a 6-core CPU writes about 80 tokens/s on this card (LatticeRun's own measurement).

Writing

In three sentences: what can a 125-billion-parameter model running on my own gaming PC do for me that a small 8B model can't?

A 125-billion-parameter model offers significantly deeper reasoning and more nuanced contextual understanding, allowing it to handle complex multi-step problems that smaller models often oversimplify or hallucinate. It generates higher-quality, more coherent long-form outputs with better adherence to instructions, making it superior for creative writing, detailed analysis, and sophisticated code generation. However, running such a large model on a personal gaming PC requires substantial VRAM and may result in slower inference speeds compared to an 8B model, limiting its practicality for real-time tasks unless optimized or quantized.

Inside the PC

Graphics card 12 GB VRAM · 4,263 busiest experts 45,016expert uses

RAM + processor 33 GB of experts · all 24,576 8,744expert uses

SSD lookup table & spill 0reads

24,576 experts · 48 layers × 512 each token: 10 per layer

on the graphics card RAM, computed by the processor read from SSD

Writing speed 115tok/s

GPU busy60%

GPU cache hit83%

Processor82%

Guess, then check

A small helper guesses the next words; the big model checks them all at once.

guessed 88 kept 56 63% kept

Speed, timing, cache hit rate, RAM/SSD reads, helper guesses and GPU/RAM/CPU load are measured by LatticeRun during the recording. Which exact experts light up is illustrative: LatticeRun does not log per-token expert IDs, so the grid draws a pattern that matches the measured hit rate.

Run it on your own PC

LatticeRun is free and open source. A 12 GB graphics card and 32 to 64 GB of RAM are enough.

Will it run on my PC? →