LatticeRun bench

Community results · unofficial

Will a 125B model run on your PC?

LatticeRun runs Qwen3.8-Flash-Next on a gaming PC. This site collects every speed report people share on GitHub and tells you what to expect on your hardware.

1,061measurements
76GPU setups
245reports read

Check my PC

Fastest single cards

Best reported answer speed (decode) per card, one GPU only. Bigger quants are slower, so compare like with like in the table below.

NVIDIA AMD Intel Apple
rtx pro 6000 258
rtx 5090 233
rtx 6000 203
rtx 4090 182
rtx 5090 laptop 144
l40s 136
rtx pro 4500 124
rtx 5070 ti 118
rtx 4080 super 116
rtx 3090 112
rtx a6000 109
rtx 4080 104
rtx 5080 102
radeon ai pro r9700 102
v100 99
rtx 4070 ti super 98
rtx 3090 ti 95
rx 7900 xtx 91
rtx 5070 89
rx 9070 xt 80
rx 7900 xt 78
rtx 5060 ti 73
rtx pro 4000 71
rtx 3080 ti 69
rx 7900 gre 65
rx 9070 65
rtx 3080 62
radeon 8060s 59
rx 6900 xt 58
rx 9060 xt 58

All results

Pulled from bench/results/ and benchmark issues, read automatically. Always check the source report before you buy anything.

GPU VRAM RAM Model Prompt Writes
tok/s		 Reads
tok/s		 OS Date
rtx pro 6000 MTP enabled (T4) mode 96 GB 128 GB Flash-Next Q4_K_XL – 257.6 2,700 linux 2026-10-06 source
rtx 5090 32 GB 123 GB Flash-Next IQ3_S – 233 – linux 2026-10-11 source
rtx 5090 32 GB 96 GB Swift IQ3_XXS 3K 215 3,187 windows 2026-10-01 source
rtx 6000 84 GB 122 GB Swift 1.5 IQ2_XS 46 203.2 – linux 2026-10-12 source
rtx 5090 32 GB 92 GB Swift 1.5 IQ3_XXS – 198.9 6,687 linux 2026-10-09 source
rtx 5090 32 GB 120 GB Flash-Next IQ3_S – 195.8 – linux 2026-10-10 source
rtx 5090 32 GB 89 GB Flash-Next IQ3_S 4K 192.7 4,846 linux 2026-10-03 source
rtx 5090 32 GB 89 GB Flash-Next IQ3_S 31K 191.7 6,036 linux 2026-10-03 source
rtx 5090 32 GB 92 GB Swift 1.5 IQ3_XXS – 190.6 6,426 linux 2026-10-09 source
rtx 5090 32 GB 63 GB Swift 1.5 IQ3_S – 185.6 – windows 2026-10-12 source
rtx 4090 24 GB 32 GB Flash-Next IQ2_XS – 182 3,494 linux 2026-10-04 source
rtx 4090 24 GB 32 GB Flash-Next IQ2_XS – 182 3,904 linux 2026-10-04 source
rtx 4090 24 GB 32 GB Flash-Next IQ2_XS – 182 3,833 linux 2026-10-04 source
rtx 5090 33 GB 94 GB Flash-Next IQ3_XXS 32K 181.2 6,367 windows 2026-10-04 source
rtx 5090 32 GB 64 GB Flash-Next IQ2_XS – 179.4 4,270 linux 2026-09-30 source
rtx 5090 33 GB 94 GB Flash-Next IQ3_XXS 125K 178.3 6,080 windows 2026-10-04 source
rtx 5090 32 GB 64 GB Flash-Next IQ2_XS – 175.7 5,543 linux 2026-09-30 source
rtx 6000 84 GB 122 GB Flash-Next UD-Q4_K_XL – 171.8 – linux 2026-10-12 source
rtx 5090 33 GB 94 GB Flash-Next IQ3_XXS 4K 171.7 3,775 windows 2026-10-04 source
rtx 5090 32 GB 94 GB Flash-Next IQ3_S 256K 170.5 5,535 windows 2026-10-05 source
rtx 5090 32 GB 94 GB Flash-Next IQ3_S 125K 168.4 5,803 windows 2026-10-05 source
rtx 5090 32 GB 89 GB Flash-Next IQ3_S 1K 168 1,753 linux 2026-10-03 source
rtx 5090 32 GB 64 GB Flash-Next IQ2_XS – 165 5,779 linux 2026-09-30 source
rtx 5090 32 GB 94 GB Flash-Next IQ3_S 4K 164.3 3,072 windows 2026-10-05 source
2× rtx pro 4500 32 GB 64 GB Swift 1.5 IQ3_XXS 1K 162.3 1,889 linux 2026-10-06 source
rtx 5090 32 GB 94 GB Flash-Next IQ3_S 32K 161.3 5,963 windows 2026-10-05 source
2× rtx pro 4500 32 GB 64 GB Swift 1.5 IQ3_XXS 4K 154.8 3,073 linux 2026-10-06 source
rtx 5090 32 GB 89 GB Flash-Next IQ3_S 125K 152.3 5,862 linux 2026-10-03 source
rtx 5090 32 GB 96 GB Flash-Next IQ3_XXS – 150.5 – windows 2026-10-16 source
rtx pro 6000 48 GB 128 GB Flash-Next Q8_0 32K 147.3 4,150 – 2026-10-06 source
rtx 5080 + rtx 4060 ti – 96 GB Flash-Next IQ3_S 4K 146 3,451 linux 2026-10-09 source
rtx 5090 laptop 24 GB 64 GB Flash-Next Q2_0 32K 144.4 2,893 windows 2026-10-08 source
rtx pro 6000 48 GB 128 GB Flash-Next Q8_0 128K 143.5 4,099 – 2026-10-06 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 163 143.3 218 linux 2026-10-08 source
rtx 5090 32 GB 123 GB Flash-Next IQ3_S – 142.8 – linux 2026-10-11 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --kv-resident 32768 24 GB 32 GB Swift 1.5 IQ3_S – 142.6 – linux 2026-10-08 source
2× rtx pro 4500 32 GB 64 GB Swift 1.5 IQ3_XXS 243K 142.5 5,977 linux 2026-10-06 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --kv-resident off 24 GB 32 GB Swift 1.5 IQ3_S – 142.4 – linux 2026-10-08 source
2× rtx pro 4500 32 GB 64 GB Swift 1.5 IQ3_XXS 30K 142.3 5,244 linux 2026-10-06 source
rtx 5090 32 GB 96 GB Flash-Next IQ3_S 32K 142.2 4,168 windows 2026-10-02 source
rtx 5090 33 GB 94 GB Flash-Next IQ3_XXS 256K 141.8 5,800 windows 2026-10-04 source
rtx 5090 32 GB 96 GB Flash-Next IQ3_S 4K 141.8 2,473 windows 2026-10-02 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --batch 2, --kv-resident 32768 24 GB 32 GB Swift 1.5 IQ3_S – 140.3 – linux 2026-10-08 source
rtx pro 6000 MTP enabled (T4) mode 96 GB 128 GB Flash-Next Q8_0 – 140.2 640 linux 2026-10-06 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 31K 138.4 3,342 linux 2026-10-08 source
rtx 5090 laptop 24 GB 64 GB Flash-Next Q2_0 4K 138.1 1,950 windows 2026-10-08 source
rtx 5090 24 GB 64 GB Flash-Next Q2_0 – 137.7 2,878 windows 2026-10-06 source
rtx 5090 24 GB 64 GB Flash-Next Q2_0 – 136.9 1,953 windows 2026-10-06 source
rtx 5090 32 GB 96 GB Flash-Next IQ3_S 1K 136.2 961 windows 2026-10-02 source
l40s 48 GB 251 GB Flash-Next UD-IQ4_XS 32K 136 4,166 linux 2026-10-11 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 163 135.8 205 linux 2026-10-04 source
rtx 5090 + rtx 4000 ada – 125 GB Flash-Next IQ3_S – 135.7 – linux 2026-10-12 source
rtx 4090 24 GB 32 GB Flash-Next IQ2_XS 116K 135 3,975 linux 2026-10-04 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 32K 134.9 2,974 linux 2026-10-04 source
rtx 5090 24 GB 64 GB Flash-Next Q2_0 – 134.6 2,764 windows 2026-10-06 source
2× rtx pro 4500 32 GB 64 GB Swift 1.5 IQ3_XXS 118K 133.6 6,127 linux 2026-10-06 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS – 133 200 linux 2026-10-03 source
2× rtx pro 4500 32 GB 64 GB Swift 1.5 IQ3_XXS 1K 132.3 1,862 linux 2026-10-02 source
rtx 5080 + rtx 4060 ti – 96 GB Flash-Next IQ3_S 32K 132.2 4,277 linux 2026-10-09 source
rtx 5090 laptop 24 GB 64 GB Flash-Next Q2_0 125K 131 2,780 windows 2026-10-08 source
l40s 48 GB 251 GB Flash-Next UD-IQ4_XS 4K 130 2,341 linux 2026-10-11 source
2× rtx pro 4500 32 GB 64 GB Swift 1.5 IQ3_XXS 30K 130 5,065 linux 2026-10-02 source
rtx pro 6000 No MTP (plain) mode 96 GB 128 GB Flash-Next Q4_K_XL – 129.9 2,632 linux 2026-10-06 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 4K 128.2 2,146 linux 2026-10-08 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS – 127.9 2,976 linux 2026-10-03 source
rtx 5090 32 GB 96 GB Flash-Next IQ3_S 128K 127.2 4,450 windows 2026-10-02 source
2× rtx pro 4500 32 GB 64 GB Swift 1.5 IQ3_XXS 4K 127 2,960 linux 2026-10-02 source
rtx 4090 24 GB 32 GB Flash-Next IQ2_XS 233K 126.5 3,778 linux 2026-10-04 source
2× rtx 3090 24 GB 60 GB Flash-Next IQ3_S – 126 1,831 linux 2026-10-06 source
rtx 5080 + rtx 4060 ti – 96 GB Flash-Next IQ3_S 125K 124.9 4,166 linux 2026-10-09 source
2× rtx pro 4500 32 GB 64 GB Flash-Next IQ3_XXS 1K 124 903 linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB Flash-Next IQ3_S – 123.8 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB IQ3 IQ3_S – 123.8 667 linux 2026-10-01 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 4K 123.6 1,966 linux 2026-10-04 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 122K 121.7 3,292 linux 2026-10-08 source
rtx 4090 24 GB 32 GB Flash-Next IQ2_XS 466K 121.6 3,473 linux 2026-10-04 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 128K 120.3 3,085 linux 2026-10-04 source
rtx pro 4500 32 GB 512 GB Flash-Next IQ3_S – 120.3 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB IQ3 IQ3_S – 120.3 3,281 linux 2026-10-01 source
rtx 4090 24 GB 32 GB Flash-Next IQ2_XS 31K 120.1 3,791 linux 2026-10-04 source
2× rtx pro 4500 32 GB 64 GB Flash-Next IQ3_XXS 30K 120 2,454 linux 2026-10-01 source
rtx 4090 24 GB 64 GB Flash-Next IQ3_XXS – 120 131 windows 2026-09-30 source
rtx 4090 Whole request speed 98.1 tok/s (11485 tokens in 118 s). Peak decode ~120 tok/s, prefill 131 tok/s. Prompt length not spe 24 GB 64 GB Flash-Next IQ3_XXS – 120 131 windows 2026-09-30 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --batch 2 24 GB 32 GB Swift 1.5 IQ3_S – 119.8 – linux 2026-10-08 source
l40s 48 GB 251 GB Flash-Next UD-IQ4_XS 128K 119 4,014 linux 2026-10-11 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 128K 118.9 3,336 linux 2026-10-08 source
rtx 5070 ti single card, calibrated, --prefill auto, --pcie-frac 0.20, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next IQ3_S – 118.3 845 linux 2026-10-04 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS – 118 1,844 linux 2026-10-03 source
rtx pro 4500 32 GB 512 GB Flash-Next IQ3_S – 116.5 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB IQ3 IQ3_S – 116.5 3,102 linux 2026-10-01 source
rtx 4080 super 16 GB 63 GB Flash-Next IQ2_XS 46 116 – linux 2026-10-14 source
2× rtx pro 4500 32 GB 64 GB Flash-Next IQ3_XXS 4K 115.4 1,428 linux 2026-10-01 source
rtx 4090 24 GB 64 GB Flash-Next IQ2_XS – 115 147 windows 2026-09-30 source
rtx 4090 Whole request speed 106.1 tok/s (3127 tokens in 30 s). Peak decode ~115 tok/s, prefill 147 tok/s. Prompt length not spec 24 GB 64 GB Flash-Next IQ2_XS – 115 147 windows 2026-09-30 source
2× rtx 3090 24 GB 60 GB Flash-Next IQ3_S – 114.5 1,831 linux 2026-10-06 source
rtx 4090 KV-prefetch opt‑in enabled, 32 768‑token prompt 24 GB 7 GB Flash-Next IQ3_XXS – 113.8 3,002 – 2026-10-06 source
rtx 3060 + rtx 5060 ti + rtx 3090 plain mode – 123 GB Flash-Next IQ3_XXS 4K 113.5 1,254 – 2026-10-12 source
rtx 3090 24 GB 64 GB Flash-Next IQ2_XS 2K 112.2 188 windows 2026-10-06 source
rtx 3090 24 GB 64 GB Flash-Next IQ2_XS 26K 111.5 618 windows 2026-10-06 source
rtx 5060 ti + rtx 3090 + rtx 3060 16 GB 123 GB Flash-Next IQ3_XXS – 111.3 – linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 – 123 GB Flash-Next IQ3_XXS 4K 111.3 1,296 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 – 123 GB Flash-Next IQ3_XXS 16K 111.3 2,690 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 – 123 GB Flash-Next IQ3_XXS 31K 111.3 3,228 linux 2026-10-12 source
rtx 3060 + rtx 5060 ti + rtx 3090 resident-experts mode – 123 GB Flash-Next IQ3_XXS 4K 111.1 1,285 – 2026-10-12 source
rtx 3060 + rtx 5060 ti + rtx 3090 resident-experts mode – 123 GB Flash-Next IQ3_XXS 34K 110.7 3,143 – 2026-10-12 source
2× rtx 3090 24 GB 60 GB Flash-Next IQ3_S – 110.4 1,842 linux 2026-10-06 source
rtx 3060 + rtx 5060 ti + rtx 3090 plain mode – 123 GB Flash-Next IQ3_XXS 34K 110.3 3,097 – 2026-10-12 source
rtx 5090 32 GB 96 GB Flash-Next IQ3_XXS – 110.2 – windows 2026-10-16 source
rtx a6000 48 GB 125 GB Flash-Next IQ3_S – 108.6 – linux 2026-10-14 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS – 108.4 2,962 linux 2026-10-03 source
rtx 5090 32 GB 96 GB Flash-Next IQ3_XXS – 107.7 – windows 2026-10-16 source
2× rtx 3090 24 GB 60 GB Flash-Next IQ3_S – 107.4 1,831 linux 2026-10-06 source
2× rtx 3090 24 GB 60 GB Flash-Next IQ3_S – 107.1 1,842 linux 2026-10-06 source
rtx 5090 laptop + rtx 4090 laptop Long prompt of 424 tokens with 128-token generation – 96 GB Flash-Next IQ3_S – 107 321 windows 2026-10-08 source
rtx 4090 KV-prefetch opt‑in enabled, 131 072‑token prompt 24 GB 7 GB Flash-Next IQ3_XXS – 106.5 3,052 – 2026-10-06 source
rtx pro 4500 32 GB 512 GB Flash-Next IQ3_S – 106.3 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB IQ3 IQ3_S – 106.3 1,855 linux 2026-10-01 source
rtx 3060 + rtx 5060 ti + rtx 3090 resident-experts mode – 123 GB Flash-Next IQ3_XXS 136K 105.5 3,579 – 2026-10-12 source
l40s 48 GB 251 GB Flash-Next UD-Q6_K_XL 32K 105 2,211 linux 2026-10-11 source
rtx 5090 Cold prefill throughput and thinking decode median for ~4k prompt (actual 4,013 tokens). 33 GB 122 GB Flash-Next IQ3_S – 105 2,485 linux 2026-10-02 source
2× rtx pro 4500 32 GB 64 GB Swift 1.5 IQ3_XXS 118K 104.8 5,762 linux 2026-10-02 source
rtx 4080 16 GB 64 GB Flash-Next IQ2_XS 46 104.4 – windows 2026-10-12 source
rtx pro 6000 96 GB 124 GB ISTA IQ3_XXS 512K 103.6 5,479 linux 2026-10-09 source
rtx 3060 + rtx 5060 ti + rtx 3090 plain mode – 123 GB Flash-Next IQ3_XXS 136K 103.3 3,515 – 2026-10-12 source
rtx 5090 + rtx pro 4000 32 GB 128 GB Flash-Next UD-Q4_K_XL 5K 103.3 1,514 windows 2026-10-07 source
2× rtx 3090 24 GB 165 GB Coder IQ1_M – 103.3 – linux 2026-09-29 source
l40s 48 GB 251 GB Flash-Next UD-Q6_K_XL 4K 103 952 linux 2026-10-11 source
rtx 5060 ti + rtx 3090 + rtx 3060 short chat (decode only), median decode speed 102.6 tok/s, prompt length 46 tokens – 123 GB Flash-Next IQ3_S – 102.6 – linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 – 123 GB Flash-Next IQ3_S 4K 102.6 811 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 – 123 GB Flash-Next IQ3_S 16K 102.6 1,580 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 – 123 GB Flash-Next IQ3_S 31K 102.6 1,946 linux 2026-10-12 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_S 31K 102.5 2,991 linux 2026-10-03 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_S 124K 102.2 3,078 linux 2026-10-03 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 4K 102 2,199 linux 2026-10-08 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ2_XS 4K 102 1,992 windows 2026-10-06 source
rtx 5080 16 GB 96 GB Flash-Next IQ1_M – 102 3,588 windows 2026-10-05 source
rtx 5090 Cold prefill throughput and thinking decode median for ~32k prompt (actual 31,376 tokens). 33 GB 122 GB Flash-Next IQ3_S – 102 3,284 linux 2026-10-02 source
radeon ai pro r9700 32 GB 503 GB Flash-Next IQ3_S 1K 101.9 633 linux 2026-10-08 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ2_XS 32K 101.3 2,817 windows 2026-10-06 source
rtx 3090 24 GB 64 GB Flash-Next IQ2_XS 6K 101.1 560 windows 2026-10-06 source
rtx 5070 ti single card, calibrated, --prefill auto, --pcie-frac 0.20, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next IQ3_S – 100.3 2,822 linux 2026-10-04 source
rtx 5090 Cold prefill throughput and thinking decode median for ~128k prompt (actual 125,204 tokens). 33 GB 122 GB Flash-Next IQ3_S – 100 3,398 linux 2026-10-02 source
radeon ai pro r9700 32 GB 62 GB Flash-Next IQ2_XS 4K 99.8 856 windows 2026-10-03 source
rtx 5070 ti single card, calibrated, --prefill auto, --pcie-frac 0.20, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next IQ3_S – 99.4 3,373 linux 2026-10-04 source
l40s 48 GB 251 GB Flash-Next UD-Q6_K_XL 128K 99 2,146 linux 2026-10-11 source
v100 32 GB 32 GB Swift 1.5 IQ2_XS 6K 99 1,018 windows 2026-10-10 source
2× rtx 3090 24 GB 165 GB Coder IQ1_M – 98.6 – linux 2026-09-29 source
v100 32 GB 32 GB Swift 1.5 IQ2_XS 4K 98.5 793 windows 2026-10-10 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 32K 98.5 3,375 linux 2026-10-08 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_S 114K 98.3 2,313 windows 2026-10-11 source
rtx pro 4500 32 GB 512 GB Flash-Next UD-IQ4_XS – 98 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB Unsloth UD-IQ4_XS – 98 484 linux 2026-10-01 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ2_XS – 97.9 2,695 linux 2026-10-05 source
radeon ai pro r9700 32 GB 62 GB Flash-Next IQ2_XS 33K 97.8 1,182 windows 2026-10-03 source
rtx 3090 Cold short prompts (baseline v0142), median decode speed 24 GB 64 GB Flash-Next IQ3_S – 97.2 – windows 2026-10-11 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_S 32K 97.2 2,948 linux 2026-10-03 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ2_XS 29K 97.2 167 linux 2026-10-01 source
rtx 4080 super workload with ~6,845‑token prompt (decode only) 32 GB 96 GB Flash-Next IQ3_S – 97.1 – windows 2026-10-04 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_S 114K 97 2,173 windows 2026-10-11 source
rtx 3090 Long prompt 116,833 tokens (baseline v0142), prefill and decode speeds 24 GB 64 GB Flash-Next IQ3_S – 97 2,092 windows 2026-10-11 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 228 97 320 linux 2026-10-08 source
rtx 5090 laptop + rtx 4090 laptop Short prompt (length not specified) with 256-token generation – 96 GB Flash-Next IQ3_S – 97 – windows 2026-10-08 source
rx 7900 xtx + rx 6800 xt 24 GB 121 GB Flash-Next UD-Q4_K_XL – 97 – linux 2026-10-07 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_S 54K 96.5 2,313 windows 2026-10-11 source
radeon ai pro r9700 32 GB 503 GB Flash-Next IQ3_S 4K 96.4 768 linux 2026-10-08 source
v100 32 GB 32 GB Swift 1.5 IQ2_XS 1K 96 562 windows 2026-10-10 source
rtx pro 4500 32 GB 512 GB Flash-Next UD-IQ4_XS – 95.8 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB Unsloth UD-IQ4_XS – 95.8 3,053 linux 2026-10-01 source
radeon ai pro r9700 32 GB 503 GB Flash-Next IQ3_S 32K 95.6 751 linux 2026-10-08 source
rtx 4080 super Chinese prose workload with ~6,867‑token prompt (decode only) 32 GB 96 GB Flash-Next IQ3_S – 95.4 – windows 2026-10-04 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_S 54K 95.3 2,173 windows 2026-10-11 source
rtx 3090 Long prompt 55,630 tokens (baseline v0142), prefill and decode speeds 24 GB 64 GB Flash-Next IQ3_S – 95.3 2,173 windows 2026-10-11 source
rtx 3090 ti warm requests, node‑0 binding (31‑token prompt, 512 generated tokens) 24 GB 384 GB Flash-Next IQ3_S – 94.9 – linux 2026-10-03 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ2_XS – 94.3 2,678 linux 2026-10-05 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS – 94.2 1,824 linux 2026-10-03 source
l40s 48 GB 251 GB Flash-Next Q8_0 32K 94 1,817 linux 2026-10-11 source
rtx 4090 Default configuration (parallel off), ~4 000‑token prompt 24 GB 7 GB Flash-Next IQ3_XXS – 94 – – 2026-10-06 source
rtx 5090 Cold prefill throughput and thinking decode median for ~512k prompt (actual 500,558 tokens). 33 GB 122 GB Flash-Next IQ3_S – 94 3,019 linux 2026-10-02 source
v100 32 GB 32 GB Swift 1.5 IQ2_XS 17K 93.9 1,102 windows 2026-10-10 source
rtx 5090 + rtx pro 4000 32 GB 128 GB Flash-Next UD-Q4_K_XL 112K 93.9 2,201 windows 2026-10-07 source
rtx 4080 super English prose workload with ~5,121‑token prompt (decode only) 32 GB 96 GB Flash-Next IQ3_S – 93.8 – windows 2026-10-04 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 228 93.6 296 linux 2026-10-04 source
rtx 4080 super Code‑engine workload with ~4,170‑token prompt (decode only) 32 GB 96 GB Flash-Next IQ3_S – 93.5 – windows 2026-10-04 source
l40s 48 GB 251 GB Flash-Next IQ3_S 88K 93 3,950 linux 2026-10-05 source
2× rtx pro 4500 32 GB 64 GB Flash-Next IQ3_XXS 118K 93 2,785 linux 2026-10-01 source
rtx 5090 + rtx pro 4000 32 GB 128 GB Flash-Next UD-Q4_K_XL 32K 92.2 2,048 windows 2026-10-07 source
rtx 5070 ti single card, calibrated, --prefill auto, --pcie-frac 0.35, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next Q4_K_M – 92.1 668 linux 2026-10-04 source
l40s 48 GB 251 GB Flash-Next Q8_0 4K 92 778 linux 2026-10-11 source
v100 32 GB 32 GB Swift 1.5 IQ2_XS 6K 91.8 914 windows 2026-10-10 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_S – 91.6 – windows 2026-10-14 source
rtx pro 4500 32 GB 512 GB Flash-Next UD-IQ4_XS – 91.4 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB Unsloth UD-IQ4_XS – 91.4 2,893 linux 2026-10-01 source
rx 7900 xtx 24 GB 64 GB Flash-Next IQ3_XXS – 91.1 – windows 2026-10-17 source
rtx 3090 24 GB 121 GB Flash-Next IQ3_XXS 26K 91 – linux 2026-10-03 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 4K 91 1,331 linux 2026-10-01 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 16K 91 1,310 linux 2026-10-01 source
rtx 5060 ti + rtx 3090 + rtx 3060 short chat (≈46‑token prompt), median decode speed – 123 GB Flash-Next UD-IQ4_XS – 90.9 – linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 – 123 GB Flash-Next UD-IQ4_XS 4K 90.9 520 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 – 123 GB Flash-Next UD-IQ4_XS 16K 90.9 926 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 – 123 GB Flash-Next UD-IQ4_XS 31K 90.9 1,270 linux 2026-10-12 source
2× rtx 3080 20 GB 128 GB Flash-Next UD-Q4_K_XL 50K 90.6 2,585 linux 2026-10-08 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 32K 90.5 3,128 linux 2026-10-04 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS – 90.3 2,984 linux 2026-10-03 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ2_XS – 90 2,670 linux 2026-10-05 source
radeon ai pro r9700 32 GB 62 GB Flash-Next IQ2_XS 128K 89.9 1,232 windows 2026-10-03 source
2× rtx 3090 24 GB 60 GB Flash-Next IQ3_S – 89.4 1,842 linux 2026-10-06 source
2× rtx 2080 ti DDR4-2666, own+4096, layer-split 24, STRATA_STAGE_TRIM=1, STRATA_PREFILL_HELP=1, STRATA_BF16_TC=1, --pipeline-windows 2, 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 89.3 759 linux 2026-10-07 source
2× rtx 2080 ti DDR4-2666, own+4096, layer-split 24, STRATA_STAGE_TRIM=1, STRATA_PREFILL_HELP=1, STRATA_BF16_TC=1, --pipeline-windows 2, 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 89.3 975 linux 2026-10-07 source
2× rtx 2080 ti DDR4-2666, own+4096, layer-split 24, STRATA_STAGE_TRIM=1, STRATA_PREFILL_HELP=1, STRATA_BF16_TC=1, --pipeline-windows 2, 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 89.3 1,003 linux 2026-10-07 source
2× rtx 2080 ti DDR4-2666, own+4096, layer-split 24, STRATA_STAGE_TRIM=1, STRATA_PREFILL_HELP=1, STRATA_BF16_TC=1, --pipeline-windows 2, 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 89.3 1,395 linux 2026-10-07 source
2× rtx 2080 ti DDR4-2666, own+4096, layer-split 24, STRATA_STAGE_TRIM=1, STRATA_PREFILL_HELP=1, STRATA_BF16_TC=1, --pipeline-windows 2, 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 89.3 1,706 linux 2026-10-07 source
rtx 4090 24 GB 128 GB Flash-Next IQ1_M – 89.1 – windows 2026-10-13 source
l40s 48 GB 251 GB Flash-Next Q8_0 128K 89 1,779 linux 2026-10-11 source
rtx 5080 16 GB 96 GB Flash-Next IQ3_S – 89 2,974 windows 2026-10-05 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_S 163 88.7 226 linux 2026-10-03 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 4K 88.7 1,223 linux 2026-10-01 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 16K 88.7 1,324 linux 2026-10-01 source
rtx 5070 Generation speed for 4K prompt, default quantization. 12 GB 64 GB Flash-Next ? – 88.5 – windows 2026-09-28 source
v100 32 GB 32 GB Swift 1.5 IQ2_XS 2K 88.4 750 windows 2026-10-10 source
rx 7900 xtx ROCm 10.2 nightly warm steady‑state median decode speed (≈61 token prompt, 200 token generation). 24 GB 96 GB Flash-Next IQ3_S – 88.4 – linux 2026-10-03 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 128K 87.8 3,126 linux 2026-10-04 source
radeon ai pro r9700 32 GB 503 GB Flash-Next IQ3_S 128K 87.6 718 linux 2026-10-08 source
radeon ai pro r9700 32 GB 62 GB Flash-Next IQ2_XS 242K 87.5 1,130 windows 2026-10-03 source
2× rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_XXS – 86.5 2,087 linux 2026-10-05 source
2× rtx 3080 20 GB 128 GB Flash-Next UD-Q4_K_XL 102K 86.3 2,996 linux 2026-10-08 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS – 85.6 2,912 linux 2026-10-03 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 4K 85.2 1,712 linux 2026-10-01 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 16K 85.2 1,759 linux 2026-10-01 source
v100 + rtx 4070 – 16 GB Flash-Next IQ2_XS 4K 84.9 996 linux 2026-10-04 source
2× v100 prose (zh) content, 512K context 32 GB 125 GB Flash-Next IQ3_S – 84.8 – linux 2026-10-07 source
2× rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_XXS – 84.8 911 linux 2026-10-05 source
rtx 5070 12 GB 64 GB Flash-Next Q2_0 1K 84.7 519 windows 2026-09-29 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_S 128K 84.4 3,048 linux 2026-10-03 source
2× rtx 3080 20 GB 128 GB Flash-Next UD-Q4_K_XL 24K 84 2,219 linux 2026-10-08 source
2× rtx 2080 ti 22 GB 94 GB Flash-Next IQ3_XXS 100 84 – linux 2026-10-06 source
l40s 48 GB 251 GB Flash-Next UD-IQ4_XS 88K 84 3,300 linux 2026-10-05 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ2_XS 59K 83.9 128 linux 2026-10-01 source
2× rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_XXS – 83.8 2,440 linux 2026-10-05 source
rtx 4070 + v100 – 16 GB Flash-Next IQ2_XS 32K 83.8 1,535 linux 2026-10-04 source
rtx 5080 + rtx 3090 Prompt 16K tokens, story decode speed, split K=26 (best two‑GPU split). 16 GB 62 GB Coder IQ1_M – 83.8 2,039 windows 2026-09-29 source
rtx 5080 + rtx 3090 Prompt 28K tokens, story decode speed, split K=26 (best two‑GPU split). 16 GB 62 GB Coder IQ1_M – 83.8 2,357 windows 2026-09-29 source
rx 7900 xtx short chat (up to 100 generated tokens, prompt ~46 tokens) 24 GB 127 GB Swift 1.5 IQ2_XS – 83.2 – windows 2026-10-12 source
rtx 5080 Prompt 16K tokens, story decode speed, single RTX 5080 (this branch). 16 GB 62 GB Coder IQ1_M – 83.1 2,017 windows 2026-09-29 source
rtx 5080 Prompt 28K tokens, story decode speed, single RTX 5080 (this branch). 16 GB 62 GB Coder IQ1_M – 83.1 1,974 windows 2026-09-29 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS – 82.9 280 linux 2026-10-03 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 4K 82.9 1,336 linux 2026-10-01 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 16K 82.9 1,314 linux 2026-10-01 source
2× v100 code content, 512K context 32 GB 125 GB Flash-Next IQ3_S – 82.8 – linux 2026-10-07 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_S 128K 82.8 3,523 windows 2026-10-04 source
rtx pro 6000 96 GB 124 GB ISTA IQ3_XXS 1024K 82.7 4,505 linux 2026-10-09 source
rtx pro 6000 No MTP (plain) mode 96 GB 128 GB Flash-Next Q8_0 – 82.6 641 linux 2026-10-06 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_S 1K 82.5 521 linux 2026-10-17 source
rtx 3090 short chat (≈100 generated tokens, 46 prompt tokens) 24 GB 128 GB Flash-Next IQ3_S – 82.5 – windows 2026-10-12 source
v100 + p100 Decode speed for 128‑token answers (median of 79.6, 80.3, 84.7, 84.9 tok/s). 32 GB 120 GB Flash-Next IQ3_XXS – 82.5 – linux 2026-10-06 source
rtx pro 6000 96 GB 124 GB ISTA IQ3_XXS 1024K 82.4 4,452 linux 2026-10-09 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ2_XS – 82.4 2,484 linux 2026-10-05 source
rtx pro 4500 32 GB 512 GB Flash-Next UD-IQ4_XS – 82 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB Unsloth UD-IQ4_XS – 82 1,533 linux 2026-10-01 source
rtx 4080 laptop + rtx 3090 12 GB 32 GB Flash-Next IQ2_XS 4K 82 584 windows 2026-10-04 source
rtx 5070 12 GB 64 GB Flash-Next Q2_0 31K 81.8 2,171 windows 2026-09-29 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 32K 81.8 1,632 linux 2026-10-12 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 4K 81.7 1,215 linux 2026-10-01 source
radeon ai pro r9700 32 GB 62 GB Coder IQ1_M 16K 81.7 1,312 linux 2026-10-01 source
v100 + rtx 4070 – 16 GB Flash-Next IQ2_XS 32K 81.5 1,602 linux 2026-10-04 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_XXS 26K 81.3 468 windows 2026-10-06 source
rtx 5090 32 GB 96 GB Flash-Next UD-Q4_K_XL 14K 81.3 2,043 windows 2026-10-01 source
2× rtx 3080 20 GB 128 GB Flash-Next UD-Q4_K_XL 4K 81.2 718 linux 2026-10-08 source
rtx 3090 24 GB 64 GB Flash-Next IQ2_XS 25 81.1 – windows 2026-10-06 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_XXS 4K 81.1 1,908 linux 2026-10-04 source
rtx 4080 laptop + rtx 3090 12 GB 32 GB Flash-Next IQ2_XS 16K 80.9 1,046 windows 2026-10-04 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 32K 80.8 3,304 wsl2 2026-10-07 source
2× rtx 3080 20 GB 128 GB Flash-Next UD-Q4_K_XL 124K 80.7 3,080 linux 2026-10-08 source
v100 32 GB 32 GB Swift 1.5 IQ2_XS 32 80.4 – windows 2026-10-10 source
rx 7900 xtx ROCm 7.1.1 warm steady‑state median decode speed (≈61 token prompt, 200 token generation). 24 GB 96 GB Flash-Next IQ3_S – 80.4 – linux 2026-10-03 source
rx 9070 xt 16 GB 64 GB Flash-Next IQ2_XS 46 80.3 – windows 2026-10-17 source
2× titan rtx 24 GB 125 GB Flash-Next IQ3_S 32K 80.2 1,642 linux 2026-10-08 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_S 253K 79.9 1,480 linux 2026-10-17 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 8K 79.9 3,009 wsl2 2026-10-07 source
rx 7900 xtx 24 GB 192 GB Flash-Next IQ3_S 37K 79.7 1,612 windows 2026-10-06 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_S 160K 79.6 3,442 windows 2026-10-04 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 32K 79.5 1,636 linux 2026-10-12 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 16K 79.1 3,197 wsl2 2026-10-07 source
rtx 4070 + v100 – 16 GB Flash-Next IQ2_XS 4K 79.1 985 linux 2026-10-04 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 16K 78.5 3,001 wsl2 2026-10-07 source
rx 7900 xt 20 GB 64 GB Flash-Next IQ2_XS – 77.9 – windows 2026-10-13 source
rtx 5070 ti single card, calibrated, --prefill auto, --pcie-frac 0.35, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next Q4_K_M – 77.8 2,987 linux 2026-10-04 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 4K 77.7 850 linux 2026-10-12 source
2× v100 Decode speed with vision enabled on ~1700‑token generations 16 GB 64 GB Flash-Next IQ3_S – 77.5 – linux 2026-10-17 source
rx 7900 xtx 24 GB 192 GB Flash-Next IQ3_S 100 77.5 85 windows 2026-10-06 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_S 4K 77.5 2,059 linux 2026-10-03 source
2× rtx 5060 ti 16 GB 125 GB Flash-Next IQ3_XXS 4K 77.5 808 linux 2026-10-02 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_S – 77.5 500 – 2026-09-27 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_S 96K 77.4 1,810 linux 2026-10-17 source
rtx 5070 12 GB 64 GB Flash-Next Q2_0 31K 77.4 1,844 windows 2026-09-29 source
rtx 4080 laptop + rtx 3090 12 GB 32 GB Flash-Next IQ2_XS 1K 77.4 264 windows 2026-10-04 source
2× rx 6900 xt layer split, PRs, 4K prompt, --pipeline-windows 2 16 GB 128 GB Flash-Next IQ3_S – 77.2 968 linux 2026-10-09 source
rtx 5070 12 GB 64 GB Flash-Next IQ2_XS 1K 77.2 524 windows 2026-09-29 source
rtx 4070 + v100 – 16 GB Flash-Next IQ2_XS 125K 77.1 1,533 linux 2026-10-04 source
v100 + p100 32 GB 129 GB Flash-Next IQ3_XXS 809 77 425 linux 2026-10-07 source
rtx 5080 + rtx 3090 Speed projection, auto split 16 GB 62 GB Coder IQ1_M – 77 1,813 windows 2026-09-29 source
v100 v0.1.39 ready-made CUDA 12 engine, dual Xeon E5-2696 v3, 128 GB DDR3L-1600 ECC REG, PCIe Gen3 x16, Windows 10 Enterprise 32 GB 128 GB Flash-Next IQ3_XXS – 76.9 1,127 windows 2026-10-03 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 8K 76.4 2,782 wsl2 2026-10-07 source
v100 32 GB 96 GB Flash-Next Q2_0 – 76.4 1,268 windows 2026-10-05 source
rtx 5070 12 GB 64 GB Flash-Next IQ2_XS 31K 76.3 2,092 windows 2026-09-29 source
2× rtx 2080 ti 22 GB 94 GB Flash-Next IQ3_XXS 20K 76.1 1,183 linux 2026-10-06 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_S 4K 76.1 2,084 linux 2026-10-03 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ3_XXS – 76 2,747 linux 2026-10-05 source
2× rtx 2080 ti DDR4-2666, own+4096, layer-split 24, STRATA_STAGE_TRIM=1, STRATA_PREFILL_HELP=1, STRATA_BF16_TC=1, --pipeline-windows 2, 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 75.9 503 linux 2026-10-07 source
2× rtx 2080 ti DDR4-3600, own+4096, layer-split 24, STRATA_STAGE_TRIM=1, STRATA_PREFILL_HELP=1, STRATA_BF16_TC=1, --pipeline-windows 2, 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 75.9 521 linux 2026-10-07 source
2× rtx 2080 ti DDR4-2666, own+4096, no layer-split, no STRATA_STAGE_TRIM, no STRATA_PREFILL_HELP, --peer-device 0 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 75.9 593 linux 2026-10-07 source
2× rtx 2080 ti DDR4-2666, borrow+8192, no layer-split, no STRATA_STAGE_TRIM, no STRATA_PREFILL_HELP, --peer-device 0 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 75.9 557 linux 2026-10-07 source
2× rtx 2080 ti DDR4-2666, peer-device, no layer-split, no STRATA_STAGE_TRIM, no STRATA_PREFILL_HELP, --peer-device 1 22 GB 48 GB Qwen3.8-Flash-Next IQ3_XXS – 75.9 1,010 linux 2026-10-07 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_S 5K 75.8 3,012 windows 2026-10-04 source
rtx 5080 + rtx 3090 KV streaming (128K), auto split 16 GB 62 GB Coder IQ1_M – 75.8 1,685 windows 2026-09-29 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 4K 75.7 2,253 wsl2 2026-10-07 source
v100 32 GB 96 GB Flash-Next IQ2_XS – 75.7 1,165 windows 2026-10-05 source
2× titan rtx 24 GB 125 GB Flash-Next IQ3_S 4K 75.4 923 linux 2026-10-08 source
v100 + rtx 4070 – 16 GB Flash-Next IQ2_XS 125K 75.1 1,552 linux 2026-10-04 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 4K 74.8 848 linux 2026-10-12 source
rtx 3090 16K fresh prompt (16058 prompt tokens, 24 generated tokens) 24 GB 63 GB Flash-Next IQ4_XS – 74.7 2,043 linux 2026-10-11 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_S 40K 74.7 3,487 windows 2026-10-04 source
v100 + p100 32 GB 129 GB Flash-Next IQ3_XXS 43K 74.2 1,446 linux 2026-10-07 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_XXS 6K 74.2 421 windows 2026-10-06 source
rtx 5090 32 GB 96 GB Flash-Next UD-Q4_K_XL 3K 74.2 916 windows 2026-10-01 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ2_XS 118K 74 133 linux 2026-10-01 source
2× v100 Final calibrated decode speed (text only) after calibration 16 GB 64 GB Flash-Next IQ3_S – 73.9 – linux 2026-10-17 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 64K 73.8 3,299 wsl2 2026-10-07 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 64K 73.8 3,028 wsl2 2026-10-07 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_XXS 32K 73.7 2,482 windows 2026-10-06 source
3× rtx 4000 ada 3-GPU, layer split 20 GB 125 GB Flash-Next IQ3_S – 73.7 779 linux 2026-10-02 source
rtx 5070 12 GB 64 GB Flash-Next Q2_0 125K 73.7 2,107 windows 2026-09-29 source
rtx 5060 ti 16 GB 64 GB Flash-Next Q2_0 – 73.4 1,549 windows 2026-10-09 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 4K 73.1 2,194 wsl2 2026-10-07 source
2× rtx 5060 ti layer split, calibrated, --prefill auto, --pcie-frac 0.00, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next IQ3_S – 73 542 linux 2026-10-04 source
2× rtx 4000 ada 2-GPU, layer split 20 GB 125 GB Flash-Next IQ3_S – 73 2,110 linux 2026-10-02 source
v100 + p100 32 GB 129 GB Flash-Next IQ3_XXS 18K 72.8 1,437 linux 2026-10-07 source
2× rtx 5060 ti 16 GB 125 GB Flash-Next IQ3_XXS 32K 72.7 1,931 linux 2026-10-02 source
2× rtx 2080 ti 22 GB 94 GB Flash-Next IQ3_S 46 72.5 – linux 2026-10-12 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_XXS 4K 72.5 1,653 windows 2026-10-06 source
rtx 4090 48 GB 61 GB Flash-Next UD-Q4_K_XL 46 72.4 – linux 2026-10-15 source
2× rtx 5060 ti 16 GB 125 GB Flash-Next IQ3_XXS 125K 72.4 2,199 linux 2026-10-02 source
rtx pro 4500 32 GB 512 GB Flash-Next UD-Q4_K_XL – 72.4 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB Unsloth UD-Q4_K_XL – 72.4 330 linux 2026-10-01 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 128K 72.2 3,077 wsl2 2026-10-07 source
2× rx 7900 gre layer split, --resident-budget-gib 30 16 GB 63 GB Coder IQ1_M – 72.1 – linux 2026-10-06 source
rtx pro 4500 32 GB 512 GB Flash-Next UD-Q4_K_XL – 72.1 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB Unsloth UD-Q4_K_XL – 72.1 2,454 linux 2026-10-01 source
rtx 5080 + rtx 5060 ti Pass 2 – auto layer‑split configuration, short‑chat decode median speed 16 GB 94 GB Flash-Next UD-Q4_K_XL – 71.7 – linux 2026-10-13 source
rtx 3090 Prompt 16K tokens, story decode speed, single RTX 3090. 24 GB 62 GB Coder IQ1_M – 71.7 1,253 windows 2026-09-29 source
rtx 3090 Prompt 28K tokens, story decode speed, single RTX 3090. 24 GB 62 GB Coder IQ1_M – 71.7 1,211 windows 2026-09-29 source
rtx 5080 16 GB 55 GB Flash-Next IQ1_M 32K 71.5 3,035 wsl2 2026-10-07 source
rtx pro 4000 24 GB 220 GB Swift 1.5 Flash Next IQ3_S 256K 71.3 – linux 2026-10-15 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ3_XXS – 71.3 2,545 linux 2026-10-05 source
v100 32 GB 96 GB Flash-Next IQ2_XS – 71.2 1,183 windows 2026-10-05 source
2× v100 prose (en) content, 512K context 32 GB 125 GB Flash-Next IQ3_S – 70.9 – linux 2026-10-07 source
v100 + p100 Decode speed for 512‑token answer (median of 70.6, 72.2, 70.7 tok/s). 32 GB 120 GB Flash-Next IQ3_XXS – 70.7 – linux 2026-10-06 source
2× titan rtx 24 GB 125 GB Flash-Next IQ3_S 125K 70.6 1,854 linux 2026-10-08 source
3× rtx 4000 ada SFF secondary expert cache, 4,000 slots 20 GB 125 GB Flash-Next IQ3_S – 70.6 2,117 linux 2026-10-02 source
2× rx 6900 xt PRs, layer split, STRATA_HIP_PROMPT_F16=1 STRATA_SH_STREAM=1 STRATA_ROCBLAS_TUNING=/work/prs/tools/hip STRATA_SELECT_SGE 16 GB 128 GB Flash-Next IQ3_S – 70.4 2,003 linux 2026-10-06 source
2× rtx 5060 ti layer split, calibrated, --prefill auto, --pcie-frac 0.00, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next IQ3_S – 70.4 1,458 linux 2026-10-04 source
rtx 5080 16 GB 128 GB Flash-Next UD-IQ4_XS 32K 70.2 3,299 windows 2026-10-09 source
rtx pro 4500 32 GB 512 GB Flash-Next UD-Q4_K_XL – 70.2 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB Unsloth UD-Q4_K_XL – 70.2 2,351 linux 2026-10-01 source
rtx pro 4000 24 GB 221 GB Swift 1.5 IQ3_XXS – 70.1 – linux 2026-10-12 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ3_XXS – 70 2,478 linux 2026-10-05 source
v100 Prompt length 8192 tokens (prefill 8192), STRATA_SM70_TABLE=1. 32 GB 96 GB Flash-Next Q2_0 – 69.7 1,562 windows 2026-10-08 source
v100 + p100 32 GB 129 GB Flash-Next IQ3_XXS 117K 69.5 1,411 linux 2026-10-07 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_S 1K 69.4 646 linux 2026-10-04 source
2× rx 7900 gre layer split, --pipeline-windows 2 16 GB 63 GB Coder IQ1_M – 69.3 1,583 linux 2026-10-06 source
v100 32 GB 16 GB Flash-Next IQ1_M – 69.3 1,233 linux 2026-10-04 source
rtx 3080 ti 12 GB 128 GB Flash-Next IQ3_S – 69 – windows 2026-10-14 source
rtx 5080 16 GB 128 GB Flash-Next UD-IQ4_XS 4K 68.6 1,814 windows 2026-10-09 source
v100 32 GB 16 GB Flash-Next IQ1_M – 68.6 1,580 linux 2026-10-04 source
rtx 5060 ti 16 GB 64 GB Flash-Next Q2_0 – 68.5 1,514 windows 2026-10-09 source
2× rtx 4000 ada pure 2-GPU, current config 20 GB 125 GB Flash-Next IQ3_S – 68.5 2,118 linux 2026-10-02 source
rtx 4090 "parallel": 2 batch slots enabled, ~4 000‑token prompt 24 GB 7 GB Flash-Next IQ3_XXS – 68.5 – – 2026-10-06 source
2× rx 7900 gre layer split, --batch 2, one request 16 GB 63 GB Coder IQ1_M – 68.3 – linux 2026-10-06 source
2× rtx 5060 ti memory overclock (+1500 MHz), short prompt (104 tokens) 16 GB 128 GB Flash-Next IQ3_S – 68.2 55 windows 2026-10-02 source
2× titan rtx 24 GB 125 GB Flash-Next IQ3_S 4K 68.1 865 linux 2026-10-01 source
rtx 5060 ti 16 GB 64 GB Flash-Next Q2_0 – 68 782 windows 2026-10-09 source
2× rtx 3090 24 GB 128 GB Flash-Next UD-Q4_K_XL 46 67.9 – windows 2026-10-17 source
rtx 4080 super 16 GB 64 GB Flash-Next IQ2_XS – 67.9 – windows 2026-10-13 source
2× rx 7900 gre layer split, STRATA_STAGE_TRIM=0 16 GB 63 GB Coder IQ1_M – 67.6 1,103 linux 2026-10-06 source
2× rtx 3090 short chat (≈100 generated tokens, 46 prompt tokens) 24 GB 128 GB Flash-Next UD-Q4_K_XL – 67.5 – windows 2026-10-17 source
rx 7900 xtx short chat decode (100 tokens out, n=15) median of two runs 24 GB 128 GB Flash-Next IQ3_S – 67.5 – windows 2026-10-13 source
rtx 5070 12 GB 64 GB Flash-Next IQ2_XS 31K 67.5 1,799 windows 2026-09-29 source
v100 32 GB 16 GB Flash-Next IQ1_M – 67.1 1,394 linux 2026-10-04 source
rtx 3090 ti cold start, first request (31‑token prompt, 512 generated tokens) 24 GB 384 GB Flash-Next IQ3_S – 67.1 – linux 2026-10-03 source
2× rx 6900 xt post‑fix with --pcie-frac 0 16 GB 128 GB Flash-Next IQ3_S – 67 – linux 2026-10-04 source
v100 32 GB 32 GB Swift 1.5 IQ2_XS 15K 66.9 814 windows 2026-10-10 source
rtx 3090 24 GB 165 GB Coder IQ1_M – 66.8 – linux 2026-09-29 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 32K 66.7 1,563 linux 2026-10-07 source
2× rx 6900 xt post‑fix (commit ffca1b0) with default PCIe share (pcie_frac 0.39) 16 GB 128 GB Flash-Next IQ3_S – 66.7 – linux 2026-10-04 source
rtx 5060 ti + rtx 3060 16 GB – Flash-Next IQ3_S – 66.6 836 linux 2026-10-12 source
rtx 5060 ti + rtx 3060 16 GB – Flash-Next IQ3_S – 66.6 1,227 linux 2026-10-12 source
rtx 5060 ti + rtx 3060 16 GB – Flash-Next IQ3_S – 66.6 1,291 linux 2026-10-12 source
rtx 5080 KV streaming (128K), single GPU 16 GB 62 GB Coder IQ1_M – 66.6 1,487 windows 2026-09-29 source
2× v100 short chat (decode) test, prompt length ~46 tokens 16 GB 123 GB Flash-Next IQ3_S – 66.5 – linux 2026-10-17 source
rtx 3090 4K fresh prompt (4019 prompt tokens, 24 generated tokens) 24 GB 63 GB Flash-Next IQ4_XS – 66.5 1,397 linux 2026-10-11 source
rtx pro 4000 24 GB 220 GB Swift 1.5 Flash Next IQ3_S 512K 66.3 – linux 2026-10-15 source
rtx pro 4000 24 GB 221 GB Swift 1.5 IQ3_XXS 46 66.2 – linux 2026-10-12 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_S 32K 66.1 2,261 windows 2026-10-06 source
rtx 4080 16 GB 128 GB Flash-Next IQ3_S 46 66 – windows 2026-10-12 source
rtx 4090 23 GB 45 GB Flash-Next IQ3_S 228 65.9 313 linux 2026-10-03 source
rtx 5090 32 GB 96 GB Flash-Next UD-Q4_K_XL 103 65.9 87 windows 2026-10-01 source
rtx pro 4000 24 GB 220 GB Swift 1.5 Flash Next IQ3_S 128K 65.5 – linux 2026-10-15 source
rx 7900 gre 16 GB 64 GB Flash-Next IQ2_XS 9K 65.3 1,155 linux 2026-10-04 source
2× rtx 5060 ti layer split, calibrated, --prefill auto, --pcie-frac 0.00, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next IQ3_S – 65.2 2,141 linux 2026-10-04 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_XXS 1K 65.1 472 windows 2026-09-29 source
rx 7900 xtx + rx 6800 xt 24 GB 121 GB Flash-Next UD-Q4_K_XL – 65 – linux 2026-10-07 source
rx 9070 Short chat (prompt length not specified, median values). 16 GB 32 GB Flash-Next Q2_0 – 64.7 106 windows 2026-10-17 source
2× rx 6900 xt layer split, stock, 4K prompt 16 GB 128 GB Flash-Next IQ3_S – 64.7 464 linux 2026-10-09 source
2× rx 7900 gre layer split, --batch 4, four requests 16 GB 63 GB Coder IQ1_M – 64.6 – linux 2026-10-06 source
rx 7900 gre 16 GB 64 GB Flash-Next IQ2_XS 4K 64.5 1,146 linux 2026-10-04 source
rtx 3090 short chat (≈46 prompt tokens, 100 generated tokens) 24 GB 63 GB Flash-Next IQ4_XS – 64.4 – linux 2026-10-11 source
2× rtx 5060 ti layer split, calibrated, --prefill auto, --pcie-frac 0.00, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next Q4_K_M – 64.4 452 linux 2026-10-04 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 125K 64.4 1,728 linux 2026-10-07 source
rtx 5080 16 GB 128 GB Flash-Next UD-IQ4_XS 126K 64.1 3,398 windows 2026-10-09 source
2× rx 6900 xt baseline with --pcie-frac 0 (work‑around) 16 GB 128 GB Flash-Next IQ3_S – 64.1 – linux 2026-10-04 source
2× rx 7900 gre layer split, --pipeline-windows 1 16 GB 63 GB Coder IQ1_M – 63.8 – linux 2026-10-06 source
rtx 5070 12 GB 64 GB Flash-Next Q2_0 125K 63.1 1,682 windows 2026-09-29 source
rx 7900 xtx 24 GB 184 GB Swift 1.5 IQ3_S – 63 – linux 2026-10-12 source
rtx 5070 ti 16 GB 96 GB Flash-Next IQ3_XXS – 63 – windows 2026-10-11 source
3× rtx 5060 ti short chat (decode) test, prompt length ~46 tokens 16 GB 224 GB Flash-Next UD-Q6_K_XL – 62.9 – linux 2026-10-17 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_S 4K 62.9 1,439 windows 2026-10-06 source
2× rtx 5060 ti stock settings, short prompt (104 tokens) 16 GB 128 GB Flash-Next IQ3_S – 62.9 56 windows 2026-10-02 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 4K 62.8 460 linux 2026-10-05 source
rtx 5070 12 GB 64 GB Flash-Next IQ2_XS 125K 62.7 1,752 windows 2026-09-29 source
rtx pro 4000 short chat (≈100 generated tokens, prompt ≈46 tokens) 24 GB 221 GB Flash-Next IQ3_S – 62.6 – linux 2026-10-12 source
2× rtx 2080 ti 22 GB 94 GB Flash-Next IQ3_XXS 155K 62.5 1,388 linux 2026-10-06 source
rtx 5080 + rtx 3090 K=30, no helper 16 GB 62 GB Coder IQ1_M – 62.5 – windows 2026-09-29 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_S 128K 62.3 1,687 linux 2026-10-04 source
rtx pro 4000 24 GB 221 GB Flash-Next IQ3_S 46 62.2 – linux 2026-10-12 source
2× rtx 4060 ti greedy decoding (no sampling) – reported ~62 tokens/s generation speed 16 GB – Flash-Next UD-IQ4_XS – 62 – – 2026-10-11 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 32K 62 473 linux 2026-10-05 source
2× rtx 5060 ti memory overclock (+1500 MHz), long-cold prompt (~20,909 tokens) 16 GB 128 GB Flash-Next IQ3_S – 61.8 1,125 windows 2026-10-02 source
v100 v0.1.39 ready-made CUDA 12 engine, dual Xeon E5-2696 v3, 128 GB DDR3L-1600 ECC REG, PCIe Gen3 x16, Windows 10 Enterprise 32 GB 128 GB Flash-Next IQ3_S – 61.6 1,101 windows 2026-10-03 source
2× titan rtx 24 GB 125 GB Flash-Next IQ3_S 125K 61.6 1,785 linux 2026-10-01 source
rtx 3080 10 GB 64 GB Flash-Next IQ3_S 127K 61.5 1,529 – 2026-10-14 source
rtx pro 4500 32 GB 512 GB Flash-Next UD-Q4_K_XL – 61.5 – linux 2026-10-01 source
rtx pro 4500 32 GB 512 GB Unsloth UD-Q4_K_XL – 61.5 1,149 linux 2026-10-01 source
rx 9070 NixOS, RDNA 4, Ryzen 9 9900X, build-fhs environment, sustained generation speeds averaging between 55 and 70 tok/s, prom 16 GB 64 GB Swift 1.5 IQ3_XXS – 61.4 1,500 linux 2026-10-01 source
rtx 3080 10 GB 64 GB Flash-Next IQ3_S 127K 61.2 1,608 – 2026-10-14 source
2× rtx 5060 ti memory overclock (+1500 MHz), long-reuse prompt (~20,895 fresh tokens, 16,384 reused) 16 GB 128 GB Flash-Next IQ3_S – 61.1 581 windows 2026-10-02 source
rtx 5080 16 GB 128 GB Flash-Next UD-IQ4_XS 346K 60.7 3,111 windows 2026-10-09 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_S 128K 60.7 926 linux 2026-10-04 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_XXS 25 60.3 – windows 2026-10-06 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_XXS 2K 60.2 128 windows 2026-10-06 source
rtx 5060 ti 16 GB 61 GB Flash-Next IQ3_S – 60.2 1,225 linux 2026-10-06 source
2× arc pro b60 24 GB 64 GB Flash-Next IQ2_XS 23 60 – linux 2026-10-04 source
2× rtx 4070 short chat (up to 100 new tokens), prompt length 46 tokens 24 GB 31 GB Flash-Next IQ1_M – 59.6 – linux 2026-10-13 source
2× titan rtx 24 GB 125 GB Flash-Next IQ3_S 32K 59.5 1,612 linux 2026-10-01 source
4× rx 7900 xt 20 GB 60 GB Flash-Next IQ3_S 4K 59.3 756 linux 2026-10-09 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 4K 59.3 853 linux 2026-10-05 source
radeon 8060s 112 GB 128 GB Flash-Next UD-IQ4_XS 8K 59.2 969 linux 2026-10-09 source
rx 7900 gre 16 GB 64 GB Flash-Next IQ3_XXS 9K 59.1 1,140 linux 2026-10-04 source
rtx 5070 12 GB 64 GB Flash-Next IQ2_XS 125K 59.1 1,495 windows 2026-09-29 source
2× rtx 4060 ti sampling (T=1.0, top-p=0.95, top-k=20) – reported ~59 tokens/s generation speed 16 GB – Flash-Next UD-IQ4_XS – 59 – – 2026-10-11 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 32K 59 1,616 linux 2026-10-05 source
rx 9070 16 GB 64 GB Flash-Next IQ3_XXS 4K 58.7 586 windows 2026-10-10 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_XXS 31K 58.5 1,745 windows 2026-09-29 source
2× rtx 5060 ti layer split, calibrated, --prefill auto, --pcie-frac 0.00, context 262144, KV int8, expert cache auto, vision encoder on 16 GB 96 GB Flash-Next Q4_K_M – 58.4 2,134 linux 2026-10-04 source
rtx pro 4000 24 GB 221 GB Flash-Next IQ3_S 46 58.2 – linux 2026-10-12 source
rtx 5080 16 GB 96 GB Flash-Next UD-IQ4_XS – 58 1,945 windows 2026-10-05 source
rtx 4090 24 GB 192 GB Flash-Next IQ3_S – 58 – linux 2026-09-27 source
rx 6900 xt final automatic default (63 workers auto), 4K input (≈4,022 tokens) 16 GB 128 GB Coder IQ1_M – 57.9 – windows 2026-10-03 source
rtx 3090 32K fresh prompt (32049 prompt tokens, 24 generated tokens) 24 GB 63 GB Flash-Next IQ4_XS – 57.9 2,123 linux 2026-10-11 source
rx 9070 16 GB 64 GB Flash-Next IQ3_XXS 33K 57.9 805 windows 2026-10-10 source
rx 9070 16 GB 64 GB Flash-Next IQ3_XXS 127K 57.9 833 windows 2026-10-10 source
rx 9060 xt 16 GB 64 GB Flash-Next IQ2_XS 28K 57.9 347 windows 2026-10-10 source
rtx 5070 windows, --spec 4, --mtp rt 12 GB 64 GB Flash-Next IQ3_XXS 32K 57.9 1,618 windows 2026-10-06 source
rx 9060 xt 16 GB 64 GB Flash-Next IQ2_XS 28K 57.8 811 windows 2026-10-10 source
radeon 8060s 112 GB 128 GB Flash-Next UD-IQ4_XS 32K 57.8 1,067 linux 2026-10-09 source
rx 9060 xt 16 GB 64 GB Flash-Next IQ2_XS 28K 57.6 807 windows 2026-10-10 source
rtx 5080 + rtx 5060 ti Pass 1 – tuned layer‑split configuration, short‑chat decode median speed 16 GB 94 GB Flash-Next UD-Q4_K_XL – 57.4 – linux 2026-10-13 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 125K 57.4 1,823 linux 2026-10-05 source
rtx 5060 ti 16 GB 61 GB Flash-Next IQ3_S – 57.2 1,176 linux 2026-10-06 source
2× arc pro b60 24 GB 64 GB Coder IQ1_M 2K 57.2 425 linux 2026-10-04 source
v100 short chat (256‑token prompt) median decode speed across three runs 32 GB 180 GB Flash-Next IQ2_XS – 57.1 – linux 2026-10-07 source
rx 9060 xt 16 GB 64 GB Flash-Next IQ2_XS 28K 57 593 windows 2026-10-10 source
2× rtx 5060 ti stock settings, long-reuse prompt (~20,895 fresh tokens, 16,384 reused) 16 GB 128 GB Flash-Next IQ3_S – 56.8 574 windows 2026-10-02 source
rtx 5060 ti 16 GB 61 GB Flash-Next IQ3_S – 56.6 1,158 linux 2026-10-06 source
2× rx 7900 gre layer split, --batch 2, two requests 16 GB 63 GB Coder IQ1_M – 56.6 – linux 2026-10-06 source
2× rtx 4060 ti 16 GB 128 GB Flash-Next UD-IQ4_XS 32K 56.6 1,917 linux 2026-10-05 source
2× arc pro b60 24 GB 64 GB Coder IQ1_M 23 56.6 – linux 2026-10-04 source
rtx 3090 24 GB 62 GB Flash-Next IQ3_XXS – 56.5 948 linux 2026-10-15 source
2× rtx 5060 ti stock settings, long-cold prompt (~20,909 tokens) 16 GB 128 GB Flash-Next IQ3_S – 56.4 1,108 windows 2026-10-02 source
rx 9060 xt 16 GB 64 GB Flash-Next IQ2_XS 8K 56.3 879 windows 2026-10-10 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 125K 56.3 458 linux 2026-10-05 source
rtx 4070 ti super 16 GB 94 GB Flash-Next IQ3_XXS – 56 2,404 linux 2026-10-05 source
rtx 4060 ti 16 GB 96 GB Flash-Next Q2_0 – 55.7 – linux 2026-10-05 source
2× rtx 4060 ti 16 GB 128 GB Flash-Next UD-IQ4_XS 125K 55.7 2,234 linux 2026-10-05 source
rx 6800 16 GB 64 GB Flash-Next Q2_0 – 55.5 336 windows 2026-10-04 source
rx 6800 16 GB 64 GB Flash-Next Q2_0 – 55.5 321 windows 2026-10-04 source
2× arc pro b60 24 GB 64 GB Flash-Next IQ2_XS 2K 55.5 445 linux 2026-10-04 source
2× rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S – 55.5 – windows 2026-10-01 source
rx 9060 xt 16 GB 64 GB Flash-Next IQ2_XS 8K 55.3 876 windows 2026-10-10 source
2× rtx 4060 ti 16 GB 128 GB Flash-Next UD-IQ4_XS 32K 55.3 1,934 linux 2026-10-05 source
v100 32 GB 32 GB Swift 1.5 IQ2_XS – 55 1,295 windows 2026-10-10 source
rx 9070 xt RAM I/O prefill 78k tokens 16 GB 128 GB Flash-Next IQ3_S – 54.9 1,644 – 2026-10-06 source
rtx 5070 12 GB 64 GB Flash-Next CODER 31K 54.9 2,177 windows 2026-09-29 source
rtx 5060 ti short chat (up to 100 generated tokens), prompt length ~46 tokens 16 GB 64 GB Flash-Next IQ3_S – 54.7 – windows 2026-10-13 source
2× rtx 4060 ti 16 GB 128 GB Flash-Next UD-IQ4_XS 125K 54.7 2,221 linux 2026-10-05 source
rx 7900 gre 16 GB 64 GB Coder IQ1_M 9K 54.7 1,213 linux 2026-10-04 source
rtx 5070 windows, --spec 4, --mtp rt 12 GB 64 GB Flash-Next IQ3_XXS 32K 54.4 1,606 windows 2026-10-06 source
2× rtx 4060 ti 16 GB 128 GB Flash-Next UD-IQ4_XS 4K 54.4 880 linux 2026-10-05 source
rx 7900 gre 16 GB 64 GB Flash-Next IQ3_XXS 4K 54.2 1,148 linux 2026-10-04 source
rx 6800 16 GB 64 GB Flash-Next IQ2_XS – 53.8 326 windows 2026-10-04 source
rx 6800 16 GB 64 GB Flash-Next IQ2_XS – 53.8 311 windows 2026-10-04 source
rtx 5070 12 GB 64 GB Coder IQ3_S 1K 53.3 599 windows 2026-09-28 source
rtx 5070 12 GB 64 GB Coder IQ3_S 31K 53.3 1,298 windows 2026-09-28 source
rx 9060 xt 16 GB 64 GB Flash-Next IQ2_XS 8K 53.2 346 windows 2026-10-10 source
rtx 5070 windows, --spec 4, --mtp rt 12 GB 64 GB Flash-Next IQ3_XXS 4K 52.9 1,282 windows 2026-10-06 source
2× rtx 2080 ti 22 GB 62 GB Flash-Next IQ3_XXS 4K 52.8 411 linux 2026-10-06 source
rx 9060 xt 16 GB 64 GB Flash-Next IQ2_XS 8K 52.7 593 windows 2026-10-10 source
rx 9070 xt direct I/O (default) prefill 78k tokens 16 GB 128 GB Flash-Next IQ3_S – 52.6 1,660 – 2026-10-06 source
2× rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S – 52.3 – windows 2026-10-01 source
rtx pro 4000 24 GB 220 GB Swift 1.5 Flash Next IQ3_S 1024K 52.1 – linux 2026-10-15 source
rx 6800 16 GB 64 GB Flash-Next IQ3_XXS – 52.1 – windows 2026-10-04 source
rx 6800 16 GB 64 GB Flash-Next IQ3_XXS – 52.1 – windows 2026-10-04 source
4× rx 7900 xt 20 GB 60 GB Flash-Next IQ3_S 32K 52 1,722 linux 2026-10-09 source
2× rtx 4060 ti 16 GB 128 GB Flash-Next UD-IQ4_XS 4K 51.9 906 linux 2026-10-05 source
rx 6900 xt PRs, one card, STRATA_HIP_PROMPT_F16=1 STRATA_SH_STREAM=1 STRATA_ROCBLAS_TUNING=/work/prs/tools/hip STRATA_SELECT_SGEMM= 16 GB 128 GB Flash-Next IQ3_S – 51.8 969 linux 2026-10-06 source
rx 6900 xt single card, switches, 32K prompt 16 GB 128 GB Flash-Next IQ3_S – 51.6 1,102 linux 2026-10-09 source
rtx 5080 16 GB 128 GB Flash-Next UD-IQ4_XS 344K 51.3 2,253 windows 2026-10-09 source
rtx 5080 16 GB 128 GB Flash-Next UD-IQ4_XS 344K 51.3 2,635 windows 2026-10-09 source
rx 6900 xt stock, one card, STRATA_HIP_PROMPT_F16=1 STRATA_SH_STREAM=1, 128K prompt 16 GB 128 GB Flash-Next IQ3_S – 51.3 913 linux 2026-10-06 source
2× rtx 5060 ti 16 GB 224 GB Flash-Next UD-Q6_K_XL 46 51.2 – linux 2026-10-17 source
rtx 5090 laptop 24 GB 95 GB Flash-Next UD-IQ4_XS 46 51.2 – windows 2026-10-13 source
radeon 8060s – 128 GB Flash-Next Q2_0 320 51.2 159 linux 2026-10-05 source
rtx 4090 laptop Context 131,072 tokens, 4,096 prompt tokens 16 GB 64 GB Flash-Next IQ3_S – 51.2 1,345 linux 2026-10-04 source
rtx 5060 ti 16 GB 64 GB Flash-Next IQ1_M – 51 1,601 windows 2026-10-09 source
rx 6900 xt single card, PRs, 32K prompt 16 GB 128 GB Flash-Next IQ3_S – 50.9 1,174 linux 2026-10-09 source
rx 6900 xt PRs, one card, STRATA_HIP_PROMPT_F16=1 STRATA_SH_STREAM=1 STRATA_ROCBLAS_TUNING=/work/prs/tools/hip STRATA_SELECT_SGEMM= 16 GB 128 GB Flash-Next IQ3_S – 50.9 1,152 linux 2026-10-06 source
rtx 5070 12 GB 64 GB Coder IQ3_S 63K 50.8 1,350 windows 2026-09-28 source
2× radeon pro vii 16 GB – Flash-Next IQ3_XXS 64K 50.7 625 – 2026-10-08 source
rtx 5070 12 GB 64 GB Coder IQ3_S 4K 50.6 1,152 windows 2026-09-28 source
rx 6900 xt stock, one card, STRATA_HIP_PROMPT_F16=1 STRATA_SH_STREAM=1, 4K prompt 16 GB 128 GB Flash-Next IQ3_S – 50.5 811 linux 2026-10-06 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_S 1K 50.5 419 windows 2026-09-29 source
rtx 3080 ti short chat (≈46 prompt tokens, 100 generated tokens) 12 GB 64 GB Flash-Next IQ3_S – 50.4 – windows 2026-10-14 source
2× mi50 16 GB 32 GB Flash-Next IQ1_M – 50.1 321 linux 2026-10-03 source
rx 6900 xt PRs, one card, STRATA_HIP_PROMPT_F16=1 STRATA_SH_STREAM=1 STRATA_ROCBLAS_TUNING=/work/prs/tools/hip STRATA_SELECT_SGEMM= 16 GB 128 GB Flash-Next IQ3_S – 50 1,126 linux 2026-10-06 source
rtx 4090 laptop Context 131,072 tokens, 32,768 prompt tokens 16 GB 64 GB Flash-Next IQ3_S – 49.9 2,506 linux 2026-10-04 source
rx 6900 xt single card, PRs, 128K prompt 16 GB 128 GB Flash-Next IQ3_S – 49.8 1,147 linux 2026-10-09 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_XXS 31K 49.8 1,555 windows 2026-09-29 source
rtx 5070 windows, --spec 4, --mtp rt 12 GB 64 GB Flash-Next IQ3_XXS 4K 49.7 1,273 windows 2026-10-06 source
4× rx 7900 xt 20 GB 60 GB Flash-Next IQ3_S 125K 49.6 2,433 linux 2026-10-09 source
rx 6900 xt single card, switches, 128K prompt 16 GB 128 GB Flash-Next IQ3_S – 49.3 1,027 linux 2026-10-09 source
rtx 5060 ti 16 GB 64 GB Flash-Next IQ1_M – 49.1 1,860 windows 2026-10-09 source
rtx 3080 10 GB 64 GB Flash-Next IQ3_S 127K 49 628 – 2026-10-14 source
rx 6900 xt single card, switches, 4K prompt 16 GB 128 GB Flash-Next IQ3_S – 49 850 linux 2026-10-09 source
rx 6900 xt single card, PRs, 4K prompt 16 GB 128 GB Flash-Next IQ3_S – 49 976 linux 2026-10-09 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_XXS 125K 49 1,602 windows 2026-09-29 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_S 31K 49 1,499 windows 2026-09-29 source
2× rtx 3060 short chat (decode) test, prompt length ~46 tokens 12 GB 63 GB Flash-Next IQ3_S – 48.9 – linux 2026-10-15 source
rx 6900 xt stock, one card, STRATA_HIP_PROMPT_F16=1 STRATA_SH_STREAM=1, 32K prompt 16 GB 128 GB Flash-Next IQ3_S – 48.5 973 linux 2026-10-06 source
rx 7900 gre 16 GB 64 GB Coder IQ1_M 4K 48.5 1,224 linux 2026-10-04 source
rx 6900 xt stock, one card, GFXOFF default, 128K prompt 16 GB 128 GB Flash-Next IQ3_S – 48.3 459 linux 2026-10-06 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_S 31K 48.3 1,624 windows 2026-09-29 source
rtx 4070 Median accepted decode tokens per second over five runs (E4/S24 configuration). Prompt length 28,912 tokens. 12 GB 32 GB Flash-Next IQ2_XS – 48.2 – windows 2026-10-08 source
rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S 48 48.2 45 wsl2 2026-10-09 source
rtx 4060 ti 16 GB 96 GB Flash-Next Q2_0 – 48.1 – linux 2026-10-05 source
rtx 4090 laptop Context 131,072 tokens, 128,000 prompt tokens 16 GB 64 GB Flash-Next IQ3_S – 48.1 2,386 linux 2026-10-04 source
rx 6900 xt single card, stock, 32K prompt 16 GB 128 GB Flash-Next IQ3_S – 48 502 linux 2026-10-09 source
rtx 5080 16 GB 96 GB Flash-Next UD-Q4_K_XL – 48 1,550 windows 2026-10-05 source
rx 6900 xt 16 GB 128 GB Coder IQ1_M 94 48 47 windows 2026-10-03 source
2× mi50 16 GB 32 GB Flash-Next IQ1_M – 47.8 523 linux 2026-10-03 source
rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S 47 47.7 42 wsl2 2026-10-09 source
radeon 8060s – 128 GB Flash-Next UD-IQ4_XS 125K 47.5 1,109 linux 2026-10-09 source
rx 6900 xt stock, one card, GFXOFF default, 4K prompt, adaptive swaps on (default, --adapt-every 4, not passed, but on in every con 16 GB 128 GB Flash-Next IQ3_S – 47.5 458 linux 2026-10-06 source
rtx 5060 ti 16 GB 64 GB Flash-Next IQ1_M – 47.5 1,180 windows 2026-10-09 source
rx 6900 xt fixed affinity 63 workers, 4K input (≈4,022 tokens) 16 GB 128 GB Coder IQ1_M – 47.4 – windows 2026-10-03 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_XXS 125K 47 1,386 windows 2026-09-29 source
radeon 8060s – 128 GB Flash-Next Q2_0 – 47 – linux 2026-10-05 source
rx 9060 xt 16 GB 64 GB Flash-Next IQ2_XS 111K 46.9 790 windows 2026-10-10 source
rx 6900 xt stock, one card, GFXOFF default, 32K prompt 16 GB 128 GB Flash-Next IQ3_S – 46.9 473 linux 2026-10-06 source
radeon 8060s – 128 GB Flash-Next Q2_0 – 46.8 – linux 2026-10-05 source
rtx 5070 stock (baseline) speed, median of three runs (44.34, 46.74, 47.02 tok/s) on same tokens – – Flash-Next Q2_0 – 46.7 – – 2026-09-27 source
rx 6900 xt single card, stock, 128K prompt 16 GB 128 GB Flash-Next IQ3_S – 46.6 486 linux 2026-10-09 source
rtx 5070 with experimental speed projection vector, median of three runs (44.26, 46.56, 46.83 tok/s) on same tokens – – Flash-Next Q2_0 – 46.6 – – 2026-09-27 source
rx 9070 Fresh 32K prompt (32049 tokens). 16 GB 32 GB Flash-Next Q2_0 – 46.5 305 windows 2026-10-17 source
rx 6900 xt fixed affinity 63 workers, short input (≈94 tokens) 16 GB 128 GB Coder IQ1_M – 46.1 – windows 2026-10-03 source
rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 9K 46 744 linux 2026-10-04 source
rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 34K 46 915 linux 2026-10-04 source
rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 103K 46 926 linux 2026-10-04 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 8K 46 833 linux 2026-10-04 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 33K 46 1,356 linux 2026-10-04 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 8K 46 798 linux 2026-10-04 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 33K 46 887 linux 2026-10-04 source
rx 6900 xt 16 GB 128 GB Coder IQ1_M 32K 46 352 windows 2026-10-03 source
radeon 8060s – 128 GB Flash-Next UD-IQ4_XS 16K 45.9 729 linux 2026-10-09 source
2× mi50 16 GB 32 GB Flash-Next IQ1_M – 45.7 517 linux 2026-10-03 source
rx 6900 xt single card, stock, 4K prompt 16 GB 128 GB Flash-Next IQ3_S – 45.6 473 linux 2026-10-09 source
rtx 3060 12 GB 64 GB Flash-Next IQ2_XS 4K 45.5 829 windows 2026-10-09 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_S 125K 45.5 1,443 windows 2026-09-29 source
rtx 3060 12 GB 64 GB Flash-Next Q2_0 128 45.3 177 windows 2026-10-09 source
rx 6900 xt 16 GB 128 GB Coder IQ1_M 4K 45.3 313 windows 2026-10-03 source
rx 9070 xt 16 GB 128 GB Flash-Next IQ3_S – 45.2 – windows 2026-10-02 source
arc pro b65 prefill 512, 8K context, 512 input tokens 32 GB 128 GB Flash-Next IQ2_XS – 45 352 linux 2026-10-07 source
rtx 5080 16 GB 96 GB Flash-Next UD-IQ4_XS – 45 1,961 windows 2026-10-05 source
v100 Prompt length 7,194 tokens (new kernel speed). 32 GB 125 GB Flash-Next IQ4_XS – 45 1,164 linux 2026-10-03 source
v100 Prompt length 28,650 tokens (new kernel speed). 32 GB 125 GB Flash-Next IQ4_XS – 45 1,251 linux 2026-10-03 source
v100 Prompt length 114,338 tokens (new kernel speed). 32 GB 125 GB Flash-Next IQ4_XS – 45 1,123 linux 2026-10-03 source
rtx 5060 ti 16 GB 224 GB Flash-Next UD-Q6_K_XL 46 44.7 – linux 2026-10-17 source
radeon ai pro r9700 32 GB 27 GB Flash-Next IQ1_M – 44.7 63 linux 2026-09-29 source
arc pro b65 prefill 4096, 8K context, 512 input tokens 32 GB 128 GB Flash-Next IQ2_XS – 44.6 331 linux 2026-10-07 source
radeon 8060s – 128 GB Flash-Next UD-IQ4_XS 34 44.4 88 linux 2026-10-09 source
radeon 8060s – 128 GB Flash-Next UD-IQ4_XS 32K 44.4 687 linux 2026-10-09 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_XXS – 44.3 – windows 2026-10-11 source
rtx 5070 12 GB 64 GB Coder IQ3_S 125K 44 1,266 windows 2026-09-28 source
rx 9060 xt 16 GB 50 GB Flash-Next IQ3_XXS 88 43.9 – linux 2026-10-12 source
rtx 5080 16 GB 128 GB Flash-Next UD-Q4_K_XL – 43.8 – windows 2026-10-18 source
2× rtx 3060 12 GB 122 GB Flash-Next IQ3_S – 43.6 603 linux 2026-10-04 source
arc pro b65 prefill 512, 262K context, 512 input tokens 32 GB 128 GB Flash-Next IQ2_XS – 43.5 321 linux 2026-10-07 source
arc pro b65 prefill 512, 8K context, 7000 input tokens 32 GB 128 GB Flash-Next IQ2_XS – 43.5 315 linux 2026-10-07 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 32K 43.4 474 linux 2026-10-05 source
rtx 4090 laptop Context 262,144 tokens, 4,096 prompt tokens 16 GB 64 GB Flash-Next IQ3_S – 43.4 1,343 linux 2026-10-04 source
rtx 4090 laptop Context 262,144 tokens, 32,768 prompt tokens 16 GB 64 GB Flash-Next IQ3_S – 43.3 2,458 linux 2026-10-04 source
rx 6900 xt final automatic default (63 workers auto), short input (≈94 tokens) 16 GB 128 GB Coder IQ1_M – 43.1 – windows 2026-10-03 source
rtx 3090 24 GB 128 GB Flash-Next UD-IQ4_XS – 43 – windows 2026-10-12 source
rtx 5070 12 GB 64 GB Flash-Next CODER 125K 43 2,208 windows 2026-09-29 source
rtx 5080 + rtx 3090 + rtx 2080 ti K=30, with 2080 Ti helper 16 GB 62 GB Coder IQ1_M – 43 – windows 2026-09-29 source
arc pro b65 prefill 4096, 8K context, 7000 input tokens 32 GB 128 GB Flash-Next IQ2_XS – 42.9 830 linux 2026-10-07 source
rtx 3060 12 GB 64 GB Flash-Next Q2_0 4K 42.9 882 windows 2026-10-09 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 32K 42.9 1,100 linux 2026-10-05 source
radeon 8060s – 128 GB Flash-Next UD-IQ4_XS 4K 42.8 627 linux 2026-10-09 source
rx 6800 local build with #380, auto 16 GB 64 GB Flash-Next IQ3_XXS – 42.8 317 windows 2026-10-02 source
rtx 5070 12 GB 64 GB Coder IQ3_S 256K 42.8 1,034 windows 2026-09-28 source
radeon 8060s – 123 GB Flash-Next UD-IQ4_XS 46 42.7 – linux 2026-10-12 source
rtx 3060 12 GB 64 GB Flash-Next IQ2_XS 128 42.6 171 windows 2026-10-09 source
radeon 8060s Decoded at 40‑45 tok/s (median 42.5) with 200K context after patch; 16 GB carve‑out used as VRAM. 16 GB 96 GB GSQ-RCO IQ3_S – 42.5 – windows 2026-10-17 source
2× rtx 2080 ti 22 GB 62 GB Flash-Next IQ3_XXS 32K 42.5 477 linux 2026-10-06 source
2× rtx 3060 12 GB 122 GB Flash-Next IQ3_S – 42.5 1,156 linux 2026-10-04 source
arc pro b65 prefill 512, 262K context, 7000 input tokens 32 GB 128 GB Flash-Next IQ2_XS – 42.4 288 linux 2026-10-07 source
rx 9070 Fresh 4K prompt (4019 tokens). 16 GB 32 GB Flash-Next Q2_0 – 42.3 188 windows 2026-10-17 source
radeon 8060s – 128 GB Flash-Next UD-IQ4_XS 125K 42.1 617 linux 2026-10-09 source
radeon 8060s 112 GB 128 GB Flash-Next UD-IQ4_XS 128K 42 1,031 linux 2026-10-09 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 4K 42 868 linux 2026-10-05 source
rx 6800 zip 0.1.34, --expert-cache 3072 16 GB 64 GB Flash-Next IQ3_XXS – 42 310 windows 2026-10-02 source
rtx 3060 linux, --spec 4, --mtp rt 12 GB 46 GB Flash-Next IQ3_XXS 4K 41.9 910 linux 2026-10-06 source
rtx 3060 12 GB 64 GB Flash-Next Q2_0 32K 41.8 981 windows 2026-10-09 source
rtx 3060 linux, --spec 4, --mtp rt 12 GB 46 GB Flash-Next IQ3_XXS 32K 41.8 1,050 linux 2026-10-06 source
radeon 8060s 6 pool workers with STRATA_VERIFY_PROFILE=1 – 96 GB Flash-Next IQ3_XXS – 41.7 700 windows 2026-10-16 source
rtx 3060 linux, --spec 4, --mtp rt 12 GB 46 GB Flash-Next IQ3_XXS 4K 41.6 916 linux 2026-10-06 source
arc pro b60 24 GB 96 GB Flash-Next IQ2_XS – 41.4 – linux 2026-10-14 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 125K 41.4 459 linux 2026-10-05 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_S 125K 41.3 1,245 windows 2026-09-29 source
arc pro b65 32 GB 123 GB Flash-Next IQ2_XS 512 41.3 353 linux 2026-10-05 source
arc pro b65 32 GB 123 GB Flash-Next IQ2_XS 7K 40.9 308 linux 2026-10-05 source
rtx 3060 12 GB 64 GB Flash-Next IQ2_XS 32K 40.8 928 windows 2026-10-09 source
rx 6800 local build with #380, --expert-cache 3072 16 GB 64 GB Flash-Next IQ3_XXS – 40.5 319 windows 2026-10-02 source
rtx 4090 laptop Context 262,144 tokens, 250,000 prompt tokens 16 GB 64 GB Flash-Next IQ3_S – 40.3 2,116 linux 2026-10-04 source
rtx a3000 12 GB – Swift 1.5 IQ3_XXS – 40.3 – – 2026-10-03 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 4K 40.2 457 linux 2026-10-05 source
5× p100 12 GB 52 GB Flash-Next IQ3_S 16K 40.1 467 linux 2026-10-09 source
2× arc pro b60 24 GB 64 GB Coder IQ1_M 2K 40.1 428 linux 2026-10-04 source
rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S – 40.1 – windows 2026-10-01 source
2× rx 6900 xt baseline (pre‑fix) with default PCIe share (pcie_frac 0.39) 16 GB 128 GB Flash-Next IQ3_S – 40 – linux 2026-10-04 source
2× rtx 3060 12 GB 122 GB Flash-Next IQ3_S – 40 1,323 linux 2026-10-04 source
arc pro b60 24 GB 96 GB Coder IQ1_M – 39.7 – linux 2026-10-14 source
rtx 4090 laptop Context 262,144 tokens, 128,000 prompt tokens 16 GB 64 GB Flash-Next IQ3_S – 39.7 2,292 linux 2026-10-04 source
rx 9070 Fresh 16K prompt (16058 tokens). 16 GB 32 GB Flash-Next Q2_0 – 39.6 220 windows 2026-10-17 source
radeon 8060s – 128 GB Flash-Next UD-IQ4_XS 4K 39.6 677 linux 2026-10-09 source
2× rx 6900 xt 16 GB 128 GB Flash-Next IQ3_S 125K 39.6 1,023 linux 2026-10-05 source
rtx 2080 ti 11 GB 128 GB Flash-Next IQ3_S – 39.6 497 linux 2026-10-02 source
rx 9060 xt 16 GB 50 GB Flash-Next IQ3_S 46 39.2 – linux 2026-10-12 source
5× p100 12 GB 52 GB Flash-Next IQ3_S 4K 38.9 257 linux 2026-10-09 source
rtx 3060 linux, --spec 4, --mtp rt 12 GB 46 GB Flash-Next IQ3_XXS 32K 38.8 1,049 linux 2026-10-06 source
2× arc pro b60 24 GB 64 GB Flash-Next IQ2_XS 2K 38.8 436 linux 2026-10-04 source
2× p100 Speed measured with 192K context window; median of reported 27‑50 tokens/s. 16 GB 48 GB Flash-Next IQ3_XSS – 38.5 – linux 2026-10-16 source
p100 + rtx 2070 super 16 GB 32 GB Flash-Next Q2_0 – 38.4 295 linux 2026-10-06 source
rtx 5060 ti --mmap-experts + --kv-resident 20480 + STRATA_IQ_PREFETCH=16384, warm run (median of 37.5–39.0) 16 GB 64 GB Flash-Next IQ4_XS – 38.3 – windows 2026-10-01 source
radeon 8060s low-ram mmap, --pool-workers 6, STRATA_VERIFY_PROFILE=1, --mmap-experts 64 GB 128 GB Flash-Next IQ3_XXS – 38.2 548 windows 2026-10-18 source
rtx 5070 ti short chat (up to 100 new tokens), prompt length median 46 tokens 16 GB 64 GB Flash-Next IQ3_S – 38.1 – windows 2026-10-16 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_S – 38.1 – windows 2026-10-16 source
radeon 8060s default 15 pool workers, with STRATA_VERIFY_PROFILE=1 – 96 GB Flash-Next IQ3_XXS – 38.1 700 windows 2026-10-16 source
p40 24 GB 98 GB Flash-Next IQ3_S 565 38.1 256 linux 2026-10-12 source
rtx 2080 ti 11 GB 128 GB Flash-Next IQ3_S – 38 637 linux 2026-10-02 source
2× p40 22 GB 153 GB Flash-Next IQ2_XS 4K 37.8 353 linux 2026-10-05 source
radeon 8060s low-ram, --pool-workers 6, STRATA_VERIFY_PROFILE=1, --mmap-experts 64 GB 128 GB Flash-Next IQ3_XXS – 37.7 549 windows 2026-10-18 source
radeon 9600 xt short chat (up to 100 new tokens), prompt length 46 tokens 16 GB 30 GB Flash-Next IQ1_M – 37.6 – linux 2026-10-13 source
5× p100 12 GB 52 GB Flash-Next IQ3_S 511 37.2 105 linux 2026-10-09 source
2× p40 22 GB 153 GB Flash-Next IQ2_XS 16K 37.2 501 linux 2026-10-05 source
rtx 2080 ti 11 GB 64 GB Flash-Next IQ3_XXS – 36.9 295 windows 2026-10-03 source
rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S – 36.7 – windows 2026-10-01 source
radeon 8060s 6 pool workers, no STRATA_VERIFY_PROFILE – 96 GB Flash-Next IQ3_XXS – 36.6 700 windows 2026-10-16 source
5× p100 12 GB 52 GB Flash-Next IQ3_S 4K 36.5 255 linux 2026-10-09 source
radeon 8060s – 128 GB Unsloth UD-IQ4_XS – 36.4 – linux 2026-10-05 source
radeon 8060s 4 pool workers, no STRATA_VERIFY_PROFILE – 96 GB Flash-Next IQ3_XXS – 36.4 700 windows 2026-10-16 source
p40 24 GB 98 GB Flash-Next IQ3_S 7K 36 443 linux 2026-10-12 source
mi50 32 GB 64 GB Flash-Next IQ2_XS – 36 332 linux 2026-10-04 source
rtx 4060 + rtx 2060 8 GB 62 GB Flash-Next IQ2_XS – 35.8 – linux 2026-10-13 source
5× p100 12 GB 52 GB Flash-Next IQ3_S 4K 35.8 234 linux 2026-10-09 source
p100 + rtx 2070 super 16 GB 32 GB Flash-Next Q2_0 – 35.8 296 linux 2026-10-06 source
radeon 8060s --mmap-experts 64 GB 128 GB Flash-Next IQ3_XXS – 35.5 593 windows 2026-10-18 source
radeon 8060s --pool-workers 6, STRATA_VERIFY_PROFILE=1 64 GB 128 GB Flash-Next IQ3_XXS – 35.5 567 windows 2026-10-18 source
radeon 8060s --pool-workers 6, STRATA_VERIFY_PROFILE=1, host-core last 64 GB 128 GB Flash-Next IQ3_XXS – 35.4 553 windows 2026-10-18 source
arc pro b60 24 GB 96 GB Flash-Next IQ3_S – 35.4 – linux 2026-10-14 source
2× p40 short system prompt (~15‑27 tokens) 24 GB 125 GB Flash-Next IQ3_S – 35 – linux 2026-10-10 source
rtx 4060 ti 16 GB 96 GB Flash-Next IQ3_S – 35 – linux 2026-10-05 source
rtx 2080 ti 11 GB 64 GB Flash-Next IQ3_XXS – 34.9 525 windows 2026-10-03 source
2× p40 22 GB 153 GB Flash-Next IQ2_XS 34 34.8 55 linux 2026-10-05 source
rtx 3060 short chat decode speed 12 GB 64 GB Flash-Next IQ3_XXS – 34.7 – windows 2026-10-13 source
rx 7900 xtx 24 GB 64 GB Unsloth UD-IQ4_XS 58 34.7 45 windows 2026-10-05 source
5× p100 12 GB 52 GB Flash-Next IQ3_S 511 34.1 104 linux 2026-10-09 source
rtx 2080 ti 11 GB 128 GB Flash-Next IQ3_S – 34.1 566 linux 2026-10-02 source
mi50 32 GB 64 GB Flash-Next IQ2_XS – 34 327 linux 2026-10-04 source
rtx 5060 ti Pinned arena (no --mmap-experts), warm run 16 GB 64 GB Flash-Next IQ4_XS – 33.9 – windows 2026-10-01 source
rtx 2080 ti 11 GB 64 GB Flash-Next IQ3_XXS – 33.8 445 windows 2026-10-03 source
rtx 2080 ti 11 GB 64 GB Flash-Next IQ3_XXS – 33.8 491 windows 2026-10-03 source
rx 7900 xtx 24 GB 64 GB Unsloth UD-IQ4_XS 82 33.7 39 windows 2026-10-05 source
quadro p6000 24 GB 94 GB Flash-Next Q2_0 – 33.6 – linux 2026-10-13 source
4× p100 short reasoning prompt (~300 tokens) 16 GB 128 GB Flash-Next IQ3_XXS – 33.5 94 linux 2026-10-06 source
mi50 32 GB 64 GB Flash-Next IQ2_XS – 33.5 305 linux 2026-10-04 source
v100 v0.1.38 source build, dual Xeon E5-2696 v3, 128 GB DDR3L-1600 ECC REG, PCIe Gen3 x16, Windows 10 Enterprise LTSC (19044) 32 GB 128 GB Flash-Next IQ3_XXS – 33.5 863 windows 2026-10-03 source
rtx 3050 + quadro p6000 short chat (≈46 prompt tokens, 100 generated tokens) – 94 GB Flash-Next Q2_0 – 33.4 – linux 2026-10-13 source
radeon 8060s --pool-workers 6 64 GB 128 GB Flash-Next UD-IQ4_XS – 33.3 324 windows 2026-10-18 source
5× p100 12 GB 52 GB Flash-Next IQ3_S 16K 33.2 467 linux 2026-10-09 source
radeon 8060s low-ram mmap 64 GB 128 GB Flash-Next IQ3_XXS – 33.1 545 windows 2026-10-18 source
rx 6700 xt short chat (up to 100 generated tokens), prompt length 46 tokens 12 GB 252 GB Flash-Next IQ3_XXS – 33.1 – linux 2026-10-11 source
radeon 8060s low-ram resident 64 GB 128 GB Flash-Next IQ3_XXS – 32.9 549 windows 2026-10-18 source
p100 linux, --spec 4, --mtp rt 16 GB 62 GB Flash-Next IQ3_XXS 32K 32.8 323 linux 2026-10-06 source
5× p100 12 GB 52 GB Flash-Next IQ3_S 16K 32.7 401 linux 2026-10-09 source
quadro p6000 24 GB 94 GB Flash-Next Q2_0 – 32.6 – linux 2026-10-13 source
4× p100 short code prompt (~240 tokens) 16 GB 128 GB Flash-Next IQ3_XXS – 32.6 86 linux 2026-10-06 source
radeon 8060s --pool-workers 6, STRATA_VERIFY_PROFILE=1 64 GB 128 GB Flash-Next UD-IQ4_XS – 32.4 349 windows 2026-10-18 source
rx 6800 local build with #380, auto, STRATA_WDDM_BUDGET=0 16 GB 64 GB Flash-Next IQ3_XXS – 32.4 298 windows 2026-10-02 source
rx 6700 xt 12 GB 64 GB Flash-Next IQ2_XS – 32.1 302 linux 2026-10-05 source
radeon 8060s resident-budget-gib 36 64 GB 128 GB Flash-Next UD-IQ4_XS – 32 318 windows 2026-10-18 source
radeon 8060s default 15 pool workers, no STRATA_VERIFY_PROFILE – 96 GB Flash-Next IQ3_XXS – 32 700 windows 2026-10-16 source
v100 Prompt length 7999 tokens, STRATA_SM70_TABLE=1 (opt‑in kernels). 32 GB 96 GB Flash-Next UD-IQ4_XS – 31.9 1,626 windows 2026-10-08 source
quadro p6000 24 GB 94 GB Flash-Next IQ2_XS – 31.8 – linux 2026-10-13 source
radeon 8060s – 128 GB Unsloth UD-IQ4_XS 320 31.8 144 linux 2026-10-05 source
rtx 2080 ti 11 GB 128 GB Flash-Next IQ3_S – 31.8 467 linux 2026-10-02 source
radeon 8060s strix-prefill-switches 64 GB 128 GB Flash-Next IQ1_M – 31.6 842 windows 2026-10-18 source
5× p100 12 GB 52 GB Flash-Next IQ3_S 511 31.4 93 linux 2026-10-09 source
rx 6800 zip 0.1.35, --expert-cache auto 16 GB 64 GB Flash-Next IQ3_XXS – 31.4 285 windows 2026-10-02 source
2× p100 16 GB 126 GB Flash-Next IQ3_S – 31.2 531 linux 2026-10-06 source
p100 linux, --spec 4, --mtp rt 16 GB 62 GB Flash-Next IQ3_XXS 4K 31.1 284 linux 2026-10-06 source
rx 6700 xt 12 GB 64 GB Flash-Next IQ2_XS – 30.7 309 linux 2026-10-05 source
p100 16 GB 32 GB Flash-Next Q2_0 – 30.6 287 linux 2026-10-06 source
p100 16 GB 32 GB Flash-Next Q2_0 – 30.5 284 linux 2026-10-06 source
rtx 4060 ti 16 GB 96 GB Flash-Next IQ3_S – 30.5 – linux 2026-10-05 source
p100 linux, --spec 4, --mtp rt 16 GB 62 GB Flash-Next IQ3_XXS 4K 30.3 285 linux 2026-10-06 source
rx 6700 xt 12 GB 64 GB Flash-Next IQ2_XS – 30.2 298 linux 2026-10-05 source
rtx 3060 short chat decode speed 12 GB 64 GB Flash-Next IQ3_S – 29.2 – windows 2026-10-13 source
radeon 8060s default 64 GB 128 GB Flash-Next IQ1_M – 29.1 654 windows 2026-10-18 source
radeon 8060s Decoded at ~29 tok/s with 200K context; 32 GB carve‑out used as VRAM. 32 GB 96 GB GSQ-RCO IQ3_S – 29 – windows 2026-10-17 source
4× p100 LatticeRun docs prompt (~4.4K tokens) – prompt_tokens: 4441 16 GB 128 GB Flash-Next IQ3_XXS – 29 239 linux 2026-10-06 source
p100 linux, --spec 4, --mtp rt 16 GB 62 GB Flash-Next IQ3_XXS 32K 28.7 323 linux 2026-10-06 source
v100 Prompt length 7999 tokens (engine‑level paired runs). 32 GB 96 GB Flash-Next UD-IQ4_XS – 28.4 1,628 windows 2026-10-08 source
arc a770 16 GB – Flash-Next ? 30K 28.1 209 – 2026-10-14 source
rtx 5090 laptop Median decode speed for Chinese prose prompt (default settings, no experimental flags). 24 GB 188 GB Flash-Next UD-IQ4_XS – 28.1 – linux 2026-10-07 source
4× p100 LatticeRun docs prompt (~36.4K tokens) – prompt_tokens unknown 16 GB 128 GB Flash-Next IQ3_XXS – 28 594 linux 2026-10-06 source
quadro p6000 24 GB 94 GB Flash-Next IQ3_XXS 46 27.9 – linux 2026-10-14 source
rtx 5060 ti GGUF in place + --resident-budget-gib 40 + kv-resident/prefetch knobs, warm run 16 GB 64 GB Flash-Next IQ4_XS – 27.5 – windows 2026-10-01 source
rtx 3060 12 GB 64 GB Flash-Next IQ3_XXS 32K 27.1 884 windows 2026-10-09 source
apple m1 max – 64 GB Flash-Next Q2_0 – 26.7 – macos 2026-10-17 source
rx 6800m 12 GB 31 GB Flash-Next Q2_0 556 26.5 65 windows 2026-10-06 source
arc a770 16 GB – Flash-Next ? 2K 26.4 181 – 2026-10-14 source
v100 32 GB 96 GB Flash-Next UD-IQ4_XS – 26.3 665 windows 2026-10-05 source
rtx 3060 12 GB 64 GB Flash-Next IQ3_XXS 4K 26.2 681 windows 2026-10-09 source
rtx 5060 ti 16 GB 96 GB Flash-Next NVFP4 – 26.1 – windows 2026-10-17 source
rtx 3060 12 GB 64 GB Flash-Next IQ3_XXS 128 26 79 windows 2026-10-09 source
rtx 5060 ti GGUF in place + --resident-budget-gib 40, warm run 16 GB 64 GB Flash-Next IQ4_XS – 26 – windows 2026-10-01 source
rtx 5060 ti experts.bin + --resident-budget-gib 40, warm run 16 GB 64 GB Flash-Next IQ4_XS – 25.4 – windows 2026-10-01 source
radeon pro vii 16 GB 126 GB Flash-Next IQ2_XS – 23.9 349 linux 2026-10-09 source
radeon pro vii 16 GB 126 GB Flash-Next IQ2_XS – 23.9 322 linux 2026-10-09 source
p40 22 GB 153 GB Flash-Next IQ2_XS 16K 23.7 388 linux 2026-10-05 source
radeon pro vii 16 GB 126 GB Flash-Next IQ2_XS – 23.5 280 linux 2026-10-09 source
rtx 5090 32 GB 64 GB Flash-Next IQ3_S – 23 – linux 2026-10-13 source
quadro p6000 24 GB 94 GB Swift 1.5 IQ3_XXS – 22.8 – linux 2026-10-13 source
2× quadro rtx 4000 8 GB 64 GB Flash-Next IQ2_XS 72K 22.8 126 linux 2026-10-07 source
rtx 3090 long prompt 47k tokens 24 GB 62 GB Flash-Next IQ3_XXS – 22.6 1,628 linux 2026-10-15 source
2× quadro rtx 4000 8 GB 64 GB Flash-Next IQ2_XS 71K 22.6 139 linux 2026-10-07 source
4× t4 2,048 context, 1.7K prompt (prefill) measured on LatticeRun v0.1.45 16 GB 251 GB Flash-Next UD-Q6_K_XL – 21.7 173 linux 2026-10-18 source
rtx 5090 laptop Decode speed for English code prompt (default settings, single measurement). 24 GB 188 GB Flash-Next UD-IQ4_XS – 21.3 – linux 2026-10-07 source
2× quadro rtx 4000 8 GB 64 GB Flash-Next IQ2_XS 72K 21.2 249 linux 2026-10-07 source
rtx 3090 24 GB 62 GB Flash-Next UD-IQ4_XS – 20.9 387 linux 2026-10-15 source
rtx 3060 12 GB 64 GB Flash-Next IQ3_S 4K 20.2 569 windows 2026-10-09 source
rtx 3060 12 GB 64 GB Flash-Next IQ3_S 32K 20.2 820 windows 2026-10-09 source
4× t4 128K max-context, 1.7K prompt (prefill) measured on LatticeRun v0.1.45 16 GB 251 GB Flash-Next UD-Q6_K_XL – 19.9 165 linux 2026-10-18 source
p40 22 GB 153 GB Flash-Next IQ2_XS 34 19.6 53 linux 2026-10-05 source
2× a2 16 GB 62 GB Flash-Next UD-IQ4_XS 63K 19.3 742 linux 2026-10-14 source
rx 6800m 12 GB 31 GB Flash-Next Q2_0 556 19.3 50 windows 2026-10-06 source
rtx 3060 12 GB 64 GB Flash-Next IQ3_S 128 19.1 50 windows 2026-10-09 source
radeon 890m 28 K token prompt, default settings 47 GB 96 GB Flash-Next IQ3_XXS – 18.7 129 linux 2026-10-06 source
4× t4 93,111-token prompt (128K max-context) measured on LatticeRun v0.1.45 16 GB 251 GB Flash-Next UD-Q6_K_XL – 18.6 221 linux 2026-10-18 source
2× a2 16 GB 62 GB Flash-Next UD-IQ4_XS 121K 18.6 767 linux 2026-10-14 source
2× quadro rtx 4000 8 GB 64 GB Flash-Next IQ2_XS 73K 18.1 100 linux 2026-10-07 source
rx 6800m 12 GB 64 GB Flash-Next IQ2_XS 57 18 22 windows 2026-10-05 source
a10 short chat (up to 100 generated tokens), prompt length 46 tokens 22 GB 455 GB Flash-Next IQ3_S – 17.1 – linux 2026-10-12 source
radeon 890m 3.6 K token prompt, default settings 47 GB 96 GB Flash-Next IQ3_XXS – 17 126 linux 2026-10-06 source
2× a2 16 GB 62 GB Flash-Next UD-IQ4_XS 245K 16.9 742 linux 2026-10-14 source
radeon 890m 7 K token prompt, default settings 47 GB 96 GB Flash-Next IQ3_XXS – 16.9 135 linux 2026-10-06 source
rtx 5060 ti --mmap-experts over experts.bin, warm run 16 GB 64 GB Flash-Next IQ4_XS – 16.9 – windows 2026-10-01 source
radeon 890m short chat (51‑token prompt, 180‑token output), first token latency 0.25 s → 16.7‑16.9 tok/s 47 GB 96 GB Flash-Next IQ3_XXS – 16.8 – linux 2026-10-06 source
v100 Long multi‑method math reasoning – 1,761 output tokens 32 GB 128 GB Flash-Next IQ3_XXS – 16.8 – windows 2026-10-03 source
rx 6800m 12 GB 64 GB Flash-Next IQ2_XS 30 16.7 21 windows 2026-10-05 source
v100 31k‑token interactive session – 31,796 output tokens, steady speed 32 GB 128 GB Flash-Next IQ3_XXS – 16.7 – windows 2026-10-03 source
radeon 780m – 60 GB Flash-Next Q2_0 4K 16.4 277 linux 2026-10-12 source
radeon 780m – 60 GB Flash-Next Q2_0 16K 16.2 315 linux 2026-10-12 source
p40 22 GB 153 GB Flash-Next IQ2_XS 4K 16.1 348 linux 2026-10-05 source
radeon 780m – 60 GB Flash-Next Q2_0 8K 15.9 314 linux 2026-10-12 source
v100 Long‑form writing – 1,492 output tokens 32 GB 128 GB Flash-Next IQ3_XXS – 15.8 – windows 2026-10-03 source
radeon 780m – 60 GB Flash-Next Q2_0 31K 15.5 309 linux 2026-10-12 source
radeon 890m table+switches configuration, short chat decode (256‑token output) 64 GB 96 GB Flash-Next IQ3_XXS – 15.3 – linux 2026-10-07 source
2× a2 16 GB 62 GB Flash-Next UD-IQ4_XS 486K 15.1 591 linux 2026-10-14 source
radeon 890m 80 GB 93 GB Flash-Next UD-IQ4_XS – 15 – linux 2026-10-12 source
apple m1 max short chat (decode only), prompt length 46 tokens – 32 GB Flash-Next Q2_0 – 14.7 – macos 2026-10-15 source
rx 6800m 12 GB 31 GB Flash-Next Q2_0 9K 14.7 92 windows 2026-10-06 source
rtx pro 4000 short chat (≈100 generated tokens, 46 prompt tokens) 24 GB 221 GB Flash-Next Q8_0 – 14.6 – linux 2026-10-16 source
rx 6800m 12 GB 31 GB Flash-Next Q2_0 2K 14.5 63 windows 2026-10-06 source
v100 ~400‑word essay – 322 output tokens 32 GB 128 GB Flash-Next IQ3_XXS – 14.2 – windows 2026-10-03 source
rx 6800m 12 GB 31 GB Flash-Next Q2_0 9K 14.1 114 windows 2026-10-06 source
v100 Medium answer – 289 output tokens 32 GB 128 GB Flash-Next IQ3_XXS – 13.7 – windows 2026-10-03 source
radeon 780m – 60 GB Flash-Next Q2_0 4K 13.1 90 linux 2026-10-12 source
radeon 780m – 60 GB Flash-Next Q2_0 32K 13.1 90 linux 2026-10-12 source
rx 6900 xt baseline 63 workers, 4K input (≈4,022 tokens) 16 GB 128 GB Coder IQ1_M – 13.1 – windows 2026-10-03 source
rx 6900 xt baseline 63 workers, short input (≈94 tokens) 16 GB 128 GB Coder IQ1_M – 12.7 – windows 2026-10-03 source
rx 6800m 12 GB 64 GB Flash-Next IQ2_XS 25 12.5 17 windows 2026-10-05 source
rx 6900 xt 16 GB 128 GB Coder IQ1_M 94 12.5 42 windows 2026-10-03 source
rtx 3090 long prompt 47k tokens 24 GB 62 GB Flash-Next UD-IQ4_XS – 12.2 784 linux 2026-10-15 source
radeon 780m – 60 GB Flash-Next Q2_0 63K 12.1 258 linux 2026-10-12 source
2× a2 16 GB 62 GB Flash-Next UD-IQ4_XS 1017K 12 398 linux 2026-10-14 source
arc pro b60 24 GB 64 GB Coder IQ1_M – 11.9 – linux 2026-10-04 source
rx 6900 xt 16 GB 128 GB Coder IQ1_M 4K 11.6 313 windows 2026-10-03 source
v100 Short answer – 39 output tokens 32 GB 128 GB Flash-Next IQ3_XXS – 10.1 – windows 2026-10-03 source
radeon 780m – 60 GB Flash-Next Q2_0 94K 9.9 216 linux 2026-10-12 source
radeon 780m – 60 GB Flash-Next Q2_0 125K 9.8 199 linux 2026-10-12 source
rtx 5060 ti GGUF read in place (no experts.bin), warm run 16 GB 64 GB Flash-Next IQ4_XS – 9.7 – windows 2026-10-01 source
rx 6800m 12 GB 31 GB Flash-Next Q2_0 2K 9.1 60 windows 2026-10-06 source
v100 First request ever (JIT warm‑up) – 175 output tokens 32 GB 128 GB Flash-Next IQ3_XXS – 7.5 – windows 2026-10-03 source
2× a2 16 GB 62 GB Flash-Next Q8_0 63K 6.2 192 linux 2026-10-14 source
rx 6900 xt 16 GB 128 GB Coder IQ1_M 32K 6.1 350 windows 2026-10-03 source
2× a2 16 GB 62 GB Flash-Next Q8_0 121K 5.9 183 linux 2026-10-14 source
2× a2 16 GB 62 GB Flash-Next Q8_0 245K 4.9 189 linux 2026-10-14 source
2× a2 16 GB 62 GB Flash-Next Q8_0 486K 4.6 178 linux 2026-10-14 source
2× a2 16 GB 62 GB Flash-Next Q8_0 1017K 3.9 166 linux 2026-10-14 source
rtx 5080 16 GB 128 GB Flash-Next UD-Q4_K_XL – – 1,359 windows 2026-10-18 source
rtx 5080 16 GB 128 GB Flash-Next UD-Q4_K_XL – – 2,428 windows 2026-10-18 source
3× rtx 5060 ti 4K fresh prompt test, prompt length 4019 tokens 16 GB 224 GB Flash-Next UD-Q6_K_XL – – 540 linux 2026-10-17 source
3× rtx 5060 ti 16K fresh prompt test, prompt length 16058 tokens 16 GB 224 GB Flash-Next UD-Q6_K_XL – – 1,357 linux 2026-10-17 source
2× rtx 5060 ti 16 GB 224 GB Flash-Next UD-Q6_K_XL 4K – 512 linux 2026-10-17 source
2× rtx 5060 ti 16 GB 224 GB Flash-Next UD-Q6_K_XL 16K – 1,185 linux 2026-10-17 source
rtx 5060 ti 16 GB 224 GB Flash-Next UD-Q6_K_XL 4K – 498 linux 2026-10-17 source
rtx 5060 ti 16 GB 224 GB Flash-Next UD-Q6_K_XL 16K – 937 linux 2026-10-17 source
rx 9070 xt 16 GB 64 GB Flash-Next IQ2_XS 4K – 810 windows 2026-10-17 source
rx 9070 xt 16 GB 64 GB Flash-Next IQ2_XS 16K – 958 windows 2026-10-17 source
2× rtx 3090 24 GB 128 GB Flash-Next UD-Q4_K_XL 4K – 833 windows 2026-10-17 source
2× rtx 3090 24 GB 128 GB Flash-Next UD-Q4_K_XL 16K – 1,567 windows 2026-10-17 source
2× rtx 3090 4K fresh prompt (4019 prompt tokens, 24 generated tokens) 24 GB 128 GB Flash-Next UD-Q4_K_XL – – 818 windows 2026-10-17 source
2× rtx 3090 16K fresh prompt (16058 prompt tokens, 24 generated tokens) 24 GB 128 GB Flash-Next UD-Q4_K_XL – – 1,532 windows 2026-10-17 source
2× v100 fresh 4K prompt test, prompt length 4019 tokens 16 GB 123 GB Flash-Next IQ3_S – – 821 linux 2026-10-17 source
2× v100 fresh 16K prompt test, prompt length 16058 tokens 16 GB 123 GB Flash-Next IQ3_S – – 1,579 linux 2026-10-17 source
apple m1 max – 64 GB Flash-Next Q2_0 – – 226 macos 2026-10-17 source
apple m1 max – 64 GB Flash-Next Q2_0 – – 234 macos 2026-10-17 source
rx 7900 xtx 24 GB 64 GB Flash-Next IQ3_XXS – – 939 windows 2026-10-17 source
rx 7900 xtx 24 GB 64 GB Flash-Next IQ3_XXS – – 2,008 windows 2026-10-17 source
rx 7900 xtx 24 GB 64 GB Flash-Next IQ3_XXS – – 2,077 windows 2026-10-17 source
2× v100 Prompt evaluation (prefill) speed on short prompts (length not specified) 16 GB 64 GB Flash-Next IQ3_S – – 78 linux 2026-10-17 source
2× v100 Prefill speed for a 10 000‑token prompt (~3 s total) – ~3300 tok/s 16 GB 64 GB Flash-Next IQ3_S – – 3,300 linux 2026-10-17 source
rtx 5060 ti 16 GB 96 GB Flash-Next NVFP4 – – 363 windows 2026-10-17 source
rtx 5060 ti 16 GB 96 GB Flash-Next NVFP4 – – 627 windows 2026-10-17 source
rtx 5060 ti 16 GB 96 GB Flash-Next NVFP4 – – 635 windows 2026-10-17 source
rtx pro 4000 4K fresh prompt (4019 prompt tokens, 24 generated tokens) 24 GB 221 GB Flash-Next Q8_0 – – 436 linux 2026-10-16 source
rtx pro 4000 16K fresh prompt (16058 prompt tokens, 24 generated tokens) 24 GB 221 GB Flash-Next Q8_0 – – 849 linux 2026-10-16 source
rtx 5070 ti 4K fresh prompt, prompt length 4019 tokens 16 GB 64 GB Flash-Next IQ3_S – – 932 windows 2026-10-16 source
rtx 5070 ti 16K fresh prompt, prompt length 16058 tokens 16 GB 64 GB Flash-Next IQ3_S – – 1,200 windows 2026-10-16 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_S – – 932 windows 2026-10-16 source
rtx 5070 ti 16 GB 64 GB Flash-Next IQ3_S – – 1,200 windows 2026-10-16 source
rtx 4090 48 GB 61 GB Flash-Next UD-Q4_K_XL 4K – 1,104 linux 2026-10-15 source
rtx 4090 48 GB 61 GB Flash-Next UD-Q4_K_XL 16K – 2,062 linux 2026-10-15 source
apple m1 max fresh 4K prompt, prompt length 4019 tokens – 32 GB Flash-Next Q2_0 – – 289 macos 2026-10-15 source
apple m1 max fresh 16K prompt, prompt length 16058 tokens – 32 GB Flash-Next Q2_0 – – 287 macos 2026-10-15 source
2× rtx 3060 fresh 4K prompt test, prompt length 4019 tokens 12 GB 63 GB Flash-Next IQ3_S – – 676 linux 2026-10-15 source
2× rtx 3060 fresh 16K prompt test, prompt length 16058 tokens 12 GB 63 GB Flash-Next IQ3_S – – 947 linux 2026-10-15 source
rtx 3080 ti 12 GB 128 GB Flash-Next IQ3_S – – 1,864 windows 2026-10-14 source
rtx 3080 ti 12 GB 128 GB Flash-Next IQ3_S – – 2,003 windows 2026-10-14 source
rtx a6000 48 GB 125 GB Flash-Next IQ3_S – – 2,184 linux 2026-10-14 source
rtx a6000 48 GB 125 GB Flash-Next IQ3_S – – 2,800 linux 2026-10-14 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_S – – 1,149 windows 2026-10-14 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_S – – 2,091 windows 2026-10-14 source
rtx 3090 24 GB 64 GB Flash-Next IQ3_S – – 2,141 windows 2026-10-14 source
quadro p6000 24 GB 94 GB Flash-Next IQ3_XXS 4K – 321 linux 2026-10-14 source
quadro p6000 24 GB 94 GB Flash-Next IQ3_XXS 16K – 310 linux 2026-10-14 source
rtx 4080 super 16 GB 63 GB Flash-Next IQ2_XS 4K – 2,405 linux 2026-10-14 source
rtx 4080 super 16 GB 63 GB Flash-Next IQ2_XS 16K – 3,396 linux 2026-10-14 source
rtx 3080 ti 4K fresh prompt (4019 prompt tokens, 24 generated tokens) 12 GB 64 GB Flash-Next IQ3_S – – 985 windows 2026-10-14 source
rtx 3080 ti 16K fresh prompt (16058 prompt tokens, 24 generated tokens) 12 GB 64 GB Flash-Next IQ3_S – – 1,223 windows 2026-10-14 source
arc pro b60 24 GB 96 GB Flash-Next IQ2_XS – – 922 linux 2026-10-14 source
arc pro b60 24 GB 96 GB Flash-Next IQ2_XS – – 1,051 linux 2026-10-14 source
arc pro b60 24 GB 96 GB Coder IQ1_M – – 947 linux 2026-10-14 source
arc pro b60 24 GB 96 GB Coder IQ1_M – – 1,063 linux 2026-10-14 source
arc pro b60 24 GB 96 GB Flash-Next IQ3_S – – 837 linux 2026-10-14 source
arc pro b60 24 GB 96 GB Flash-Next IQ3_S – – 1,034 linux 2026-10-14 source
rtx 3060 4K prompt prefill speed (4096 tokens) 12 GB 64 GB Flash-Next IQ3_S – – 806 windows 2026-10-13 source
rtx 3060 4K prompt prefill speed (4096 tokens) 12 GB 64 GB Flash-Next IQ3_XXS – – 883 windows 2026-10-13 source
rtx 3060 16K prompt prefill speed (16384 tokens) 12 GB 64 GB Flash-Next IQ3_S – – 959 windows 2026-10-13 source
rtx 3060 16K prompt prefill speed (16384 tokens) 12 GB 64 GB Flash-Next IQ3_XXS – – 1,050 windows 2026-10-13 source
rx 7900 xt 20 GB 64 GB Flash-Next IQ2_XS – – 1,220 windows 2026-10-13 source
rx 7900 xt 20 GB 64 GB Flash-Next IQ2_XS – – 1,300 windows 2026-10-13 source
rx 7900 xt 20 GB 64 GB Flash-Next IQ2_XS – – 1,249 windows 2026-10-13 source
rtx 5090 laptop 24 GB 95 GB Flash-Next UD-IQ4_XS 4K – 1,691 windows 2026-10-13 source
rtx 5090 laptop 24 GB 95 GB Flash-Next UD-IQ4_XS 16K – 2,557 windows 2026-10-13 source
rtx 4080 super 16 GB 64 GB Flash-Next IQ2_XS – – 1,406 windows 2026-10-13 source
rtx 4080 super 16 GB 64 GB Flash-Next IQ2_XS – – 2,508 windows 2026-10-13 source
quadro p6000 24 GB 94 GB Swift 1.5 IQ3_XXS – – 303 linux 2026-10-13 source
quadro p6000 24 GB 94 GB Swift 1.5 IQ3_XXS – – 298 linux 2026-10-13 source
rx 7900 xtx 4K prompt (4019 tokens) median of two runs 24 GB 128 GB Flash-Next IQ3_S – – 958 windows 2026-10-13 source
rx 7900 xtx 16K prompt (16058 tokens) median of two runs 24 GB 128 GB Flash-Next IQ3_S – – 1,825 windows 2026-10-13 source
rx 7900 xtx 32K prompt (32049 tokens) median of two runs 24 GB 128 GB Flash-Next IQ3_S – – 2,053 windows 2026-10-13 source
rtx 4090 24 GB 128 GB Flash-Next IQ1_M – – 3,517 windows 2026-10-13 source
rtx 4090 24 GB 128 GB Flash-Next IQ1_M – – 4,573 windows 2026-10-13 source
rtx 5080 + rtx 5060 ti Pass 1 – tuned split, fresh 4K prompt prefill speed (tokens/s) median 16 GB 94 GB Flash-Next UD-Q4_K_XL – – 1,884 linux 2026-10-13 source
rtx 5080 + rtx 5060 ti Pass 2 – auto split, fresh 4K prompt prefill speed (tokens/s) median 16 GB 94 GB Flash-Next UD-Q4_K_XL – – 1,880 linux 2026-10-13 source
rtx 5080 + rtx 5060 ti Pass 1 – tuned split, fresh 16K prompt prefill speed (tokens/s) median 16 GB 94 GB Flash-Next UD-Q4_K_XL – – 2,900 linux 2026-10-13 source
rtx 5080 + rtx 5060 ti Pass 2 – auto split, fresh 16K prompt prefill speed (tokens/s) median 16 GB 94 GB Flash-Next UD-Q4_K_XL – – 2,905 linux 2026-10-13 source
radeon 9600 xt 4K fresh prompt, prompt length 4019 tokens 16 GB 30 GB Flash-Next IQ1_M – – 906 linux 2026-10-13 source
radeon 9600 xt 16K fresh prompt, prompt length 16058 tokens 16 GB 30 GB Flash-Next IQ1_M – – 967 linux 2026-10-13 source
rtx 4060 + rtx 2060 8 GB 62 GB Flash-Next IQ2_XS – – 288 linux 2026-10-13 source
rtx 4060 + rtx 2060 8 GB 62 GB Flash-Next IQ2_XS – – 299 linux 2026-10-13 source
2× rtx 4070 fresh 4K prompt, actual prompt tokens 4019 24 GB 31 GB Flash-Next IQ1_M – – 1,652 linux 2026-10-13 source
2× rtx 4070 fresh 16K prompt, actual prompt tokens 16058 24 GB 31 GB Flash-Next IQ1_M – – 2,444 linux 2026-10-13 source
rtx 3050 + quadro p6000 4K fresh prompt (4019 prompt tokens, 24 generated tokens) – 94 GB Flash-Next Q2_0 – – 254 linux 2026-10-13 source
rtx 3050 + quadro p6000 16K fresh prompt (16058 prompt tokens, 24 generated tokens) – 94 GB Flash-Next Q2_0 – – 245 linux 2026-10-13 source
quadro p6000 24 GB 94 GB Flash-Next IQ2_XS – – 300 linux 2026-10-13 source
quadro p6000 24 GB 94 GB Flash-Next IQ2_XS – – 294 linux 2026-10-13 source
quadro p6000 24 GB 94 GB Flash-Next Q2_0 – – 305 linux 2026-10-13 source
quadro p6000 24 GB 94 GB Flash-Next Q2_0 – – 296 linux 2026-10-13 source
rtx 5060 ti fresh 4K prompt, 4019 prompt tokens, 24 generated tokens 16 GB 64 GB Flash-Next IQ3_S – – 1,129 windows 2026-10-13 source
rtx 5060 ti fresh 16K prompt, 16058 prompt tokens, 24 generated tokens 16 GB 64 GB Flash-Next IQ3_S – – 1,761 windows 2026-10-13 source
rtx pro 4000 24 GB 221 GB Flash-Next IQ3_S 4K – 1,233 linux 2026-10-12 source
rtx pro 4000 24 GB 221 GB Flash-Next IQ3_S 16K – 2,087 linux 2026-10-12 source
rtx pro 4000 24 GB 221 GB Flash-Next IQ3_S 31K – 1,956 linux 2026-10-12 source
rtx pro 4000 4K fresh prompt (≈4019 prompt tokens, 24 generated tokens) 24 GB 221 GB Flash-Next IQ3_S – – 1,188 linux 2026-10-12 source
rtx pro 4000 16K fresh prompt (≈16058 prompt tokens, 24 generated tokens) 24 GB 221 GB Flash-Next IQ3_S – – 2,172 linux 2026-10-12 source
rtx pro 4000 32K fresh prompt (≈32049 prompt tokens, 24 generated tokens) 24 GB 221 GB Flash-Next IQ3_S – – 2,210 linux 2026-10-12 source
rtx pro 4000 24 GB 221 GB Flash-Next IQ3_S 4K – 1,232 linux 2026-10-12 source
rtx pro 4000 24 GB 221 GB Flash-Next IQ3_S 16K – 2,089 linux 2026-10-12 source
rtx pro 4000 24 GB 221 GB Swift 1.5 IQ3_XXS 4K – 1,432 linux 2026-10-12 source
rtx pro 4000 24 GB 221 GB Swift 1.5 IQ3_XXS 16K – 2,194 linux 2026-10-12 source
rtx pro 4000 24 GB 221 GB Swift 1.5 IQ3_XXS 31K – 2,227 linux 2026-10-12 source
rtx pro 4000 24 GB 221 GB Swift 1.5 IQ3_XXS – – 1,513 linux 2026-10-12 source
rtx pro 4000 24 GB 221 GB Swift 1.5 IQ3_XXS – – 2,315 linux 2026-10-12 source
rx 9060 xt 16 GB 50 GB Flash-Next IQ3_S 4K – 616 linux 2026-10-12 source
rx 9060 xt 16 GB 50 GB Flash-Next IQ3_S 16K – 783 linux 2026-10-12 source
rx 9060 xt 16 GB 50 GB Flash-Next IQ3_XXS 4K – 632 linux 2026-10-12 source
rx 9060 xt 16 GB 50 GB Flash-Next IQ3_XXS 16K – 772 linux 2026-10-12 source
rtx 5090 32 GB 63 GB Swift 1.5 IQ3_S – – 5,096 windows 2026-10-12 source
rtx 5090 32 GB 63 GB Swift 1.5 IQ3_S – – 6,639 windows 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 fresh 4K prompt (4,019 tokens), median prefill speed – 123 GB Flash-Next UD-IQ4_XS – – 520 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 fresh 16K prompt (16,058 tokens), median prefill speed – 123 GB Flash-Next UD-IQ4_XS – – 926 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 fresh 32K prompt (32,049 tokens), median prefill speed – 123 GB Flash-Next UD-IQ4_XS – – 1,270 linux 2026-10-12 source
rx 7900 xtx 24 GB 184 GB Swift 1.5 IQ3_S – – 1,282 linux 2026-10-12 source
rx 7900 xtx 24 GB 184 GB Swift 1.5 IQ3_S – – 1,970 linux 2026-10-12 source
rx 7900 xtx 24 GB 184 GB Swift 1.5 IQ3_S – – 2,048 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 16 GB 123 GB Flash-Next IQ3_XXS – – 1,296 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 16 GB 123 GB Flash-Next IQ3_XXS – – 2,690 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 16 GB 123 GB Flash-Next IQ3_XXS – – 3,228 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 4K fresh prompt, median prefill speed 811.3 tok/s, prompt length 4019 tokens – 123 GB Flash-Next IQ3_S – – 811 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 16K fresh prompt, median prefill speed 1579.8 tok/s, prompt length 16058 tokens – 123 GB Flash-Next IQ3_S – – 1,580 linux 2026-10-12 source
rtx 5060 ti + rtx 3090 + rtx 3060 32K fresh prompt, median prefill speed 1946.0 tok/s, prompt length 32049 tokens – 123 GB Flash-Next IQ3_S – – 1,946 linux 2026-10-12 source
rtx 4080 16 GB 64 GB Flash-Next IQ2_XS 4K – 1,545 windows 2026-10-12 source
rtx 4080 16 GB 64 GB Flash-Next IQ2_XS 16K – 2,247 windows 2026-10-12 source
radeon 8060s – 123 GB Flash-Next UD-IQ4_XS 4K – 614 linux 2026-10-12 source
radeon 8060s – 123 GB Flash-Next UD-IQ4_XS 16K – 653 linux 2026-10-12 source
rtx 5090 + rtx 4000 ada – 125 GB Flash-Next IQ3_S – – 2,811 linux 2026-10-12 source
rtx 5090 + rtx 4000 ada – 125 GB Flash-Next IQ3_S – – 2,674 linux 2026-10-12 source
rtx 4080 16 GB 128 GB Flash-Next IQ3_S 4K – 1,436 windows 2026-10-12 source
rtx 4080 16 GB 128 GB Flash-Next IQ3_S 16K – 1,996 windows 2026-10-12 source
rtx 4080 16 GB 128 GB Flash-Next IQ3_S 31K – 2,012 windows 2026-10-12 source
rtx 6000 84 GB 122 GB Flash-Next UD-Q4_K_XL – – 3,745 linux 2026-10-12 source
rtx 6000 84 GB 122 GB Flash-Next UD-Q4_K_XL – – 5,346 linux 2026-10-12 source
rtx 6000 84 GB 122 GB Swift 1.5 IQ2_XS 4K – 5,036 linux 2026-10-12 source
rtx 6000 84 GB 122 GB Swift 1.5 IQ2_XS 16K – 6,294 linux 2026-10-12 source
rx 7900 xtx 4K fresh prompt (4019 prompt tokens, 24 generated tokens) 24 GB 127 GB Swift 1.5 IQ2_XS – – 747 windows 2026-10-12 source
rx 7900 xtx 16K fresh prompt (16058 prompt tokens, 24 generated tokens) 24 GB 127 GB Swift 1.5 IQ2_XS – – 843 windows 2026-10-12 source
a10 fresh 4K prompt, prompt length 4019 tokens 22 GB 455 GB Flash-Next IQ3_S – – 1,431 linux 2026-10-12 source
a10 fresh 16K prompt, prompt length 16058 tokens 22 GB 455 GB Flash-Next IQ3_S – – 1,709 linux 2026-10-12 source
rtx 3090 24 GB 128 GB Flash-Next UD-IQ4_XS – – 813 windows 2026-10-12 source
rtx 3090 24 GB 128 GB Flash-Next UD-IQ4_XS – – 1,366 windows 2026-10-12 source
rtx 3090 24 GB 128 GB Flash-Next UD-IQ4_XS – – 1,426 windows 2026-10-12 source
rtx 3090 fresh 4K prompt (4019 prompt tokens, 24 generated tokens) 24 GB 128 GB Flash-Next IQ3_S – – 1,156 windows 2026-10-12 source
rtx 3090 fresh 16K prompt (16058 prompt tokens, 24 generated tokens) 24 GB 128 GB Flash-Next IQ3_S – – 2,112 windows 2026-10-12 source
rtx 3090 fresh 32K prompt (32049 prompt tokens, 24 generated tokens) 24 GB 128 GB Flash-Next IQ3_S – – 2,120 windows 2026-10-12 source
radeon 890m 80 GB 93 GB Flash-Next UD-IQ4_XS – – 203 linux 2026-10-12 source
radeon 890m 80 GB 93 GB Flash-Next UD-IQ4_XS – – 217 linux 2026-10-12 source
2× rtx 2080 ti 22 GB 94 GB Flash-Next IQ3_S 4K – 955 linux 2026-10-12 source
2× rtx 2080 ti 22 GB 94 GB Flash-Next IQ3_S 16K – 1,375 linux 2026-10-12 source
rx 7900 xtx Median fresh prompt processing: 130,978 tokens in 59.465 s (≈2,204 t/s). Measured on a single RX 7900 XTX under Linux (N 24 GB 64 GB Flash-Next IQ3_S – – 2,204 linux 2026-10-11 source
rx 7900 xtx Median fresh prompt processing: 239,906 tokens in 110.438 s (≈2,173 t/s). Same hardware and settings as above. 24 GB 64 GB Flash-Next IQ3_S – – 2,173 linux 2026-10-11 source
rx 6700 xt 4K fresh prompt, prompt length 4019 tokens 12 GB 252 GB Flash-Next IQ3_XXS – – 350 linux 2026-10-11 source
rx 6700 xt 16K fresh prompt, prompt length 16058 tokens 12 GB 252 GB Flash-Next IQ3_XXS – – 358 linux 2026-10-11 source
rtx 5070 ti 16 GB 96 GB Flash-Next IQ3_XXS – – 1,615 windows 2026-10-11 source
rtx 5070 ti 16 GB 96 GB Flash-Next IQ3_XXS – – 2,383 windows 2026-10-11 source
rtx 5070 ti 16 GB 96 GB Flash-Next IQ3_XXS – – 2,385 windows 2026-10-11 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_XXS – – 176 windows 2026-10-11 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_XXS – – 236 windows 2026-10-11 source
rx 7900 xtx 24 GB 96 GB Flash-Next IQ3_XXS – – 244 windows 2026-10-11 source
2× p40 fresh prefill of 3,918 tokens 24 GB 125 GB Flash-Next IQ3_S – – 350 linux 2026-10-10 source
2× p40 fresh prefill of 21,858 tokens 24 GB 125 GB Flash-Next IQ3_S – – 558 linux 2026-10-10 source
rtx 5090 32 GB 120 GB Flash-Next IQ3_S – – 6,710 linux 2026-10-10 source
rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S 4K – 343 wsl2 2026-10-09 source
rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S 19K – 598 wsl2 2026-10-09 source
p4 8 GB 252 GB Flash-Next IQ3_XXS – – 78 linux 2026-10-09 source
p4 8 GB 252 GB Flash-Next IQ3_XXS – – 79 linux 2026-10-09 source
2× radeon pro vii 16 GB – Flash-Next IQ3_XXS 4K – 369 – 2026-10-08 source
2× radeon pro vii 16 GB – Flash-Next IQ3_XXS 64K – 625 – 2026-10-08 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --kv-resident off 24 GB 32 GB Swift 1.5 IQ3_S 537 – 1,000 linux 2026-10-08 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --kv-resident 32768 24 GB 32 GB Swift 1.5 IQ3_S 537 – 1,000 linux 2026-10-08 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --kv-resident off 24 GB 32 GB Swift 1.5 IQ3_S 983 – 917 linux 2026-10-08 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --kv-resident 32768 24 GB 32 GB Swift 1.5 IQ3_S 983 – 886 linux 2026-10-08 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --kv-resident off 24 GB 32 GB Swift 1.5 IQ3_S 2K – 627 linux 2026-10-08 source
2× rtx 3090 layer split, --trim-stage-weights, CPU share on, --kv-resident 32768 24 GB 32 GB Swift 1.5 IQ3_S 2K – 606 linux 2026-10-08 source
v100 prefill speed for a 6,345‑token prompt (median of 1,360‑1,390 tok/s reported) 32 GB 180 GB Flash-Next IQ2_XS – – 1,375 linux 2026-10-07 source
rx 7900 xtx + rx 6800 xt 24 GB 121 GB Flash-Next UD-Q4_K_XL – – 1,350 linux 2026-10-07 source
rx 7900 xtx + rx 6800 xt 24 GB 121 GB Flash-Next UD-Q4_K_XL – – 1,294 linux 2026-10-07 source
rx 7900 xtx + rx 6800 xt 24 GB 121 GB Flash-Next UD-Q4_K_XL – – 592 linux 2026-10-07 source
radeon 890m table+switches configuration, 987‑token prompt 64 GB 96 GB Flash-Next IQ3_XXS – – 153 linux 2026-10-07 source
radeon 890m table+switches configuration, 3,562‑token prompt 64 GB 96 GB Flash-Next IQ3_XXS – – 216 linux 2026-10-07 source
radeon 890m table+switches configuration, 7,010‑token prompt 64 GB 96 GB Flash-Next IQ3_XXS – – 226 linux 2026-10-07 source
2× v100 prefill ~256 tokens, cold prompt 32 GB 125 GB Flash-Next IQ3_S – – 182 linux 2026-10-07 source
2× v100 prefill ~1K tokens, cold prompt 32 GB 125 GB Flash-Next IQ3_S – – 531 linux 2026-10-07 source
2× v100 prefill ~4K tokens, cold prompt 32 GB 125 GB Flash-Next IQ3_S – – 1,082 linux 2026-10-07 source
2× v100 prefill ~16K tokens, cold prompt 32 GB 125 GB Flash-Next IQ3_S – – 1,684 linux 2026-10-07 source
2× rtx 2080 ti 22 GB 94 GB Flash-Next IQ3_XXS 90K – 1,491 linux 2026-10-06 source
2× rtx 2080 ti 22 GB 94 GB Flash-Next IQ3_XXS 128K – 1,436 linux 2026-10-06 source
2× rtx 2080 ti 22 GB 94 GB Flash-Next IQ3_XXS 80 – 96 linux 2026-10-06 source
v100 + p100 Prompt (prefill) speed for 13,561 fresh tokens (median of 443.5, 444.0, 444.0, 444.5 tok/s). 32 GB 120 GB Flash-Next IQ3_XXS – – 444 linux 2026-10-06 source
rx 6900 xt Prompt length 799 tokens, GPU timeline 3329 ms (default arm). 16 GB 128 GB Flash-Next IQ3_S – – 240 linux 2026-10-05 source
rx 6900 xt Prompt length 495 tokens, GPU timeline 2477 ms (default arm). 16 GB 128 GB Flash-Next IQ3_S – – 200 linux 2026-10-05 source
rx 6900 xt Prompt length 1108 tokens, GPU timeline 3391 ms (default arm). 16 GB 128 GB Flash-Next IQ3_S – – 327 linux 2026-10-05 source
rx 6900 xt Prompt length 8874 tokens, GPU timeline 11978 ms (default arm). 16 GB 128 GB Flash-Next IQ3_S – – 741 linux 2026-10-05 source
radeon 8060s – 128 GB Flash-Next Q2_0 – – 251 linux 2026-10-05 source
radeon 8060s – 128 GB Flash-Next Q2_0 – – 529 linux 2026-10-05 source
rtx 4060 ti 16 GB 96 GB Flash-Next Q2_0 – – 715 linux 2026-10-05 source
rtx 4060 ti 16 GB 96 GB Flash-Next Q2_0 – – 717 linux 2026-10-05 source
rtx 4060 ti 16 GB 96 GB Flash-Next IQ3_S – – 439 linux 2026-10-05 source
rtx 4060 ti 16 GB 96 GB Flash-Next IQ3_S – – 440 linux 2026-10-05 source
rx 6900 xt default attention kernel (STRATA_ATTN_PRE75=0) prompt speed for 9,427‑token prompt 16 GB 128 GB Flash-Next IQ3_S – – 441 linux 2026-10-04 source
rx 6900 xt default attention kernel (STRATA_ATTN_PRE75=0) prompt speed for 34,659‑token prompt 16 GB 128 GB Flash-Next IQ3_S – – 467 linux 2026-10-04 source
rx 6900 xt default attention kernel (STRATA_ATTN_PRE75=0) prompt speed for 105,811‑token prompt 16 GB 128 GB Flash-Next IQ3_S – – 460 linux 2026-10-04 source
rtx 4080 super Cold prefill speed (pool‑4 workers) – prompt length varies, value is median across workloads 32 GB 96 GB Flash-Next IQ3_S – – 2,535 windows 2026-10-04 source
rtx 3090 ti long prompt (~5,000 tokens), node‑0 binding (32 generated tokens) 24 GB 384 GB Flash-Next IQ3_S – – 1,304 linux 2026-10-03 source
v100 Prompt processing (warm) speed 32 GB 128 GB Flash-Next IQ3_XXS – – 62 windows 2026-10-03 source
rtx 3090 24 GB 121 GB Flash-Next IQ3_XXS 26K – 2,250 linux 2026-10-03 source
rx 9070 xt 16 GB 128 GB Flash-Next IQ3_S – – 370 windows 2026-10-02 source
rx 9070 xt 16 GB 128 GB Flash-Next IQ3_S – – 602 windows 2026-10-02 source
rtx 5090 32 GB 96 GB Swift IQ3_XXS 14K – 6,454 windows 2026-10-01 source
rtx 5090 32 GB 96 GB Swift IQ3_XXS 28K – 6,999 windows 2026-10-01 source
rx 7900 xtx 0.1.30 (control) median prefill throughput; decode range 16.0–34.2 tps not median 24 GB – Flash-Next IQ1_M – – 545 linux 2026-10-01 source
rx 7900 xtx 0.1.31 (candidate) median prefill throughput; decode range 16.7–64.7 tps not median 24 GB – Flash-Next IQ1_M – – 900 linux 2026-10-01 source
rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S – – 860 windows 2026-10-01 source
2× rtx 5060 ti 16 GB 128 GB Flash-Next IQ3_S – – 996 windows 2026-10-01 source
rtx 5070 Prompt prefill speed for 4K prompt (median of 465 and 467 tokens/s). 12 GB 64 GB Flash-Next IQ2_XS – – 466 windows 2026-09-28 source
rtx 5070 12 GB 64 GB Flash-Next Q2_0 – – 1,290 – 2026-09-28 source
rtx 5070 12 GB 64 GB Flash-Next IQ3_S – – 1,208 – 2026-09-28 source
rtx 5070 12 GB 64 GB Flash-Next Q2_0 – – 438 – 2026-09-28 source
rtx 5070 12 GB 64 GB Flash-Next Q2_0 – – 1,077 – 2026-09-28 source
rtx 5070 12 GB 64 GB Flash-Next Q2_0 – – 1,249 – 2026-09-28 source

1061 of 1061 measurements

Add your PC

Run BENCH.bat (Windows) or ./bench.sh (Linux) in your upstream engine folder, then paste the report into a GitHub issue. It shows up here within the hour.

How to run the benchmark →