Qwen3.8 Flash Next NVFP4 at 109 tok/s on RTX PRO 6000 | SM120 QSA Fix

Running Qwen3.8-Flash-Next locally on two RTX PRO 6000 Blackwell 96GB GPUs using SGLang's day-0 image, the RadixArk NVFP4 checkpoint, and the SM120/SM121 QSA sparse-decode fix from SGLang PR #36556. The original image crashed during warmup because SM120 fell through to an incompatible flash-attn CuTe varlen path. Commit dac5523 enabled FlashInfer TRTLLM sparse decode; autotuning and graph capture completed, and the agent workload reached approximately 119K tokens. Qwen3.8-Flash-Next is a 176B-parameter, 6B-active-per-token GDN + QSA hybrid MoE preview of Qwen4. In this post-training ModelOpt checkpoint, routed experts use NVFP4 W4A4 while the other main components remain BF16. PLE tables are FP8 on disk and BF16 at load time. Runtime KV cache was BF16. Image and runtime Base image: lmsysorg/sglang:qwen38flashnext Derived image: registry.home.arpa/lmsysorg/sglang:qwen38flashnext-sm120-qsa-dac5523 Patch: dac5523, qwen_sparse_attn_backend.py only CUDA: 13.0.3 FlashInfer: 0.6.17 ```step1 curl -fL \ ttps://github.com/sgl-project/sglang/commit/dac5523d1e5d2f4297fec40ef02fc76fb0f662d1.patch \ -o qsa-sm120.patch ``` ```step2 docker.io/lmsysorg/sglang:qwen38flashnext (same) FROM lmsysorg/sglang:qwen38flashnext COPY qsa-sm120.patch /tmp/qsa-sm120.patch RUN cd /sgl-workspace/sglang \ && git apply --check \ --include='python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py' \ /tmp/qsa-sm120.patch \ && git apply \ --include='python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py' \ /tmp/qsa-sm120.patch \ && rm -f /tmp/qsa-sm120.patch ``` ```step3 podman build \ --pull=always \ --tag registry.home.arpa/lmsysorg/sglang:qwen38flashnext-sm120-qsa-dac5523 \ --file Containerfile \ . ``` Configuration Model: RadixArk/Qwen3.8-Flash-Next-NVFP4 CPU: AMD EPYC 9175F GPU: 2x RTX PRO 6000 Blackwell 96GB, SM120 Maximum context: 262,144 Maximum running requests: 2 Chunked prefill: 4,096 Page size: 64 GPU static memory fraction: 0.80 Attention: FlashInfer QSA sparse decode: FlashInfer TRTLLM MoE / FP4 GEMM: FlashInfer CUTLASS Linear attention: Triton prefill, FlashInfer decode Mamba SSM: BF16 with extra_buffer radix cache PLE embedding: CPU offload enabled KV cache: BF16 Decode CUDA graphs: batch sizes 1 and 2 Prefill CUDA graphs: disabled for multimodal Speculative decoding / MTP: disabled Memory and initialization Weight memory: 42.04 GB per GPU, 84.08 GB combined Mamba cache per GPU: 0.59 GB convolution + 14.98 GB SSM KV cache per GPU: 8.16 GB K + 8.16 GB V Reported KV capacity: 1,425,152 tokens Memory remaining after pool allocation: 18.79 GB per GPU Weight loading: 77.38 seconds Decode graph capture: 2.27 seconds Scheduler startup: 114.45 seconds Tokenizer startup: 121.19 seconds Ready after warmup: approximately 143 seconds Observed results Generation throughput, 2,057 decode samples: Median: 108.91 tok/s Mean: 107.37 tok/s Tukey IQR: 108.30-109.41 tok/s At 100K+ full tokens, steady decode: Median: 107.98 tok/s Mean: 108.54 tok/s Tukey IQR: 107.79-108.31 tok/s At the final observed 118,976-token state: 107.62 tok/s The isolated 204.53 tok/s value was a two-to-one-request transition spike. Only one concurrency-2 sample was captured, so this is not a concurrency-2 result. Prompt processing: Five consecutive uncached 4,096-token chunks sustained 14,412.72-14,693.17 tok/s. Median: 14,604.64 tok/s Short terminal chunks produced timer spikes up to 799,048.16 tok/s; those were excluded. The final request reused 118,400 cached tokens and added 320 new tokens, approximately 99.73% prefix reuse. Agent workload artifact The coding agent produced a substantial Django restaurant reservation system with 25 models, all 9 required service operations, 12 REST routes, 18 Admin registrations, CSV support, a migration, SQLite data, and an idempotent seed command. It covers table allocation, service calendars, capacity, deposits, cancellations, reservation history, host-stand operations, guest preferences, and waitlists. The result is a strong, coherent generated prototype. *The requested automated tests were not generated. SGLang cookbook: ttps://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next NVFP4 model card: ttps://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4 SM120/SM121 QSA fix: ttps://github.com/sgl-project/sglang/pull/36556