Perf regression on Turing (RTX 2080 Ti, sm_75): Z-Image Q3_K ~2x slower per step since ~build 450 (ggml bump)

[shonsubong]

Summary

On an RTX 2080 Ti (Turing, sm_75), generating with Z-Image-Turbo (Q3_K GGUF) got ~2x slower per sampling step somewhere between release master-425-bda7fab and master-450-a2d83dd, and the latest release (master-795-87a0177) is still slow. Same GPU, same model files, same command.

• master-425-bda7fab: ~1.8 s/it, 1280x720 / 8 steps ≈ 17.7 s total
• master-450-a2d83dd and master-795-87a0177: ~3.8 s/it, 1280x720 / 8 steps ≈ 32 s total

The 2x factor matches Turing's 2:1 FP16:FP32 throughput, which strongly suggests a matmul/kernel path that used to run in FP16 now runs in FP32 (or a Turing-optimized MMQ path was dropped) for quantized (Q3_K) weights.

Environment

• GPU: NVIDIA GeForce RTX 2080 Ti (Turing, compute capability 7.5), 11 GB
• OS: Windows 10, prebuilt *-bin-win-cuda12-x64 releases
• Model: z_image_turbo-Q3_K.gguf (diffusion) + flux-vae-bf16.safetensors (vae) + Qwen3-4B-Instruct-2507-Q4_K_M.gguf (llm)
• Command (identical across builds):

  sd-server.exe --diffusion-model z_image_turbo-Q3_K.gguf --vae flux-vae-bf16.safetensors \
    --llm Qwen3-4B-Instruct-2507-Q4_K_M.gguf -l 127.0.0.1 --listen-port 8234 \
    --steps 8 -s -1 --diffusion-fa --vae-tiling --vae-tile-size 32x32 --vae-tile-overlap 0.125
  

Measurements (1280x720, 8 steps, warm server, pure generation)

release	sd-server.exe md5	ggml submodule	s/it	total
master-425-bda7fab	47bd0a705cf6a1b24fe601757c3d157e	f5425c0e	~1.8	17.7 s
master-450-a2d83dd	f35ee7703d82e0e1105bbee4a0611058	3e9f2ba3	~3.8	33 s
master-795-87a0177 (latest)	(win-cuda12 build)	eced84c8	~3.8	32 s

The regression window is builds 426–450, during which the ggml submodule was bumped f5425c0 → 3e9f2ba. The sd.cpp diff in that window is mostly features (schedulers, loras, qwen-image-edit, caching) with no obvious 2x-diffusion change, so the ggml CUDA kernels are the prime suspect.

Things tried that do NOT help (latest 795)

All keep it at ~32 s:

• --max-vram 0 (disable graph-cut segmentation), --eager-load
• --diffusion-conv-direct
• removing --diffusion-fa (makes it worse, ~55 s)
• env GGML_CUDA_FORCE_MMQ=1
• env GGML_CUDA_FORCE_CUBLAS_COMPUTE_16F=1 (no effect — Z-Image Q3_K appears to go through the MMQ path, not cuBLAS)
• env GGML_CUDA_DISABLE_FUSION=1 + GGML_CUDA_GRAPH_OPT=0

So there is no runtime workaround for Turing users on current releases; the only way to get the old speed is to run the pre-regression master-425 binary.

Ask

Could the Turing (sm_75) quantized-matmul / attention path that regressed in the ggml bump between builds 425 and 450 be restored to FP16, or exposed via a flag/env for older GPUs? Happy to bisect the ggml range (f5425c0..3e9f2ba) and test candidate builds on the 2080 Ti if that helps.

[shonsubong]

Update — confirmed it is the ggml CUDA kernel code, not a build/arch/runtime config.

I built master-795-87a0177 from source on the 2080 Ti (CUDA 12.6, MSVC) with:

cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DSD_CUDA=ON   -DGGML_CUDA_FORCE_MMQ=ON -DCMAKE_CUDA_ARCHITECTURES=75-real

i.e. native Turing SASS (no PTX-JIT fallback) and forced MMQ kernels. It is still ~32 s / 3.8 s per step for Z-Image Q3_K at 1280x720 / 8 steps — identical to the prebuilt release, and ~2x slower than the master-425 binary (~1.8 s/step, 17.7 s).

Runtime toggles also make no difference on the release build: GGML_CUDA_FORCE_MMQ, GGML_CUDA_FORCE_CUBLAS_COMPUTE_16F, GGML_CUDA_DISABLE_FUSION, --max-vram 0, --diffusion-conv-direct; removing --diffusion-fa is worse (~55 s).

So the regression is in the ggml CUDA kernels themselves (submodule bump f5425c0 -> newer between builds 425 and 450), specifically on the Q3_K matmul / attention path for sm_75. Happy to test a patch or bisect ggml on the 2080 Ti.

[Green-Sky]

If you could bisect the exact commit, that would be great.