Speculative-Decoding

Running local LLMs on a Radeon AI PRO R9700: what actually made it faster

Measured tuning notes for llama.cpp on a 32 GB Radeon AI PRO R9700: model choice, speculative decoding, KV-cache sizing, the GTT spill trap, quantization, batching, thinking control and one flag that silently did the opposite of what my config said.

ai engineering