Qwen 3.8 get stuck in massive 20k–50k token thinking loops on SGLang?
I’m running Qwen/Qwen3.8-27B via SGLang on an ASUS GX10 (Grace Blackwell GB10, 128GB unified memory) behind a LiteLLM proxy.
I use BF16 with FP8 kv.
In nothink mode, the model is amazing.
However, whenever I test it with thinking mode on large, multi-step coding prompts (e.g. a 5k token project plan), it gets trapped in an insane thinking loop. It sat generating for 45 to 80+ minutes, churning through 20,000 to 55,000+ tokens of internal thought before writing a single line of code or calling a tool.
What I tried:
Passed
reasoning_effort: “low”viachat_template_kwargs.Passed
thinking_budget: 2048.Adjusted sampling temperature.
Looking at the Jinja template and engine logs, reasoning_effort: “low” just adds a soft prompt text asking the model to keep it brief. But on big prompts, Qwen ignores the suggestion and tries to mentally draft the entire codebase inside ``, getting stuck in an endless loop of re-checking its own thoughts.
Has anyone run into this with Qwen 3 / 3.8 on SGLang? Is there a way to enforce a hard token ceiling on the reasoning block (like a real max_thinking_tokens that forces ``) so it doesn’t spin out for an hour?
Any tips or working configs would be appreciated!
I'm seeing this weird behavior with the Qwen3.8-27B-UD-Q4_K_XL.gguf model from:
https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
In a fresh chat, I asked it:
"Give me 100 jokes."
It successfully generated 100 jokes.
Then I asked it again:
"Give me another 100 jokes."
At that point, it got stuck in what seems like an endless reasoning loop. It keeps checking its own reasoning, noticing that it already used a certain joke, replacing it with a new one, then noticing that it already used that one too.
It can continue doing this for hours without ever producing the final answer. The strange part is that the reasoning itself also starts repeating the exact same patterns and wording over and over again, almost like it is trapped in a loop.
At that point, it got stuck in what seems like an endless reasoning loop. It keeps checking its own reasoning, noticing that it already used a certain joke, replacing it with a new one, then noticing that it already used that one too.
It can continue doing this for hours without ever producing the final answer. The strange part is that the reasoning itself also starts repeating the exact same patterns and wording over and over again, almost like it is trapped in a loop.
Well, are you using any penalties? Quantized Qwen models do repeat. Qwen itself recommends presence penalties.
I would start with --dry-multiplier 0.5 and --dry-penalty-last-n 4096 for taming repetitive paragraphs.
For repetition of short words, --repeat-penalty 1.1
These values are suggestions only. Increase/decrease by 0.1 at a time for fine tuning. Check the llama.cpp docs for valid values.
Thanks for the suggestion! We tested repetition_penalty (at 1.05 and 1.08), but found our loop wasn't actually lexical
word/paragraph repetition.
Instead, it was an unbounded semantic self-validation loop: the model would draft a working solution inside , but then
continuously question itself with new phrasing ("Wait, let me double-check edge case X...", "Now let me verify Y from another art
angle..."). Because it was generating unique tokens each iteration, repetition penalties didn't trigger, and pushing the
penalties higher damaged code syntax (penalizing variable re-use and language boilerplate).
The only thing that reliably solved it for us without degrading coding quality was enforcing an explicit thinking_budget (e.g.
2048 / 6144 / 16384 tokens) combined with strict temperature: 0.6 and top_p: 0.95.
I know gglooper's example was a quant. But I don't want his comment to muddy the spirit of this conversation. I only use BF16.
reasoning_effort: low is a plausible cause of the loop, not a failed fix. The model card's own warning:
▎ lower reasoning effort … may cause "insufficient analysis, more failures, and repeated retries"
Low on a 5k-token multi-step planning task — exactly the workload where a truncated analysis budget forces the model to re-derive what it couldn't finish. Default is xhigh.
Test the default before anything else
Next...
top_k is almost certainly missing. The card requires top_k 20 in both modes, and SGLang defaults it to -1 (disabled).
@tohrbaugh I'll give that a try. thanks!
Also presence_penalty is 0.0 in thinking mode and 1.5 in non-thinking — so the anti-repetition lever the card recommends is only available in the mode you might not be using.