DSv4F doesn't ship a jinja, but for distributions that do and faithfully reconstruct what DS releases in their chat template python, every system message is hoisted…
TLDR below 👇🏼 I’ve seen a lot of hype around Qwen 3.6 35B and 3.5 120B lately, especially regarding coding and tool-use capabilities. On this subreddit…
Hello, Also I want to join the hype of posting token specs. CPU: 2x Intel Xeon CPU E5-2650 v4 @ 2.20GHz RAM: 2x 4 Channel 2400MHz…
Built my 1st inference machine and have been tweaking models trying to get the most out of my modest hardware. I think I’m at a good…
I deployed K3 on 32 H100s at work a couple of weeks ago and then got annoyed that there was no way to poke at it…
Hello everyone I want to join the hype of posting specs. CPU: AMD EPYC 74F3 24-Core RAM: 8 Channel 3200 DDR4 GPU: RTX A6000 48GB Prompt…
I spent a while debugging my local DeepSeek V4 Flash setup and wanted to share a few lessons from the process in case it saves someone…
Hey y'all. I'll be concise. TL;DR: DS V4-Flash-0731 @ UD-IQ2_M running fully in VRAM on 3xMI50s (90.9 GB model, 96 GB VRAM). Actual speed on llama-server…
Hey all, tldr / who this helps: you run a mixed multi-GPU box where the experts spill to RAM, and you want to stay in the…
I know in in this community LLM's are generally used for coding but there are other usecases besides coding and those usecases should be tested too.…