llama.cpp b9966 for sm-tensor
B9966 If you run -sm tensor in production you might want to grab this fix which removes 29 regex recompilations per tensor per token on the…
Latest
[not mine] costs
Saw this on a random social media post. I believe what we are seeing here is two sxm2 gpus, right? V100 or a100? Assuming two v100…
Here’s your recipes for models for the most popular hardware
I occasionally comment and people ask how do I run something, how fast the model x on hardware y. There you go. Added a separate website…
Ultra budget 20GB vram with 448GB/s for $100 bucks.
Here is the upper limit of what can be done with $100 bucks worth of video cards. You can have 3 concurrent users with plenty of…
Performance comparison on full compute performance (Anima) and LLM prompt processing of 5090 (600,475 and 400W) vs 6000 PRO MaxQ shunt modded and water cooled (at 300, 400, 475 and 600W), and 6000 PRO WS/SE (600W).
Hello guys, hoping you're doing fine! I'm continuing after this post some time ago, comparing stock MaxQ performance and such on Anima here. This time, I…
I benched quad 5060Tis for code generation with Qwen3.6-27B so you don’t have to (it’s really good)
WEIRD DISCLAIMER: none of this was written by an LLM until you get to the Github repo/site, which was obviously assembled by your friend and mine,…
Is it possible to run Qwen 122B in 64GB ram + 24gb vram ? If so, how ?
Which settings would suffice to work with it ? submitted by /u/soyalemujica [link] [comments]
actual advice about SLM fine tuning?
hello real people and less-real bots, i'd appreciate if any of you people who have fine-tuned (either full or peft) more than half a model could…
What are some best benchmark for the llm? Like i am tired of benchmaxxed numbers. List me some test which can tell me which model is better in coding and agentic task.
I know about DEEPSWE but it lacks many models :( submitted by /u/9r4n4y [link] [comments]
I built Flaxeo Image a local desktop ui for stable diffusion cpp
Built around a recent sd.cpp release, aims to expose most of what the backend can do (generate, edit, video paths, models, hardware options), Windows + Linux…