RTX 5090 96GB spotted on Alibaba?
submitted by /u/panchovix [link] [comments]
Latest
Intel Optane – Potential?
Intel's Optane technology as we know it has been discontinued. But while that's the case, I see it being highly promising still due to its role…
No wonder Qwen and Gemma are so different
Pasted the same HTML/JS code (330 lines) into Qwen 35B A3B and Gemma 26B A4B. Qwen: tokenized the input to 1609 tokens Gemma: tokenized the input…
Kimi K3 (Unsloth) IQ2-XXS from 711GB down to 478GB!!! Only Multi-language was removed to trim the size
Firstly a big thanks to the poster hellohazine, he basically only removed the multi-lingual fat of the model and just kept the english language intact. It…
Extremely slow DSpark draft model performance (1-2 t/s) with DeepSeek-V4-Flash on llama-server compared to MTP?
Hey everyone, I could use some advice on setting up speculative decoding correctly with llama-server. My Hardware: GPUs: RTX 4090 + RTX 6000 Pro (120GB total…
Is Microsoft-Phi dead?
Phi was one of my favorite models with a bit of a mixed reputation with some claiming it's benchmaxxed and others seeing its potential and usecases.…
enabling PCI-E p2p for consumer Nvidia cards will yield you more than you think
Disclaimer - no LLM was used to write this post/note As larger post about my setup will come later, want to give heads-up to folks who…
Comparing 4bit quants for MLX
Curious what people think are the ideal 4-bit quantization types on MLX These quants seem to be the most popular, at least for Gemma4 and Qwen3.6:…
any reasonably fast public benchmarks I should run quants of deepseek flash 0731 on?
I have various quants of this model and am curious how they perform. can anyone recommend which benchmark would be a good test case for quantization…
Building a budget 32GB → 48GB VRAM home AI server: 2-3x RX 9060 XT 16GB vs RTX 5060 Ti 16GB, AM5 vs used EPYC?
I’m planning a dedicated home AI server, mainly for local LLM inference, agents/tool use, Docker services, and eventually larger MoE models with CPU offload. My plan…