What am I missing?
Training my own micro-llama-model on a dataset I have published with my own program, I somehow fail to get it working in Unsloth Studio and other…
Latest
4090 + 5060 Ti + 64GB RAM: 206 t/s on a 35B-A3B, and a 122B at 37 t/s
I've been benchmarking a two-card box for a few weeks and I still can't quite get over some of these numbers, so I'm dumping them here.…
What is the fastest local research tool (deep research) ?
I've tried grok and Claude's deep research mode and I was amazed with the speed considering the amount of sources analysed. Is there anything as fast…
Anyone tested the IQ1_M 342GB Pruned Kimi K3? Is it usable?
submitted by /u/Hannibalj2ca [link] [comments]
Jetson Nano with embedding models
Hello! Does anyone tried text embedding models on Jetson Nano 2/4Gb? I need it for the RAG. I want to know the speed. For example microsoft/harrier-oss-v1-0.6b…
Bought a 5090 to escape API fees. Ended up building a mini datacenter. Sound familiar?
I bought an RTX 5090 last year just to run 27B models natively. I even fine-tuned it with my own data using LoRA, building RAGs and…
Extracting MoE experts from Kimi K3
Has anyone yet tried to extract experts from kimi (or GLM 5.2) per chance? There is REAP that removes experts based on routing, but I could…
3090 owners, what vram tempature do you get under ai load?
Hello Can you please share the tempature you get on your rtx 3090 under active llm load? Im trying to findout if my rtx 3090's tempatures…
Quantizing Kimi K3 (2.8T A50B) to GGUF ourselves – Q3_K_S works, 1.1 TB on disk
we're experimenting with our own dynamic GGUF quants of kimi k3, made from the original weights with our llama.cpp fork. Q3_K_S is done and works 1114.76…
Are you guys not scared of where we’re heading? A year ago, GPT-5 was considered one of the best models in the world. Today, we have open-weight models like Qwen3.6-27B that are competitive enough to run locally on high-end consumer hardware. The pace of progress is absolutely brutal.
I think the claims about having Mythos level-model in our laptops in 1-2 years might not be so crazy of a theory submitted by /u/SilverRegion9394 [link]…