Linear-attention and state-space language models compress the prefix into a fixed-size recurrent state, yielding O(1) memory at the cost of a lossy exact memory: when many…
A few days ago I shared this resource I created to convert your Claude Code sessions into training data (Thank you so much for all the…
I have been running Qwen 3.5 122B at 4 bit for quite a while, and have started running it at 5 bit recently now that Llama.cpp…
You know that chrome lately downloads a local model (Gemini Nano). Which probably is a gemma vision quantized model. I tested it inside the browser, but…
This is a really cool application: counterfeit medication detection-in-a-box. Having a visual expert with you that doesn't depend on a network connection is what I expected…
Hey all. I have an older post here with these stacked / Air cooled. Just wanted to update the group here with how the build sits…
Just ran Qwen 3.6 27B using MTP for the first time. Doubled my t/s. Wow. That is all. I'm going to go look for abliterated MTP…
Given how much of a boost in output quality these methods seem to enable when it comes to the big cloud models of having several smaller,…
I’m having a hellluva time trying to fine-tune an FP8 LLM model (both FFT and LORA). I’ve been using VLLM and they sure don’t make it…
I tried running local models (qwen3.6*, ds4 flash, gemma4*, etc) on my mbp pro m5 with 128Gb of unified memory and concluded the bottleneck is context…