DeepSeek V4 Flash 0731 at 10–17 t/s (nothink) on MacBook M5 Pro **64GB***, partly via SSD streaming
Inspired by a post from u/giveen I motivated claude (no patinence on my side to work through everything myself) to help me get DS running on…
Latest
Practical question: thinking of using my M4 max 128gb MBP as a LLM server and using an iPad w/ Terminus as my daily driver. Anyone doing this?
It feels kinda redundant almost given that the MacBook is already mobile, but using it as a server would allow me to leave it always on…
Resources tutorials or videos on image gen?
Anyone recommend a source to get started with video/image gen? I pretty much only have experience with llms and right now I just tell the llm…
Agent memory layers don’t need an LLM deciding what to remember
Most agent memory setups run a model call on the way in. Something reads the turn, decides whether it's worth keeping, rewrites it into a "memory",…
Ling-3.0-flash MXFP4 released and running locally on one DGX Spark.
In tests: ~80 tok/s decoding 2,500–3,500 tok/s long-input prefilling Smooth use by 3–4 concurrent users Private, on-device inference for coding, agents, and offline batch jobs submitted…
The Session You Cannot Take With You | EARENDIL
submitted by /u/MoneyPowerNexis [link] [comments]
LFM2.5-2.6B on a OnePlus 13 at 17 tok/s ~ Pure CPU
As you all know the model is 2.69B parameters with a 128K context window and purpose-built for multi-step agent workflows. What you are seeing is the…
DGX Spark now sells for 6000-8000 euros. I still remember when it was just 4000.
submitted by /u/Afraid-Yoghurt6731 [link] [comments]
Given the MiniMax H3 LoRAs Debacle – Some Important Context for Censorship enforcement and laws in China
*I felt the need to write this post because it seems like very few people on this sub are aware of Chinese laws and how they're…
A 460M VLM gets first-token latency down to 0.3s on an iPhone by using only 64 visual tokens
A new 460M vision model called VisionPsy-Nano-460M-Flash is taking a slightly different approach to on-device VLM speed. Instead of making the language model much smaller, it…