Looking for a sanity check / review of my system, plus opinions. I'm a lifelong IT professional techie but not a software developer (I did HTML…
I've spent the last year or so playing around with open source MLX and GGUF models on iPhone hardware. Given the limitations in memory, GPU/CPU/ANE, and…
I’m trying to make a tool that edits my shorts for me and from my understanding, last time I tried to use llama and Gemma it…
Just noticed this exists, 1M context MOE with 48B par seems just like what Ive been looking for - it runs pretty damn fast too compared…
Disclosure: this is my own repo — all numbers below are measured on my own board. I've had a Jetson Orin NX 16GB sitting on my…
I got tired of "your Mac can't run that" so I forked llama.cpp to stream a MoE model's expert weights from SSD instead of forcing the…
Cuda and Vulkan Benchmark: TensorSharp vs. llama.cpp I would like to share my latest open source local Unsloth (GGUF) LLM inference engine and applications. It supports…
Everything runs through WebGPU, in-browser or in electron/tauri apps. It's fully portable and supports either Nvidia and Apple Silicon (Metal). The actual kernels are optimized for…
Tweet : https://xcancel.com/Mononofu/status/2080937562739531837#m submitted by /u/pmttyji [link] [comments]
submitted by /u/InternationalGap3698 [link] [comments]