For the last day or so I've been using Agents A1 Q8 InternScience/Agents-A1-Q8_0-GGUF on my M1 Max mac (64GB) just like this: llama-server -hf InternScience/Agents-A1-Q8_0-GGUF --host…
Hardware-agnostic strategies for accelerating text-to-image diffusion, such as timestep distillation and feature caching, can reduce inference time without custom kernels or system-level optimization. Among them, multi-resolution…
currently using Qwen3.6 35B as my main assistant model + coding agent but I think sometimes it misses basical general knowledge things, and it is more…
GitHub Copilot allows Bring Your Own Key (BYOK) for its Chat window, but it explicitly blocks those exact same custom models from being used for inline…
I have an old Xeon rig with 512Gb of 4-channel DDR4 2133 memory and E5-2699v4 processor. For GPU I have GTX 1060 with 6Gb of VRAM,…
Here is a fun test prompt: Imagine a date in the next 1000 years where the Sun, along with its gravity, suddenly disappeared. When that happens,…
I had built a pc in 2023 for pretty cheap, it has a RTX 4090 and intel i9 13900K. I’m thinking of adding a 5060Ti for…
I benchmarked 13 models at 65K-128K context to find out what actually matters for agentic workloads — prefill dominates everything, and KV head count beats parameter…
Parsed from VLLM log file ~2000 tps in aggregate performing bulk captioning on images. Above is parsed from vllm log while a client runs 30 concurrent…
Not really a tutorial, but more of sharing my attempts at getting higher contexts on Q8 of Qwen3.6-27 with 32GB VRAM. Disclaimer: Not in-depth research. Crowd…