Below Upstream Status sections are from https://github.com/PrismML-Eng/Bonsai-demo Upstream Status for Binary Q1_0 is supported out of the box in upstream llama.cpp across many backends: CPU (generic,…
Benchmark Results Benchmark configuration: threads = 96 type_k = bf16 type_v = bf16 Llama-3.1-8B-Instruct Q8_0 Prompt Size GGML_CPU_Q8_0 t/s ZenDNN_Q8_0 t/s Gain 256 472.28 730.87 54.75%…
submitted by /u/Ready_Performance_35 [link] [comments]
Efficiency defined as score over active parameters. Removed all the models that were not on the pareto frontier. Yes I'm aware that artificialanalysis.ai aggregate benchmark isn't…
What it is: ATH-MaaS/OvisOCR2 - a 0.8B document-parsing VLM post-trained from Qwen3.5-0.8B (SFT + RL + OPD), Apache 2.0, runs on vLLM 0.22.1. One prompt per…
https://huggingface.co/microsoft/bitnet-embedding-0.6b Dang, no model card submitted by /u/Silver-Champion-4846 [link] [comments]
Introduction RxBrain (Hy-Embodied-RxBrain-1.0) is a unified multimodal foundation model for embodied cognition — a single model that couples language reasoning with visual imagination to deliver three…
I've started building my rig and it turned out 24GB VRAM is not enough for my tasks (inference only without offload to RAM). What will be…
Some fixes & boost(pp) for SYCL/Intel. Merged PRs: [SYCL] Flash Attention with XMX engine via oneDNN graph API (SDPA) on KV f16 for Xe2 ; Qwen3.6-27b-Q8_0…
With the help of a LLM-wiki and a lot of AI, I am running some experiments with dLLMs (and yes, the I does quite some heavy…