b10249
server: add get_info tool (#26522) server: add get_info tool fix --rpc in docs server: harden get_info probe result handling Report the OS as unknown when the…
Infrastructure 934 stories
b10248
vocab : validate default special token ids (#26506) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
b10247
ggml: use dynamic allocation for split graph inputs (#22789) ggml: use dynamic allocation for split graph inputs Replace fixed-size GGML_SCHED_MAX_SPLIT_INPUTS arrays with dynamically allocated buffers in…
The Ultimate Guide to FLUX 3
Black Forest Labs' first video model generates audio and video from the same pass, and it takes direction like nothing else we've tried.
b10246
opencl: route large q6_K lm_head to the flat GEMV (#26427) add a direct size condition for large weights; the original dimension condition is insufficient -- q6_K…
Customize the reasoning level for Copilot cloud agent
When you delegate a task to GitHub Copilot cloud agent, you can now set the reasoning level for models that support it. This allows you to…
b10245
graph : fix unused input tensors in minimax m3 graph (#26519) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
Enterprise team specialization for managed settings
Enterprise administrators can now customize managed settings by targeting enterprise teams with itemized configuration files. Large enterprises can scale governance without bottlenecking every configuration change through…
b10244
model: M3: Move MSA into a new memory implementation (#26338) Move MSA logic from llama-kv-cache into llama-kv-cache-msa cont : minor cont : ws fix Co-authored-by: Georgi…
b10243
llama : allocate indexer cache only in "full" indexer layers (#26474) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
Trigger Copilot automations with comments
You can now create Copilot cloud agent automations that run when an issue comment or pull request comment is created. Common use cases include: Generate documentation:…
b10242
CUDA: Add backend sampler for penalties sampler (#25262) sampling: enhance penalty handling in common_sampler_init Set default value for penalty_last_n based on model context if not specified.…
How to Run Isolated Tenant Kubernetes Clusters on Shared GPU Infrastructure
Running a dedicated Kubernetes cluster per team often results in more isolation than an organization requires. While one cluster can be successfully shared...
NVIDIA Vera Storage Benchmarks: Faster Encryption, Compression, Integrity Checking, and Recovery for AI-Native Storage
Storage is an active part of every agentic AI workflow. As agents retrieve enterprise knowledge, access persistent memory, reuse key-value (KV) cache data,...
b10241
CUDA: Fix data-races when reusing SMEM in block_reduce (#26385) CUDA: Fix data-races when reusing block_reduce block_reduce currently doesn't resync after reading from SMEM, causing potential data-races…
b10240
server: add notice for upcoming default port change 8080 --> 9931 (#26508) server: add notice for upcoming default port change 8080 --> 6631 add link to…
b10238
model: MTP support for Qwen3-Next (#25589) mtp for qwen3nex fix for python type-check Fix to compute num_mtp from directly mtp layer define opt_num_mtp_layers in _QwenMtpMixin and…
b10237
llama : MTP support for DeepSeek V3.2 (#26457) llama : MTP support for DeepSeek V3.2 model : no need to include MTP layers during DeepSeek V3.2…
b10236
metal: implement DSv4 Lightning Indexer (#25893) metal: implement F16 Lightning Indexer Implement GGML_OP_LIGHTNING_INDEXER for 128-dimensional, 64-head inputs with F32 queries and weights plus F16 keys and…
b10235
metal : add SILU_BACK (#25982) feat(silu_back): implemented silu_back op for f32 fix(silu_back): removed redundant asserts in ggml-metal-ops.cpp function ggml_metal_op_silu_back. Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64)…
b10234
metal : add F16 support for bin ops (#26465) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
b10233
opencl: limit local workgroup size for GLU operation (#26383) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
b10232
metal: implement DeepSeek V4 hyper-connections (#26459) Implement GGML_OP_DSV4_HC_COMB, GGML_OP_DSV4_HC_PRE, and GGML_OP_DSV4_HC_POST with SIMDgroup register and shuffle optimized kernels. Add Metal dispatch and support plumbing and test…
b10231
common: support the DSpark sidecar resolution (#26458) The dspark- files resolve like the other speculative sidecars: the -hfd tag applies to them, a requested sidecar resolves…