b10208
SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt proc… (#25025) SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt processing fattn-mkl: fix interleaved dst layout…
Infrastructure 934 stories
b10207
[SYCL] support the missed types in cpy (#26005) support the missed types in cpy use correct funct rm unused code Website: https://llama.app macOS/iOS: macOS Apple Silicon…
NVIDIA Video Codec SDK 13.1: Zero-Copy Transcode, AV1 B-Frames, and Frame-Accurate Seek
The demand for high-quality video continues to accelerate across industries, powering everything from immersive streaming experiences to remote collaboration,...
b10206
llama : enforce the same K and V cache types for DeepSeek V4; enable FA if V cache is quantized (#25871) llama : enforce the same…
b10205
ggml-zendnn : group matmul direct API for mul_mat_id (#25918) ggml-zendnn : group matmul API for mul_mat_id ggml-zendnn : scale MUL_MAT_ID fallback threshold by expert count Website:…
b10204
sycl : support dev2dev memcpy by DEV2DEV_MEMCPY_FORWARD (#26234) Co-authored-by: Neo Zhang Jianyu jianyu.zhang@intel.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
b10203
[SYCL] Support q2 mul_mat (#26231) support q2_0 in mul_mat support more q2_0 case Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
b10202
sycl: fuse RMS_NORM + MUL (#26015) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
b10201
ggml-webgpu: improve flash_attn_vec for quantized KV at long contexts (#25956) improve fa of quantized kv cache Fix some bugs and some comments. fix v type check…
Autoscaling endpoints for LLM inference
GPU utilization can read healthy while your queue backs up, and a new replica takes minutes to warm. Here's how to pick autoscaling metrics, tune scale-up/down…
Run High-Performance Core Math at Scale with NVIDIA nvmath-python
NVIDIA nvmath-python is a library designed to bridge the gap between the Python scientific community and NVIDIA CUDA-X math libraries. It gives Python users...
Four Ways to Deploy More Secure AI Agents
Knowledge workers are increasingly integrating AI agents into their workflows. Agents that function as "digital coworkers" offer clear benefits. For example,...
b10199
server: support inp embd to generate next token (#26313) server: support embd for sampled token fix ~server_batch() Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
b10200
mtmd: add lanczos resize method [no release] (#26341)
b10198
vulkan: Support quantized concat (#25684) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
b10197
Test support for alternative conv layout (#25617) add bool cwhn = true to conv_2d test cases add layout check at graph building time extend layout checks…
b10196
llama-context : sync pending async copies before clearing embd_seq (#25676) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
b10195
tests : avoid building get-model.cpp many times (#26317) tests : remove get-model.cpp tests : fix quant type selection Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS…
NVIDIA Exemplar Cloud: Lessons for Unlocking Full Performance on AI Infrastructure
Two AI computing clusters built from identical NVIDIA H100, GB200 NVL72, or GB300 NVL72 systems can deliver materially different training throughput. We...
GitHub Copilot in Visual Studio — July update
July 2026 brought a new agent based on the Copilot SDK, built-in expertise from the .NET and Azure teams, and more ways to tailor GitHub Copilot…
b10194
ggml-cuda: Allow transpose-free gemmv computation (#26171) When matrix's weights are shaped 1xK is leverage a transpose-free computation to use mat_mul_vec_f. Website: https://llama.app macOS/iOS: macOS Apple Silicon…
Limit remote control to managed devices
Enterprises and organizations can now restrict which devices are eligible to host remotely controlled Copilot sessions, giving administrators fine-grained control over where remote control access is…
b10192
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
Best in Class: Stream PC Games and Study on the Same Laptop With GeForce NOW
Back to school means balancing assignments, deadlines and downtime. GeForce NOW makes it easy to have it all. With cloud gaming, everyday laptops used for class…