b9956
cli: fix crash on wrong server base url (#25497) llama-cli: fix crash on wrong server base url by catching exceptions and graceful exit review: leaner catch…
Infrastructure 950 stories
Reducing High-Bandwidth Memory Bottlenecks in JAX-Based LLM Training with Host Offloading
Large language model (LLM) training workloads increasingly run into GPU memory limits before compute is fully used. Model weights, gradients, optimizer states,...
b9952
llama : make all KQ masks f16 if FA is used, remove zero attention bias, remove raw_k repeats in DeepSeek V4 (#25370) llama : make all…
Kernel Fusion in NVIDIA CUDA: Optimizing Memory Traffic and Launch Overhead
There are many ways to optimize code for GPUs. In this post, you’ll learn how kernel fusion can improve memory bandwidth and reduce kernel launch overhead,...
AI Model Co-Design: Hardware-Friendly LLM Design
AI performance comes down to three dimensions: Accuracy: How well the model reasons and produces outputs Throughput: How many tokens per second a...
Agentic autofix for code scanning alerts in public preview
Agentic autofix is now in public preview for code scanning alerts. It remediates alerts by working across your codebase the way a developer would: it explores…
b9950
llama-batch: add unit test (#25471) llama-batch: add unit test fix win32 builds add not implemented assertion in unused methods remove unreachable code macOS/iOS: macOS Apple Silicon…
Accelerating End-to-End Co-Folding Performance with NVIDIA BioNeMo Agent Toolkit
Biomolecular structure prediction and co-folding with models like OpenFold3 are now mainstream, large-scale workloads powering drug discovery and protein...
b9949
opencl: cluster-parallel decode FA for Adreno (#25473) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
b9948
ggml : process data in smaller chunks in CUDA ggml_top_k() and ggml_argsort() to reduce temporary buffers memory usage (#24776) ggml : process data in smaller chunks…
b9947
cli: add --output option (#25484) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
b9951: ggml-et: Initial ET backend (#24179)
ggml-et: Add performance logging ggml-et: Quants helpers ggml-et: Add MUL_MAT kernel ggml-et: Add ROPE kernel ggml-et: Add RMS_NORM kernel ggml-et: Add GLU kernel ggml-et: Add SOFT_MAX…
b9946
hexagon: tiling, tracing and optimizations for unary ops (#25474) hexagon: tile wide rows in pointwise unary ops to avoid VTCM overflow unary: reject permuted tensors for…
b9945
server : move chat-template thinking probe inside the init try/catch (#24093) A model whose chat template parses at init but fails parser generation at apply time…
b9941
Only index by compile times + always multiply/add (#25445) The first one avoids relying on compile to optimize local memory away, and the second is cheaper…
Synthetic Data Generation for Financial AI Research with NVIDIA NeMo
Fine-tuning LLMs for financial natural language processing (NLP) is constrained by limited, imbalanced data. Real-world financial news overrepresents earnings...
A Practical Guide to GPU-Initiated Communication for Molecular Dynamics at Scale
Molecular dynamics (MD) simulations are among the most demanding workloads in computational science. Using them, researchers can observe atomic behavior in...
OpenAI’s GPT-5.6 Sol, Terra, and Luna are now available in GitHub Copilot
OpenAI’s GPT-5.6 family is now rolling out in GitHub Copilot. GPT-5.6 comes in three variants, Sol, Terra, and Luna, so you can match the model to…
Ask Copilot for a repository overview
You can now ask GitHub Copilot for a high-level overview of any repository you’re exploring for the first time. When you visit the home page of…
GeForce NOW Turns Up the Heat With New GeForce RTX 5080-Powered Toronto Server
This GFN Thursday brings more games, more power and more ways to play on GeForce NOW. The cloud gaming service is expanding with a new GeForce…
b9940
llama-bench : init params.offline (#25476) Signed-off-by: Adrien Gallouët angt@huggingface.co macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
b9939
metal : add CONV_2D_DW (depthwise convolution) support (#21565) metal : add CONV_2D_DW (depthwise 2D convolution) support test : add perf cases for CONV_2D_DW metal : use…
b9938
ggml-hip: enable -funsafe-math-optimizations (#24668) CUDA is compiled with fast math and AMD/HIP is not — this flag lets AMD use fast math too. We can't use…
b9937
cuda: align snake fusion matcher with the other backends (#25460) cuda: fix snake fusion type predicate, a and inv_b are F32 The matcher required a->type ==…