arXiv cs.AI
· Papers
SonicSampler: Unified Tile-Aware Kernels for LLM Sampling and Speculative Verification
arXiv:2607.20475v1 Announce Type: new Abstract: Sampling in LLM inference comprises a combinatorial set of logit processing, token selection, and verification operations for speculative decoding. However, existing implementations either accelerate only subsets of this pipeline, rely on multiple kernel launches, or assu