arXiv cs.LG
· Papers
Operator-Aware Mixed-Precision Tolerance Calibration for Tensor Kernels
arXiv:2607.16228v1 Announce Type: new Abstract: Most tensor-kernel correctness tests go through a fixed-shape all close-style check with hand-picked absolute and relative tolerances. The thresholds are copied across the corpus and rarely revisited. We mine the element-wise error distribution of every test case from acc