Spent two weeks on a kernel that benchmarked 29x faster. End to end it’s maybe 6-10%, and it’s not even wired in yet.
I've been building a C99 inference engine from scratch (no Python, no BLAS, just gcc and make) that runs BitNet's ternary models on CPU. A few…