r/LocalLLaMA
· Communities
Combining MTP with ngram-mod worth it for coding?
As I understood it, ngram helps with repeated token output, instead of re-generating identical output, the speculative decoding notices the same pattern in the past (e.g. think-block) and suggest a repeat, potentially leading to even higher TPS. I did a test with llama.cpp spec-type = draft-mtp,ngram-mod spec-ngram-mod