r/LocalLLaMA
· Communities
Getting the most out of MTP
If you want to get the most out of MTP. You have to run some tests / benchmarks to do so. Turning it on with defaults will get improvements, but for many models and card combinations, you are leaving a lot of performance on the table if you don't tune n_max. Can be easily missing out on 50-100% of the possible performa