r/LocalLLaMA
· Communities
Gemma4 with audio input: 16.8 tok/s on Macbook M2 Max 64GB
Here's the setup I decided on for embedding gemma4-12b into a Tauri2 desktop app: Native Rust FFI into llama.cpp via llama-cpp-2 (Metal enabled) Model:gemma-4-12b-it-Q5_K_S quantized by Unsloth, Q5_K - Small Audio input is a 607 KB 16-bit mono 16 kHz PCM WAV. Prompt path: Gemma chat template + llama.cpp mtmd (multimoda