r/LocalLLaMA July 23, 2026 · Communities

I distilled an 8B teacher into a 0.6B student on my Mac (MLX). The 0.6B went from 36% to 100% on the task, but few-shot prompting actively made it worse.

I work with financial documents for my job, and I wanted a small model I could run locally to enrich them before they hit a RAG index: for each chunk, write a faithful (3 sentence) summary and tag a few categorical facets (section type, specificity, numeric density, how forward-looking it is). Sending every chunk to an