r/LocalLLaMA
· Communities
Introducing DWARF-55M-Base
Finally after months of research, the very first model made from the DWARF architecture is available for folks to check out and experiment with! DWARF is a nearly all-sparse attention architecture that uses 9 Dynamic Sparse Query-Gather (DSQG) layers as the backbone for transportation and a single full causal attention