r/LocalLLaMA
· Communities
Today was the perfect day for Poolside to drop Laguna S 2.1 because I just got these in! Finally have a half decent amount of VRAM. 3x V620 = 96 GB.
Laguna is the first model I'm trying, Q4_K_M fits with 256K context @ F16. Doing the html flight simulator test now. These cards are getting 400 to 600 tok/s prefill and 16 to 20 tok/s gen so far (I have NOT enabled dflash yet). Not bad at all for the cost. ($350 each) In a Dell PowerEdge R740 with dual Xeon Gold 6248R