r/LocalLLaMA
· Communities
FYI You dont need expensive networking for multi-node gpu. 30t/s laguna Q2_K_XL (39.7GB) on 2×4060+1×4060 using a $20 usb->ethernet.
Turns out a regular ethernet cable between 2 nodes can run laguna UD-Q2_K_XL (39.7GB) using a direct point to point network. Interestingly on `nvidia-smi dmon -s pucvmet -d 2`, the inter/intra gpu traffic is not really capped in this setup. ubatch-size = 768 [58055] 2.53.851.946 I slot print_timing: id 0 | task 0 | pro