r/LocalLLaMA
· Communities
4x2080Ti 22GB tensor parallel possible?
Hi there, I have a setup with 4x 2080Ti 22GB, but am not able to see much benefits in running models larger than 16-20GB in speed. I did some research and saw something like tensor parallels are possible, but also that because of flash attention only being possible with 30 series and up, it’s not worth it for 2080 Ti.