X · @teortaxesTex
· X / Twitter
> and ~9.40 ExaFLOPS for the fastest cluster (31B) That's bf16, but TPU v6e doesn't support fp8 anyway all in all, this is like 2 Huawei SuperPODs, ex…
> and ~9.40 ExaFLOPS for the fastest cluster (31B)That's bf16, but TPU v6e doesn't support fp8 anywayall in all, this is like 2 Huawei SuperPODs, except 32 GB per TPU and not 96/144Ironwood is vastly more capabletesttm: its interesting to see how large the TPUs Pods are for even Gemma E2B! thats about ~3.76 ExaFLOPS of