X · @teortaxesTex
· X / Twitter
RT changyi yang: Weight sync in RL post-training looks RDMA-bound. But >99% of BF16 weights are byte-identical step to step — ship only the delta: ~1…
RT changyi yangWeight sync in RL post-training looks RDMA-bound. But >99% of BF16 weights are byte-identical step to step — ship only the delta: ~100–1000× less traffic, bit-exact. Hardware bottleneck → software problem: RL over commodity ethernet.🧵 http://changyi.fun/posts/delta-weight-sync/