Character.AI
· Frontier Labs
Optimizing Large-Scale Pretraining at Character.ai
Before Character.ai shifted its focus toward building on open-source model foundations, the company’s early pretraining team explored a range of techniques to make large-scale transformer training faster and more efficient. That work - led in part by our cofounder Noam Shazeer - is now being shared publicly