r/LocalLLaMA
· Communities
Huawei opensouced openPangu-2.0-Pro, 505B-A18B
openPangu-2.0-Pro is an MoE model trained on Ascend. The model has 505B total parameters and 18B activated parameters. Its context length is 512k. The total pretraining data contains 34T tokens. During Post-training, openPangu-2.0-Pro is trained through unified SFT with slow and fast thinking capability, multiple speci