BAAI unveils Wu Dao 2.0, a 1.75-trillion-parameter model

The Beijing Academy of Artificial Intelligence unveiled Wu Dao 2.0 on 1 June 2021, a multimodal pretrained model it said used 1.75 trillion parameters, about ten times the count in GPT-3. BAAI said it trained on 1.2 terabytes of English and Chinese text.

Why it mattered As announced it was the largest pretrained model yet built, evidence that training at that scale was no longer confined to laboratories in the United States.