Core Takeaway: ByteDance is training an AI model with up to 10 trillion parameters, more than three times the size of China’s largest existing model and potentially surpassing Anthropic’s Mythos system. The effort, supported by an estimated $2.5 billion in GPU infrastructure and a Seed research team of roughly 2,000 staff, signals China’s most aggressive bid yet to close the frontier gap with U.S. labs. However, success is far from guaranteed: the model remains in early pre-training, industry research shows parameter growth in frontier models is slowing, and even 10 trillion parameters cannot ensure superior capability without breakthroughs in data quality and training efficiency.
The Scale of the Bet
In August 2026, the Financial Times reported that ByteDance, the TikTok parent company, was in the early stages of training a large language model with parameters reaching as high as 10 trillion. For context, Moonshot AI‘s Kimi K3—currently the world’s largest open-weight model—has 2.8 trillion parameters, meaning ByteDance’s target represents more than a threefold increase over the previous Chinese record. Industry estimates place Anthropic’s most advanced Mythos 5 at approximately 8 trillion parameters, suggesting ByteDance is aiming directly at the frontier.
The training infrastructure required for this undertaking is staggering. According to industry analysis, ByteDance is renting approximately 30,000 Nvidia Blackwell GPUs—roughly 500 GB200 racks—at an estimated cost of $2.5 billion, with pre-training expected to consume between 2×10²⁵ and 1.2×10²⁶ FLOPs over three to six months. The effort is led by ByteDance’s Seed research team, which numbers around 2,000 staff across China and overseas, under the leadership of former Google DeepMind scientist Wu Yonghui.
Why ByteDance Is Going Big
ByteDance’s decision to pursue a 10-trillion-parameter model reflects both competitive necessity and strategic ambition. The company’s flagship consumer AI product, Doubao, has reached 324 million monthly active users, providing a massive distribution channel that demands a frontier-level model to sustain. Unlike many Chinese competitors that have relied on open-source strategies, ByteDance has kept its models largely proprietary, betting that genuine capability—not imitation—will determine long-term leadership.
Founder Zhang Yiming has reportedly instructed the Seed team to avoid relying on knowledge distillation from other labs’ models, pushing for fully independent development. This directive carries strategic weight: U.S. labs have accused Chinese firms of distilling their models, and Zhang has made clear that ByteDance’s path to the frontier must be built on original research rather than shortcuts.
A Three-Way Chinese Arms Race
ByteDance is not alone in pursuing trillion-parameter scale. At Alibaba’s Yunqi Conference in September 2026, CEO Wu Yongming announced that Alibaba is preparing to train models with parameters between 5 trillion and 10 trillion. DeepSeek, meanwhile, is training a roughly 2-trillion-parameter model and has signaled plans to push toward 8 trillion parameters. The collective resurgence of the “larger is better” philosophy reflects the continued influence of the Scaling Law, which holds that expanding model parameters, data, and compute simultaneously produces predictable gains in capability.
The Technical and Structural Challenges
The path to 10 trillion parameters is fraught with difficulty. Training stability at that scale, data quality and mixture, and model convergence remain open research problems. Huawei’s Wang Tao has noted that in 100,000-chip clusters, inter-chip communication costs can exceed 40% of total training time—a bottleneck that grows more severe with model size.
U.S. export controls further complicate the picture. ByteDance has worked around these restrictions by accessing advanced Nvidia chips through cloud partners in Malaysia, but the arrangement highlights the structural vulnerability at the heart of China’s frontier AI ambitions. The company’s 2026 AI infrastructure budget has grown 25% to approximately 200 billion yuan ($29.4 billion), with overall capital spending potentially reaching as high as $70 billion.
The Diminishing Returns Question
Despite the massive investment, broader industry trends raise questions about whether a 10-trillion-parameter model will deliver proportional returns. Research from Omdia indicates that growth in the largest frontier AI models has slowed sharply, while smaller and midsized models are expanding in both capability and adoption. MIT FutureTech research has similarly found that as models grow, performance gains from additional computing power begin to fade.
ByteDance’s pre-training run is expected to complete in early-to-mid 2027, with fine-tuning and release following afterward. Whether the result justifies the investment will depend not on parameter count alone, but on whether ByteDance can solve the harder problems of data curation, training efficiency, and architectural innovation that separate a large model from a capable one.
