{"id":64443,"date":"2026-07-23T10:17:42","date_gmt":"2026-07-23T14:17:42","guid":{"rendered":"https:\/\/overcentral.com\/en\/?p=64443"},"modified":"2026-07-23T10:17:42","modified_gmt":"2026-07-23T14:17:42","slug":"kimi-k3-distillation-experts","status":"publish","type":"post","link":"https:\/\/overcentral.com\/en\/kimi-k3-distillation-experts\/","title":{"rendered":"Experts Reject Distillation as Key to Kimi K3’s Success"},"content":{"rendered":"

The White House has directly accused Chinese AI<\/a> company Moonshot<\/a> of stealing US technology to build its open-weight model Kimi K3<\/a>, but technical experts are pushing back against the central claim that model distillation alone explains the model’s advanced capabilities. The controversy, which has reignited debates over export controls and the geopolitics of AI development, centers on the question of how a Chinese lab could produce a frontier-class model so quickly after the release of Anthropic’s Fable LLM.<\/p>\n

White House science advisor Michael Kratsios stated that Moonshot built Kimi K3 by copying Anthropic’s Fable while using Nvidia chips that are not cleared for export to China. “Large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable,” Kratsios wrote, amid ongoing discussions about potentially banning Chinese open-weight models. Treasury Secretary Scott Bessent echoed the sentiment, claiming that watermarks of US large language models have been found on many Chinese models, though no details about what those watermarks consist of have been publicly shared.<\/p>\n

However, researchers who have examined the technical landscape surrounding Kimi K3 are skeptical that distillation \u2014 the process of systematically querying a model to extract its capabilities and replicate them \u2014 is the primary driver of the model’s performance.<\/p>\n

Why Experts Doubt Distillation Explains Kimi K3’s Capabilities<\/h2>\n

Braden Hancock, a researcher at the Laude Institute and co-founder of Snorkel AI, dismissed the idea that Kimi K3 could have been built through distillation alone within the available timeframe. “I don’t think you get a model this strong and this quickly on the heels of Fable doing strictly distillation,” Hancock said. “There’s just not even frankly time, right? Fable’s only been publicly available since July 1st. You can’t distill that much data, train a model, and release it in two weeks.”<\/p>\n

Nathan Lambert, an AI researcher at the Allen Institute for AI, argued that distillation is becoming less impactful as Chinese models approach the frontier and the training paradigm shifts toward reinforcement learning. “If it were the case, everyone would be easily able to catch up to a GLM or to a K3 by using its data for distillation. But we have not, or we won’t see this, from supervised fine-tuning alone,” Lambert said.<\/p>\n

What Distillation Actually Entails \u2014 and Why It May Not Be Enough<\/h2>\n

Distillation requires a lab to systematically query a target model to generate data for post-training. This can involve asking the model to articulate its chain-of-thought reasoning or using its prompts and responses to train a new model through supervised fine-tuning (SFT). It is this fine-tuning process that can produce a model that closely mimics the behavior of the original \u2014 in Lambert’s view, where the “model picks up its manners.”<\/p>\n

But Lambert emphasized that the benefits of SFT are diminishing as models grow more complex. Replicating capabilities on the level of Fable would likely require reinforcement learning techniques, where an agent of the larger model grades the smaller model’s responses and iteratively adjusts based on those grades. The infrastructure demands are significant: large reinforcement learning runs can require tens of millions of agents. Using a frontier lab’s API at that scale “would be insanely expensive and potentially it would probably be a time bottleneck because these models are pretty slow and to be frank might not even give you a performance uplift,” Lambert noted.<\/p>\n

Previous Distillation Accusations and Industry Context<\/h2>\n

This is not the first time Anthropic<\/a> has accused Chinese labs of systematic distillation. Earlier this year, Anthropic publicly called out Moonshot, DeepSeek, and MiniMax, stating it had discovered millions of queries from those companies that were “distinct from normal usage patterns, reflecting deliberate capability extraction rather than legitimate use.”<\/p>\n

Yet distillation is far from a China-only practice. Elon Musk<\/a> testified earlier this year that SpaceXAI distilled OpenAI models to develop Grok, and described the practice as common across the industry. The line between distillation and developing synthetic datasets remains blurry, making it difficult to police without broader industry standards.<\/p>\n

The Technical Expertise of Chinese AI Teams<\/h2>\n

Hancock cautioned against underestimating the capabilities of Chinese AI researchers. “In general, Americans are understating the technical expertise of these Chinese teams. One of the founders of Moonshot was a CMU PhD student. These are legitimate researchers and engineers doing solid work,” he said. “If American models ground to a halt, I think China’s progress would slow, but would still continue. They’re not just riding coattails here.”<\/p>\n

Export Controls and the Chip Question<\/h2>\n

The second part of Kratsios’ allegation \u2014 that Moonshot obtained advanced Nvidia Grace Blackwell 300 chips and accessed GB300-equipped servers in Thailand \u2014 introduces a separate but equally contentious dimension. These chips are banned from export to China, but a black market exists, according to Sam Bresnick, a research fellow at Georgetown’s Center for Security and Emerging Technology. In May, the founder of Supermicro, a US server builder, was indicted for smuggling advanced chips into China.<\/p>\n

Bresnick advocated for know-your-customer laws for data centers globally. “If you are letting a company conduct huge training runs on your state-of-the-art hardware, there needs to be a reporting mechanism for who that company is and what they’re doing,” he said.<\/p>\n

The Biden administration proposed federal know-your-customer rules for data centers in 2024, but no further progress has been made under the Trump administration. Currently, exporters shipping advanced chips abroad are supposed to ensure they are used only for approved purposes, but enforcement remains uneven.<\/p>\n

What This Means for the AI Landscape<\/h2>\n

The Kimi K3 controversy illustrates the growing tension between US export control policy and the technical reality of AI development. The evidence suggests that Chinese labs are making genuine advances through their own research and engineering, not merely copying US models. Distillation may play a role in some cases, but experts are clear that it cannot explain the rapid emergence of a model as capable as Kimi K3.<\/p>\n

For professionals and developers evaluating open-weight models, the key takeaway is that the landscape is becoming genuinely multipolar. Chinese models are approaching frontier capability through independent technical progress, and the policy response will need to reckon with that reality rather than attributing success to theft alone. Monitoring the evolving regulatory environment around export controls and data center oversight will be critical for anyone building on or deploying models that may be subject to shifting geopolitical restrictions.<\/p>\n","protected":false},"excerpt":{"rendered":"

The White House has directly accused Chinese AI company Moonshot of stealing US technology to build its open-weight model Kimi K3, but technical experts are pushing back against the central claim that model distillation alone explains the model’s advanced capabilities. The controversy, which has reignited debates over export controls and the geopolitics of AI development, […]<\/p>\n","protected":false},"author":7,"featured_media":84282,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/cards.overcentral.com\/cards\/en\/64443.png","fifu_image_alt":"Experts Reject Distillation as Key to Kimi K3's Success","footnotes":""},"categories":[349],"tags":[],"class_list":["post-64443","post","type-post","status-publish","format-standard","has-post-thumbnail","category-articles"],"fifu_image_url":"https:\/\/cards.overcentral.com\/cards\/en\/64443.png","fifu_image_alt":"Experts Reject Distillation as Key to Kimi K3's Success","_links":{"self":[{"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/posts\/64443","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/comments?post=64443"}],"version-history":[{"count":0,"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/posts\/64443\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/media\/84282"}],"wp:attachment":[{"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/media?parent=64443"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/categories?post=64443"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/overcentral.com\/en\/wp-json\/wp\/v2\/tags?post=64443"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}