The high-water mark

This weekend China’s Moonshot released the weights of Kimi K3, the most powerful open AI model ever published. 2.8 trillion parameters, freely downloadable. If you want AI in a box, you can have it now. The box is admittedly a rack rather than a gaming PC; the weights alone come to 1.4 terabytes.

The map then looks roughly like this: the US has the best labs and models, closed. China has the second-best labs and the best open models. Europe has… what?

The interesting question is what this means going forward. What happens when ever stronger models are openly available to anyone with enough hardware?

Cyber is the most obvious. Kimi K3 finds real, unknown vulnerabilities and carries out complete exploits, according to Moonshot’s own report, which calls the numbers a lower bound. But note that open weights arm both sides: Hugging Face could only analyse the attack against them because there was an open model to run locally. The floor rises for attacker and defender at the same time. Where the net balance lands, nobody knows.

Then you have labs and companies that get to experiment with models with almost no restrictions. “Almost” because the filters sit in the weights, and you own the weights. For ordinary users the guardrails are real. For capable actors they are cosmetic.

And I suspect there will be side effects nobody has a name for yet. Unfiltered intelligence at this level has never been generally available before. The second-order effects are by definition unknown; even the lab doesn’t know what the model can do.

At the same time, this could dry up almost at any moment. This summer the US imposed export controls on a frontier model for the first time. Now the FT and Reuters report that China is considering restricting foreign downloads of Chinese model weights. Nothing has been decided. But the direction is the same in both capitals. (I downloaded a couple of smaller open models myself tonight. Probably unnecessary.)

Models that have been released are out forever. But perhaps the ones we have now are the best we will ever get for free. We could probably ask ourselves that every week; but making a free model is not free, and it has to fit into a strategy.

You only see the high-water mark once the water has receded.