Want to look straight into the future, or Sauron's eye? Look at Cerebras' CS-4

A human reads at about 4–5 tokens per second. ChatGPT gives you about 50, and OpenAI’s new Ultrafast gives up to 750. The latter is lightning fast, with one of the world’s smartest models. 150 times faster than you read, and definitely faster than you can program.

CS-4? 4,400 (!) tokens per second. Granted, that is with a smaller model, but Cerebras aims for the next generation, CS-5, to reach 10,000 tokens per second with mid-sized models next year, and 5,000 with frontier models. These are speeds that are hard to grasp. And somewhere along the way it stops being you who is the reader. It is another machine.

That opens up many possibilities. Software that lets you try ideas like sketches: you say “show me five ways this function could work”. It writes five runnable variants. You point to one, and it keeps building while you explain why. The value is that many cheap attempts fit into the conversation. Or game worlds that respond to what you did. Not just an NPC with more lines: the world can continuously write consequences, dialogue and small quests based on the player’s actions. And so on.

That was the nice part. The less nice part is what a frontier model at this speed can do. A swarm of 10,000 capable agents after your systems is bad enough. But the number is not what is new. What is new is that each individual agent gets a hundred times more reasoning steps before you have realised something is happening.

Unfortunately I do not think a human stands much of a chance in that matchup. The OODA loop (observe, orient, decide, act) has become too tight to fit Homo sapiens. Our place will be above the loop, not in it: we set the boundaries in advance, and then it is machine against machine.

These machines are not cheap and they are few, but they are coming. And they are really just one example. The relentless march toward ever more compute shows no sign of pausing in the foreseeable future.

https://lnkd.in/eicGKPGx