Naveen Rao on AI's Energy Wall and 4D Computing

A squirrel leaps between branches a thousand times out of a thousand, lands perfectly, and never files an error report. Its brain runs the whole performance on eight milliwatts, an amount so small that a single smartphone battery could power more than a hundred squirrel brains at once (07:58). A human brain, with its ability to write poetry and build spacecraft, needs only about twenty watts, roughly the draw of an old incandescent nightlight (07:17). Meanwhile, the data centers now teaching machines to write poetry of their own are approaching a hundred gigawatts worldwide and climbing. Somewhere in that gap between milliwatts and megawatts sits the argument that Naveen Rao, co-founder and CEO of the chip startup Unconventional AI, brought to the stage.

Rao is not new to long bets on hardware. In 2014, before AI was common conversation, he founded Nirvana Systems, one of the first AI chip companies, and later sold it to Intel, where he ran the AI group (02:10). After Intel, he built infrastructure for training large language models, a business that took off once ChatGPT arrived in 2022 and eventually merged into Databricks in 2023, where it now accounts for a quarter of that company's revenue (02:53). Unconventional AI is his next attempt to rebuild computing from the ground up. The company's stated goal is a thousandfold gain in power efficiency. Rao originally gave the project five years. He now says three and a half (03:40).

The urgency comes from arithmetic he laid out plainly. Google alone processes 3.2 quadrillion tokens a month (05:02). At a conservative ten joules per token, that works out to twelve gigawatts, just for one company's AI services, out of about forty gigawatts the entire United States currently feeds into data centers (05:27). Rao estimates the industry runs into a hard energy ceiling within about three years if demand and model size keep growing (05:50). The cost structure already reflects the strain.

"About fifty percent of the cost of serving a token, so every time you try something on ChatGPT, fifty percent of that cost is energy." — Naveen Rao [06:38]

That statistic reframes what a chip company is actually selling. Data centers used to be sold on floor space, then on networking gear, then on GPUs. Today, Rao argues, the conversation starts with a power contract, and everything else is built around filling it (06:19).

Biology is his proof that a better answer exists. The comparison he draws is not just about total wattage, it is about where that wattage goes. The human cortex, the wrinkled outer layer of the brain, moves about sixteen billion bits of information per second. A high-end GPU moves nearly thirty trillion bits in and out of memory every second, a figure that climbs another one hundred times higher once you look inside the chip itself (08:40). Moving information, not calculating with it, is what actually burns the electricity in a modern computer (08:18). That design goes back to the 1940s. The first electronic computer, ENIAC, was built to calculate artillery trajectories faster than a room full of humans with slide rules, and every computer since has mostly been sold on the same premise: faster than the alternative, never mind the power bill (09:20). Rao also points out a second wall closing in from the other direction. Moore's Law, the decades-long trend of packing more transistors onto a chip, has largely run its course, so shrinking transistors no longer buys the efficiency it once did (10:06).

Letting physics do the arithmetic

Rao's proposed fix starts with a blunt observation: even "digital," the one-and-zero logic every modern computer runs on, is itself a kind of translation layered on top of physical reality. A transistor does not naturally snap between exactly one and zero, engineers force it to behave that way, and each layer of that forcing wastes some energy (10:46). His team is trying to strip out a few of those layers and let the physics of the chip do more of the computing directly.

The intuition comes from dynamical systems theory, the study of how simple physical rules produce complex group behavior, the same principle behind a flock of birds turning as one or an ant colony solving problems with no central planner (11:44). Rao's favorite demonstration is a row of metronomes sitting on a plank that can roll slightly side to side. Set them ticking out of sync and, through nothing but the shared wobble of the plank, they will drift into perfect unison (12:04). Scale that idea up and you get a computer that does not execute instructions step by step so much as settle into a state, the way the metronomes settle into rhythm.

Unconventional AI first tested the idea in simulation, releasing an open source image-generation model called UNO built from networks of oscillators like those metronomes (13:05). Along the way the team found something counterintuitive: thinning out the connections between elements, a property they call sparsity, did not just save energy, it also made the system easier to train and scale (15:06). Then, this year, theory became metal.

"This is actually the first physical dynamical computer ever built. We did this in five months." — Naveen Rao [15:27]

The company started in earnest in January, taped out its first chip design in June, and by the time Rao spoke had working silicon generating images in the lab (15:27). The headline number: about 500 nanojoules per image, against roughly a millijoule on a standard GPU, an efficiency gain the company puts at around a thousandfold (16:22). Rao calls the underlying architecture 4D computing. Instead of separating memory and computation, the way every von Neumann machine has since the 1940s, each computing element in the new chip also stores its own memory, so there is no memory interface to shuttle bits across. Three physical dimensions of stacked circuitry, plus the fourth dimension of time-based dynamics, replace the back-and-forth movement that eats most of a modern chip's power budget (16:46).

The ambition behind the number is bigger than one prototype. Rao frames the whole project as a race against a thermodynamic limit, a theoretical ceiling on how much intelligence any physical system can produce per watt. Animal brains sit within one or two orders of magnitude of that ceiling. Today's computers, he says, sit about ten billion times away from it (18:07). Hitting the limits of two-dimensional chip fabrication within three and a half years is, in his words, an attempt to beat biology outright (18:33). If it works, he expects computing to shift away from a handful of gigawatt-scale data centers and toward many small, local ones, plus new possibilities like swarms of power-sipping robots (18:33). He is careful not to promise that cheaper AI means less energy use overall. He invokes Jevons paradox, the historical pattern in which making a resource cheaper causes total consumption to rise by more than the price drop, not less.

"When you drop the underlying cost of an asset, you actually consume more than the drop of that asset." — Naveen Rao [19:19]

None of this matters if nobody can use the chip. Rao says a full product, a rack that plugs into existing data centers with tokens going in and out over a network cable, is roughly two years away (20:05). Existing AI models will run on it, but the team ports them at the level of the model itself rather than at the level of individual operations, which still requires real engineering work (21:07). The chip does not implement matrix multiplication, the core arithmetic behind most neural networks, in the literal sense, though its behavior can be described that way; internally, it runs as a time-varying physical process (21:28). Instead of CUDA, the software layer Nvidia built to program GPUs, Unconventional AI has written its own libraries in Python to express these time-based, probabilistic behaviors (22:04). Rao says the hardest organizational problem has been getting theorists trained in dynamical systems to actually talk to the engineers who build chips, two groups that, by his account, do not naturally speak the same language (21:48).

Whether Unconventional AI hits its own timeline is a bet, not a fact. But the diagnosis underneath it, that computers waste most of their energy moving information rather than transforming it, is hard to argue with once you have watched a squirrel do more with eight milliwatts than a data center does with twelve gigawatts. One line from partway through Rao's presentation was less a slogan than a job description: he does not feel he understands something until he can build it (07:58). A chip that finished its first working image just months after the company existed suggests he means it literally.