The Chips that Ate the World
Generative AI as we know it is due almost entirely to one video game: Quake III: Arena.
Without Quake III, and in particular John Carmack's obsessive search for the perfect hardware that would squeeze even better performance out of the silicon available on graphics cards in 1998, it's unlikely that NVIDIA would have pursued parallel computing and so made the leaps in performance that - while almost bankrupting the company at the turn of the millennium, also paved the way for the kind of exponential gains that finally made the promises of machine learning and neural networks possible in the twenty-first century. As Stephen Witt observes in The Thinking Machine: Jensen Huang and Nvidia, the company shaping the future of AI, even Huang, the strangely charismatic CEO of the company, was sceptical at first, but the ability to run multiple shaders on one chip (unlike their main competitor at the time, 3dfx, who used seperate chips on one board for each type of computation) essentially blew apart Moore's Law.
But in pursuing this goal, Nvidia engineers were playing a far more dangerous game than even they realized—for inside the TNT was a secret, a lurking, demonic secret buried so deep in the architecture of the silicon that neither Huang nor Kirk nor Carmack nor Vivoli nor anyone else in the world suspected that it was there. If you popped the cover off that TNT chip and inspected the naked circuitry with a microscope, you would find a change not just in the arrangement of transistors but a change to all computers, and perhaps to all of humanity, forever. Inside that tiny chip was a secret that would change the world.
While this paragraph tends towards the melodramatic, Witt is probably right: this was the secret that would change the world, and one of the virtues of this book is that the author doesn't hold back (too much) from the technical elements of hardware and software engineering that made the Riva TNT (or "Twin Texels", referring to the dual-pixel rendering pipelines in NVIDIA's graphics card) and its successors so important. That he manages to do so in this engaging book is also very much to be commended: unlike many other authors writing about tech, he doesn't feel the need to pad out his narrative, compressing a great deal of information into its 248 pages. Occasionally this means sometimes important detail is lost - most notably when he avoids the geopolitics of the company's investments in Israel following the 7 October attack. This is one of the areas where Witt is a little too nimble for his own good, and the book occasionally carries the whiff of PR spin, but on the whole it shows off those talents that were amply demonstrated in his previous title, How Music Got Free.
This, then, is as much a book about a hardware company as it is about a man, and two things stand out in this regard: first - and most expected in terms of narrative structure - is the rise, near fall, and rise and rise again of NVIDIA. While the perils the company faced are doubtless elaborated more than a little to provide some excitement, it is also very much the case that in the early 2000s no-one saw this coming, as evidenced by NVIDIA's being constantly in the doldrums as far as its investors were concerned. What struck me more, however, was the subplot involving Intel. Once the leading semiconductor manufacturer in the world, its current CEO, Lip-Bu Tan, admitted this year that it wasn't even in the top 10 of chip companies (and barely scraped into the top 200 of all companies). By contrast, the toy for geeks was the first company to (briefly) be worth more than $4 trillion dollars as AI companies couldn't get enough of its product.
Super Normal?
Yet the rise of NVIDIA is only half the story. While the company had three co-founders - Chris Malachowsky and Curtis Priem as well as Jensen Huang - it is Huang who is, quite literally, the face of the richest company today.
Huang tells the author that he is "super normal", which is patently not true. He is as much a product of an American tech-founder culture as Steve Jobs, Bill Gates or, for better or for worse, Bill Gates. He is also, it must be said, somewhat elusive in this book: while Witt was clearly given a great deal of access to various figures at NVIDIA, Huang's time seemed to be carefully metered out. This is only to be expected from the man who leads the most successful company in the world, but it also begins to feel a little too much like a literary conceit. In the early pages, dealing with what is, frankly, an uneventful life - Huang starts out at a regular university, marries young, provides for a family and works incredibly hard - I was charmed. Until I wasn't. The final encounter between Huang and Witt is intended as some kind of shock therapy where we learn deep and meaningful things about intensely the intensely pressurised atmosphere of billionaire tech companies, but my main takeaway at that particular moment was how much of an asshole Huang could be.
“This cannot be a ridiculous sci-fi story,” he said. He gestured to his frozen PR reps at the end of the table. “Do you guys understand? I didn’t grow up on a bunch of sci-fi stories, and this is not a sci-fi movie. These are serious people doing serious work!” he said. “This is not a freaking joke! This is not a repeat of Arthur C. Clarke. I didn’t read his fucking books. I don’t care about those books! It’s not—we’re not a sci-fi repeat! This company is not a manifestation of Star Trek! We are not doing those things! We are serious people, doing serious work. And—it’s just a serious company, and I’m a serious person, just doing serious work.”
For the next twenty minutes, in a tone that alternated among accusatory, exasperated, and belittling, Jensen questioned my professionalism, questioned my interview approach, questioned my dedication to the project. He accused me of trying to psychoanalyze him; he told me how much he disliked answering my biographical questions, especially those that attempted to illuminate his mental state. “I don’t like these probing questions,” he said. “I don’t like talking about myself, OK?! I’m not into therapy.” He suggested that my questions were stupid; he called them “pedestrian.” He denied there was anything exceptional about himself, against all accumulated evidence. “Look, I’m—I am super normal,” he said.
I'll admit to feeling some sympathy with regard to Huang's comments at this time - not being taken seriously as a company when NVIDIA had done as much as any other to bring into being generative AI must sting, but Huang's chewing out of Witt - and colleagues, as happens plenty of times throughout this book - was infuriating. When I had finished the book, it left a very sour taste.
And yet... and yet... Witt's attempt to turn this into a learning experience is glib, but having pondered The Thinking Machine for a few days I am genuinely impressed by several things. His early life in Taiwan seems to have centred him, and while he graduated from Stanford, his earlier academic career in Kentucky also seems to have kept his feet on the ground. Unlike many a tech bro, he seems to have been genuinely and happily married to Lori, with loving children (something of a rarity in the field), all while keeping their private lives private. Most impressive in terms of his abilities as a CEO, however, is the way in which he seems to inspire genuine loyalty, something much more significant than any irritation caused by his bawling out of employees:
There was also the topic of loyalty. Musk did not value it; he often fired people arbitrarily and without warning, in one case canning the entire Starlink engineering team almost at random on a Sunday afternoon. Huang almost never fired anyone, and when he did, it was only after multiple cautions and the offer of a performance-improvement plan. It took truly egregious behavior to get kicked out of Nvidia, and many employees worked there for decades, including boomerang hires like Catanzaro and Aarts. Even when operating economics forced Huang to shutter a division, he reassigned employees to other useful tasks. In 2019 Curtis Priem returned to Nvidia’s offices for the first time in sixteen years to join Huang and Malachowsky for a reunion of the company’s founders. “I was astounded at how many people were still there,” he said. “Jeff Fisher, his kids were working for Nvidia.”
As we enter an age where most business leaders seem most eager to lay off as many of their staff and replace them with AI, this strikes me as the most important lesson to learn from one of the leading figures of the AI revolution.