Electronics Weekly Electronics Design & Components Tech News
nigel toon
He writes:
When Electronics Weekly first rolled off the presses 60 years ago, the mosfet transistor was as new as Graphcore’s IPU [intelligent processing unit] processor is today.
To see the mosfet was to understand its function. Its three pins effectively described its behaviour. Even if the science that made it possible eluded you, the relationship between input, process and output was easy to comprehend.
A decade and a half later, despite packing more than 3,000 transistors into a single chip, the MOS 6502 microprocessor was still recognisable by a pin‑out chart that adorned workbenches the world over. You could even track down a computational error by scanning through the machine code and checking at the outputs.
By 1985, Intel’s 80386 had 132 pins, and it was no longer possible to understand what was going on inside without a deep knowledge and the correct debug equipment close to hand.
Yet the computing that these chips were executing remained knowable. Programming languages and other abstractions placed us miles from the metal – but if needs be, you could still dump the core and piece together an input-process-output.
Now, with artificial intelligence (AI) we no longer tell the computer what to do – it learns from data. The inner life of an AI processor is as opaque as its outer shell. And yes, I’m conscious of the irony here, coming from a person whose company designs and builds the most complicated processors ever made, specifically designed for AI.
Of course, we know the layout of the silicon. The state of the logic in our IPU’s 1,216 separate processor cores could be gleaned at any point in time. But computing is about how numbers move through the silicon and how they interact mathematically.
The scale and nature of AI compute means that your chances of understanding the functioning of a model by examining the state of a chip’s registers is as fruitless as trying to work out someone’s hopes and dreams by staring at a slice of their brain.
Today we see AI models with billions of parameters, each exerting a subtle influence on combinations of the other parameters and eventually, through layers and layers of parameter interactions, probabilistically influencing the output.
The relationship between input and output in AI is hopelessly elusive, even the smartest AI researchers don’t know exactly how it works.
So what is left for electronic engineering? The most exciting challenge of all, I would argue – to build machines so sophisticated that they can accurately compute the process in problems where we can only hope to comprehend the input and the output.
When Alan Turing addressed the London Mathematical Society in 1947, he hinted – perhaps for the first time in public – at the vision he was nurturing: “What we want is a machine that can learn from experience.”
Sixty years after the first mosfet transistors, you can hold such a device in your hand. And a few of us lucky people get to make these machines that do the unknowable compute.
