A glimpse of the future of AI hardware has come with the unveiling of a prototype AI chip that is said to be more than a dozen times more energy efficient than current industry-leading digital AI.
The secret of the chip is to return to the oldest form of computing, analogue computing, which carries out calculations by harnessing continuously-varying properties of the real world – whether by manipulating light, voltages, distances, the properties of materials, volumes of water or the rotation of gears. Perhaps the oldest surviving example of an analogue computer is the 2000-year-old Antikythera mechanism.
In the mid-twentieth century, digital computers began to take over from analogue and, by the late 1970s had displaced them.
However, digital computing, for all its advantages, rests on manipulating binary numbers and, because these are rounded, there are concerns about the fidelity with which digital computers can simulate reality, while digital processes are also hugely power hungry because of the energy and time invested in shuffling data between their processors and memory.
As a result, the costs involved in training AI models are expected to skyrocket as the technology becomes more ubiquitous. ChatGPT, for example, costs more than $700,000 per day to run, according to one estimate, others talk about how it requires data centres that generates ‘thousands of tonnes of carbon emissions and cost millions of dollars’.
The development of AI will stall if it relies on existing digital hardware alone but analogue computers now raise hope of a sustainable AI future, and a way to boost computer power to deal with computationally demanding uses, such as creating virtual organs and people.
Previous simulations from IBM suggested that analogue AI could prove 40 to 140 times as energy efficient as the best Graphics Processing Units, the descendants of gaming chips optimised for rendering graphics that are now widely used in AI.
The new advance, by Stefano Ambrogio and colleagues at IBM Research, relies on using a prototype analogue chip, which was described a few days ago in the journal Nature. The development of the prototype dates back to 2021, when a team at IBM developed chips that use phase-change memory (PCM), which works when an electrical pulse is applied to a material, which changes the conductance of the device.
The phase in question refers to the state of matter in the chip, which changes from disordered (amorphous) to ordered (crystalline): a lower electrical pulse will make the device more crystalline, providing less resistance, and a higher electrical pulse makes it more amorphous, resulting in high resistance.
Instead of recording the usual 0s or 1s, typical of a digital microchip, the prototype IBM device records its state as a continuum of values between the amorphous and crystalline states. This value is called a synaptic weight, alluding to how modern AI is loosely based on the architecture of the brain in what are called deep neural networks. In all the new analogue chip contains 35 million phase-change memory devices.
The team at IBM Research devised two tests to show their analogue chip is more efficient: one, akin to awaking a smart speaker, showed that the analogue chip could react to 12 key words with the same accuracy as the software-based systems that exist today — but considerably faster; in the second experiment, transcribing audio of people speaking, the IBM prototype was estimated to be roughly 14 times more performant per watt — or efficient — than comparable digital systems.
Overall, the chip can carry out 12.4 trillion operations per second for each watt of power, an efficiency that is tens or even hundreds of times higher than for the most powerful GPUs. The team has also shown the potential of the approach for other classic AI tasks – computer vision and image recognition and says the performance marks ‘the first demonstrations of commercially relevant accuracy levels’ using analogue AI.
However, IBM says: ‘while this work is a large step forward for analogue AI systems, there is still much work to be done before we could see machines containing these sorts of devices on the market.’
In a Nature commentary, Henchen Wang of Intel adds that to unleash the full potential of the new chip, which offers ‘astonishing efficiency’, would also require the development of customised compilers, which are the programs that convert instructions into a form that can be read and executed by a computer. tailored algorithms and dedicated applications.
