AI chips and gadgets: The role of AI hardware in banking

A new technology, even if it only brings small changes at first, always has revolutionary potential. This is because, in addition to its actual purpose, it usually has other effects on our world. Benedict Evans, consultant and former investor at the renowned VC Andreessen Horowitz, illustrates this phenomenon of “second order consequences” using the example of the electric car. With its increasing popularity, the entire industry is changing. This goes far beyond the comparison of batteries with combustion engines. The change massively affects the entire supply chain, from components and workshops to the infrastructure of charging stations and the energy industry, including the petroleum industry. Just as the invention of the automobile not only enabled people to get from A to B faster, but also gave rise to new supply chains.

Artificial intelligence will bring with it changes that are almost impossible to predict. The second order consequences are already in full swing: Generative AI consumes enormous amounts of energy and millions of liters of water, mainly for calculations, data storage and cooling of devices in data centers. According to the International Energy Agency, the power consumption of data centers is expected to reach 1,000 terawatts in 2026, which is roughly equivalent to the total consumption of Japan, reports Trendingtopics (Link to German publication). 

Progress in the field of AI is also having a significant impact on the hardware sector. One of the biggest beneficiaries of the AI boom is currently the US chip manufacturer Nvidia. The company produces so-called “AI chips”, which form the basis for the functioning of the neural networks of generative artificial intelligence. In addition to the effects of AI development, there are also so-called third order consequences: the manufacture of components for such chips requires corresponding production, and there have been bottlenecks in the semiconductor sector in particular for years. Research and development as well as the establishment of production capacities and supply chains for chips require considerable financial investment, in which the international financial industry is already heavily involved. One example is the UK’s state-owned Infrastructure Bank, which recently invested more than 70 million euros in a parts manufacturer to optimize the chip supply chain in the UK.

The unique feature of Nvidia’s AI chips is their design: they are so-called CPUs (Central Processing Units), which are specially optimized for AI applications. In contrast to the conventional GPUs (Graphics Processing Units), they offer many times more computing power and energy efficiency. They can perform many similar calculations at the same time so that an AI can learn and make decisions faster. In other words: without CPUs, there would be no ChatGPT and similar applications. This is because the old GPUs simply cannot deliver the speed required for generative AI applications.

The rising demand for AI chips recently brought Nvidia record margins of 62%. Last year in particular, the company left the former top dog among chip manufacturers, Intel, far behind. According to the consulting firm Omdia, 70 percent of the AI chips currently in use come from Nvidia. Despite the high prices, the company warns that it will not be able to meet the entire demand. “We expect the supply of our next generation of products to be limited as demand far outstrips supply,” said Nvidia’s CFO, Colette Kress, according to a report by Heise (German).

It is clear that the tech giants all want to become independent sooner rather than later: Google, Meta and Microsoft are currently investing billions in chip development. OpenAI boss Sam Altman is also reportedly in talks to raise seven billion US dollars for his own chip factories.

Numerous new types of chips are currently being developed, and their manufacturers are throwing superlatives around: the start-up Groq calls its supposedly ten times faster AI chips “LPU Interference Machine” (LPU = Language Processing Unit). In Munich, the start-up Semron has just raised the equivalent of 7.3 million euros – for its so-called “memcapacitor chips”, whose energy efficiency is said to be 35 to 300 times better than that of other chips. Or the latest AI chip from Cerebras, whose performance goes hand in hand with its enormous size: the AI chip from NVIDIA is 57 times larger. 

Companies that rely on AI in their products or plan to do so in the future should consider the possible second- and third-order consequences. For example, if a financial institution is considering integrating generative AI into its products or business processes, it is crucial to first create a suitable infrastructure for this. When building up the corresponding capacities, the question arises – as with many digitalization projects: do it yourself or buy it externally? Make or buy?

Make: A bank builds up the necessary capacities itself – on premise. Or buy: it relies on external vendors such as Google, Microsoft or others and obtains the computing services from the cloud. Either way, the future second and third order consequences should be considered. Specifically: the scarcity of AI chips. 

In practice, this often proves to be more difficult than expected. A recent survey of 100 IT buyers (German) revealed that two thirds of them were not fully prepared for the massive energy requirements of AI when they expanded their AI capacities.

There is also the question of whether and how AI services can be integrated into banks’ existing IT systems. Banks that have already ventured into the cloud and are working with external providers could benefit from this. However, the downside of the speed advantage is the dependency on external providers in terms of available capacity and pricing.

However, make-or-buy decisions are dynamic and subject to regular review. This can often be observed in connection with Software-as-a-Service: More and more companies are deciding to “leave the cloud” and switch to on-premise services.

Many fintech start-ups that initially used external Banking-as-a-Service or Software-as-a-Service services switch to internal solutions as soon as they reach a certain size. Vivid, for example, recently left BaaS provider Solaris (German).

In the area of AI developments, this comparison is similar to the fundamental digitalization efforts of banks: it is possible that start-ups have significant advantages over incumbent banks simply due to their more flexible structures and lack of legacy IT systems.

There is another consequence of the AI boom on the horizon: AI gadgets that are not based on apps like smartphones, but perform tasks with the help of AI and are controlled entirely via voice commands or gestures. AI thus opens up additional possibilities for human-computer interaction. Innovative AI gadgets of the very first generation were presented at the world’s largest technology trade fair, CES, which took place in Las Vegas in January. Examples include the ‘Rabbit R1’ and the ‘AI Pin’ – devices the size of a smartwatch display that are worn like a brooch.

Deutsche Telekom is actively working on an AI gadget that uses AI to optimize smartphones in such a way that the use of apps becomes superfluous. According to rumors, the industry pioneer OpenAI is involved in such an AI gadget. We are in the early stages of the very first generation of AI hardware.

If this AI hardware catches on and the user experience lives up to its promise, it will create a new touchpoint for banks and their customers. A new communication channel is being created as part of an institution’s omnichannel strategy.

It is advisable for financial institutions to work with AI gadget providers early on to avoid repeating the same mistake they made with smartphones: leaving control of payments to AI hardware providers and only waking up when it’s too late.

Imagine buying an item of clothing with the “Rabbit R1”, where payment is confirmed by voice command with a simple “yes”. Which financial service provider makes this possible doesn’t really matter to the customer and fits seamlessly into the customer experience. There is a risk that banking will become even more interchangeable and the bank will fade completely into the background.

In any case, banks must not bury their heads in the sand, but must now address all the issues surrounding AI gadgets: In the age of AI hardware, how can we shape access to our financial services via this new channel without weakening our position?

Would you like to find out more about our view on the topic of AI? Here we tell you what it takes for AI to become trustworthy and what this means for the financial sector. You can read about the use cases for generative AI that we are already seeing in banking here.