LLMs and CPUs: The Battle for Processing Power Dominance

In the ever-evolving world of technology, one thing remains constant: the pursuit of advancements. Computers have come a long way, captivating millions of users, but their limitations in speed and storage have always been a source of frustration. The demand for enhanced performance, in the battle between LLMs and CPUs, has been insatiable, driving the need for more powerful machines. However, as soon as these new generations of computers hit the market, developers waste no time in creating applications that push them to their limits, leaving us yearning for even more improvements. The central processing unit (CPU) emerges as the primary bottleneck in this ongoing cycle.

When it comes to CPUs, one name reigns supreme: Intel. With a staggering 80-90% market share, Intel’s dominance has been unparalleled. Yet, the tides began to shift with the advent of the mobile platform, where battery efficiency took center stage. Suddenly, Intel’s stronghold started to crumble. Today, we find ourselves at the dawn of a new era, as a new kind of “central processor” emerges to define performance across a myriad of applications.

Enter LLMs (large language models), the darlings of the processing power landscape. OpenAI, the leading supplier of LLMs, recently unleashed their latest creation, GPT-4, setting new benchmarks in performance. However, the hunger for further advancements remains insatiable among developers and users alike.

While LLMs share a spiritual connection with CPUs, it remains to be seen if their business prospects will mirror the profitability and defensibility of their CPU predecessors. OpenAI faces competition on the horizon, and we can draw lessons from history, particularly Intel’s memory failure. Intel, once deeply immersed in the RAM memory business and instrumental in shaping the market, encountered challenges as manufacturing efficiency took precedence. As memory architecture standardized, competition intensified, favoring companies that excelled in speed, efficiency, and higher yields. Intel’s forte lay in chip design, not manufacturing processes.

From a functional standpoint, LLMs serve as the central “brain” behind intelligent software features. However, their current performance levels leave room for improvement, representing a significant bottleneck in achieving optimal AI product quality. The demand for enhancements from developers and users is enormous. Yet, LLMs also share striking similarities with RAM.

Although LLMs possess intricate internal complexity, their interface with integrated applications remains remarkably simple: text in, text out. (Granted, the latest LLMs now support text and image inputs, but the fundamental simplicity stands.) Swapping one LLM for another is a breeze, requiring a mere line of code, particularly when leveraging frameworks like LangChain. Unlike CPUs, the ease of interchangeability poses a challenge for LLM providers.

However, the cost of utilizing LLMs proves prohibitive for many applications, intensifying the competitive pressure to achieve efficiency at scale and cost reduction. In this regard, LLMs resemble cloud storage providers, where switching between platforms is relatively straightforward. The absence of high switching costs hampers the growth of ecosystem dynamics, creating a more competitive landscape.

OpenAI’s network effect primarily hinges on its direct connection with users through ChatGPT. While this grants OpenAI access to proprietary training data, the impact on LLM quality remains uncertain. Nevertheless, the possibility of OpenAI or its competitors open-sourcing their models could completely alter the dynamics. Operating self-hosted LLM instances and customization pose complexities, attracting larger organizations with specific privacy and customization needs. This represents an attractive customer base, even though it may involve sacrificing potential revenue from operating a hosted LLM service.

For developers, the current landscape presents a tremendous opportunity to build LLM-powered applications. While OpenAI currently enjoys a dominant position, the emergence of one or two viable