Generative Artificial Intelligence (AI), generative artificial intelligence, or generative AI is one of the techniques to create artificial intelligence (AI - Artificial Intelligence), in which computers will use deep learning models to create content from human requests including sound, code, images, text, simulations and videos. Through many comparisons, reality shows that, with the ability to process many important commands at the same time, the AI model can "surpass" humans in certain tasks.
A huge technological transformation is taking place. However, there is a question that arises “So, who will benefit from generative AI platforms?”
Last year, a16z's team of authors including Matt Bornstein, Guido Appenzeller, and Martin Casado met and discussed with dozens of founders and executives of large companies applying generative AI.
It can be seen that up to now, infrastructure providers are the leaders in the race to profit from this market. Even though leading application companies have phenomenal revenue growth, they face barriers from product maintenance, differentiation, and gross profit. The companies that pave the way for this market - companies that provide generative AI models - have not yet expanded to a commercial scale.
In other words, the companies that create generative AI models and integrate AI into new applications are the companies that make the most contributions to the market, but do not return profits. As a result, market prediction almost falls into a dead end.
The most important thing to grow in the generative AI market is to know how to create different, exclusive values and limit the risk of those different values being copied. Those different values will help change market structure (i.e. horizontal vs. vertical company growth) and long-term value drivers (like margins and retention). The authors said that in addition to the traditional moats (moats) being used, identifying protective structures in stacks (compartments) is facing many difficulties.
Generative AI is expected to make a big impact on the software industry. This article will clarify the dynamics of the generative AI market and discuss the business models of generative AI.
High-level tech stack: Infrastructure, models, and applications
To understand how generative AI models are being created, we first need to clarify the technology systems in these models.
Stack - The stack (which is an abstract data structure that operates on the “last in, first out” principle) can be divided into three layers:
Applications - Applications that integrate generative AI models into products, operate their own pipeline models (end-to-end applications) or rely on third-party APIs (Application Program Interface - allowing two software components to communicate with each other using a set of definitions and protocols).
Models - Models for providing AI products, offered as proprietary APIs or as open source checkpoints (this requires a hosting solution)
Infrastructure - Infrastructure providers (i.e. cloud platforms and hardware manufacturers) perform training and inference tasks for generative AI models
Generative AI applications are starting to grow - Barriers to maintaining product retention and differentiation
In previous technology cycles, to build a large independent company, you had to reach the end-customer, whether an individual consumer or a B2B customer. Therefore, companies applying Generative AI technology today also expect to be able to provide products to end users like their predecessors.
Besides, Generative AI applications are predicted to have incredible growth because of their novel features and flexible application capabilities. Some models such as image generation, copywriting, and code writing have achieved annual revenue of over 100 million USD.
However, growth alone is not enough to ensure long-term growth for software companies. Growth must be profitable (high gross margins) and long-term engagement (high retention) is the key factor. In the absence of technical differences, B2B and B2C applications drive long-term customer value creation through network effect, data ownership, or the ability to standardize complex workflow models.
In fact, if put in the case of generative AI, the above hypothesis is not entirely accurate. Application companies report a wide range of gross profit margins, some as high as 90%, but in many cases as low as 50-60%. This difference is mainly due to the cost of applying the model. Top-of-funnel growth is impressive, but current customer acquisition strategies cannot predict future growth, as the effectiveness of paid acquisition (attracting paying users) and retention begin to decline.
Many applications are generally undifferentiated, as they rely on similar underlying AI models and have not yet found clear network effects or data/workflows that are difficult for competitors to copy.
So it's not yet certain that providing applications to end users is the only or best path to building a sustainable generative AI business. Profits will arise as competition and efficiency in language models increase. Retention rate will increase when “tourist” AI leaves the market. And there is an argument that vertically integrated apps will have an advantage in driving differentiation. Anyway, these are all hypotheses and still need many specific practical examples to prove.
Some requirements for companies applying generative AI include:
Vertical integration (“model + application”): Using AI models as a service allows application developers to iterate on programs (algorithms) quickly with just a small team and can change model providers as the technology advances. On the other hand, some developers believe that the main product here is the model and that training from scratch is the only way to make a difference - that is, continuously training on proprietary product data. But the process will require more capital and a less flexible product team.
Build more features and applications: Generative AI products come in many different types, like desktop apps, mobile apps, Figma/Photoshop plugins, Chrome extensions, even Discord bots. AI products are increasingly easily integrated into human workplace tools, because the user interface (UI) is basically just a text box. So which of the above companies will become a monopoly, and which company will be acquired by giants that own many product lines like Microsoft or Google.
Managing through a hype cycle: It remains to be seen whether there will be a natural decline in current generative AI products, or whether the growing interest in generative AI will decline as the bubble cycle subsides. These questions have important implications for application development companies, including: When is the right time to call for capital? How to boost investment in customer attraction? Which user segments are prioritized? When is product-market fit achieved?
Model providers that create Generative AI have not yet reached a large enough commercial scale
Generative AI would not exist without the research and high level of engineering done by giants like Google, OpenAI, and Stability. Through novel techniques and huge efforts to scale the pipeline model, models such as LLM – large language (language model) and image-generation will bring enormous benefits.
However, the revenue of these companies is still relatively small compared to the value of Generative AI products and expectations about this model. Typically, Stable Diffusion's imaging model, the product has created explosive growth in the user community, supported by an ecosystem of user interfaces, hosted offerings and fine-tuning methods. Although checkpoints bring high value to the market, they are provided free of charge, and have become a core tenet of Stability's business.
In natural language models, OpenAI has the upper hand thanks to GPT-3/3.5 and ChatGPT. In the company's history, there have been relatively few market-dominant applications built with OpenAI. However, today's major competitors like Stability are currently just a young company that has not yet focused on making money. Therefore, OpenAI has the potential for further scale growth, and in particular, the integration of OpenAI products into Microsoft's product portfolio has gone smoothly, contributing to OpenAI capturing a significant market share in the total revenue of the NLP category even as more superior applications appear.
Open source operating models can be hosted by anyone. Even outside companies do not bear the costs associated with large-scale models (up to tens or hundreds of millions of USD). And it's unclear whether any similar code can maintain its advantage indefinitely.
For example, we're starting to see LLMs built by Anthropic, Cohere, and Character.ai, etc. getting closer to the performance levels of OpenAI, trained on similar datasets (i.e. the Internet) and with similar model structures. The example of Stable Diffusion suggests that if open source models achieve sufficient levels of performance and community support, it may be difficult for proprietary alternatives to compete.
By far the clearest point for model providers is that commercialization is likely to be tied to storage. Demand for proprietary APIs (from OpenAI) is growing rapidly, and hosting services for open source models, such as Hugging Face and Replicate, are emerging as hubs to make it easier to share and integrate models. Some indirect network effects are even formed between model manufacturers and consumers. Besides, many other theories suggest that it is possible to make money through hosting and tweaking agreements with corporate customers.
However, there are still some other problems that model providers face:
Commodity: Over time, AI models will all boil down to performance. The advantage of first-mover product developers is not based on unique model structures, but on high capital requirements, proprietary product interaction data, and scarcity of AI talent.
Risk level: Relying on technology from suppliers is a great way for companies to start and grow their business. However, there are also many opinions that encourage businesses to develop their own models when they reach a certain level of growth. Because many model providers have some outstanding applications that bring in most of the dishonest revenue in providing to customers. So, what happens when these customers turn to developing AI for their own companies?
Is money important?: Generative AI can bring many benefits but also comes with many limitations. Many providers have had to operate as public benefit corporations, issuing limited profit shares, or otherwise making sharing profits with the community part of their mission. While this hasn't put a damper on capital raising, another debate has arisen about whether most model providers really want to capture the full value from this market.
Infrastructure providers are reaping significant profits
Model providers or research labs running model training workloads, hosting companies doing fine-tuning, or application companies doing a combination of both all must go through GPUs (or TPUs) hosted in the cloud to be able to apply generative AI. Therefore, FLOPS (FLoating-point Operations Per Second - computer performance) is the lifeblood of Generative AI. For the first time in a very long time, advances in the most disruptive computing technology are at the frontier of mass computing.
And current results are showing that the money flow of the Generative AI market is flowing to infrastructure providers. A rough example shows that: application companies spend an average of about 20-40% of revenue on analysis and customization for customers. This is typically paid directly to cloud providers for compute instances or to third-party vendors. So cloud providers can benefit from 10 to 20% of total revenue from generative AI.
In addition, startups creating their own models have raised billions of dollars in venture capital, most of the money in initial capital calls (up to 80-90%) was spent on cloud providers. Many public-tech companies spend hundreds of millions of dollars each year on model training with external cloud providers or directly with hardware manufacturers.
For a young market like generative AI, companies have spent a lot on the technical side. Most of that is spent on the Big 3 cloud platforms, including: Amazon Web Services (AWS), Google Cloud Platform (GCP) and Microsoft Azure.
These cloud providers collectively spend more than $100 billion each year in capital investment to ensure they have the most comprehensive, reliable, and cost-competitive platforms. In generative AI in particular, they also benefit from supply constraints because they have priority access to scarce hardware (e.g. Nvidia A100 and H100 GPUs).
However, we will witness a fierce competition. Typically, Oracle has made inroads with large investment costs and sales incentives. And several other startups, like Coreweave and Lambda Labs, have grown rapidly with solutions specifically targeting large model developers. They compete on cost, availability, and personalization support, while also exposing more granular resource abstractions (i.e. containers), while large clouds only offer VM instances due to GPU virtualization limitations.
Driving the majority of AI workloads and perhaps the biggest winner in the entire AI space to date is Nvidia. The company recorded $3.8 billion in data center GPU revenue in the third quarter of 2023, a significant portion of which was used for generative AI use cases. They have spent decades investing in GPU engineering to build solid moats into the model, a robust software ecosystem, and widespread use within the community. Currently, Nvidia GPUs are cited in research papers 90 times more times than the top AI chip startups combined.
Besides Nvidia, there are also many other hardware options such as Google Tensor Processing Units (TPU); AMD Instinct GPUs; AWS Inferentia and Trainium chips; and AI accelerator tools from some startups such as Cerebras, Sambanova and Graphcore. Intel entered the generative AI game quite late with Habana chips and Ponte Vecchio GPUs. But so far, very few of these new chips have been able to capture market share, with the only two exceptions being Google (whose TPU has gained traction in the Stable Diffusion community and in several large GCP deals) and TSMC (which can produce all of the chips mentioned above, including Nvidia GPUs).
In short, generative AI infrastructure can bring in profits, as well as the ability to develop sustainably and limit the risk of technology copying.
Below are some major issues facing infrastructure development companies including:
Own stateless workloads: Most AI workloads are stateless, in the sense that model inference does not require a database or attached storage (except for model weights). This means that AI workloads can be more flexible on clouds than traditional application workloads. So in the current context, because no matter who Nvidia GPU is provided by, there is no difference in usage value, cloud providers need to build loyalty and limit the risk of customers switching to cheaper options.
Survive through the chip scarcity period: Pricing for cloud service providers and for Nvidia itself is driven by the scarcity of GPUs. The A100's list price has actually increased since its launch, going against the usual trend of computer hardware. As we address this supply constraint, through increased production and adoption of new hardware platforms, how will this impact cloud providers?
Challenges that cloud providers need to overcome: There is an expectation that vertical cloud scaling platforms will take market share from the Big 3 with more specialized services. So far, in the AI market, competing platforms attract customers thanks to their technical differences and the support of Nvidia (the largest customer and also an emerging competitor of current cloud providers). The question here is in the long term, will this be enough to overcome the scale advantages of the Big 3?
So… where are the market profits?
Of course, this cannot be confirmed with certainty. But based on the initial data we have about generative AI, combined with the authors' experience with previous AI/ML companies, we have the following prediction.
Today, there doesn't seem to be any systematic moat in generative AI. It could be that the applications lack clear differentiation in the product because they use similar models; The models will not have any obvious differences in the long term because they are trained on the same data set with the same structure, cloud providers will have almost the same techniques because they run the same GPU, and even hardware companies (hardware companies) that produce chips use the same FABS.
Of course, there are other standard moats such as scale moats, ecosystem moats, supply-chain moats, algorithmic moats, distribution moats, data pipeline moats. But none of these moats tend to be sustainable over the long term. And it's too early to know whether strong, direct network effects are dominating any layer of the stack.
The potential size of this market is difficult to grasp so we expect many, many players and healthy competition at every level of the system. We also expect both horizontal and vertical companies to succeed, with the best approach determined by the end market and end users.
For example, if the main differentiator in the final product is the AI itself, then it is likely that verticalization (i.e. tightly combining the user-facing application with the user-developed model) will prevail. Meanwhile, if AI is part of a larger long-tail feature set, it is more likely to occur horizontally. Of course, we will also see the construction of more traditional moats over time – and we may even see new types of moats taking hold.
No matter what, one thing we can be sure of is that generative AI will make a big change to the market. We are all digging deep into the rules, there is a lot of value to be unlocked and the technology landscape will be a completely different picture as a result.
Source: