Opportunity: The AI Engineer

I see three clear patterns in today’s GenAI industry:

1. The nature of LLM base models steers the market toward a natural monopoly: the high training costs, slow training times, and low supply of specialized LLM researchers imply large barriers to entry to create state of the art (SOTA) models.

2. Foundational models keep on improving: the level of compute needed to achieve a given level of performance has halved roughly every 8 months. This implies a quicker rate than the one estimated by Moore’s law (transistor size halved every 2 years), a law which has made many of the advances in technology we see possible.

3. Companies are struggling to reach production with LLMs: Forbes estimates that 90% of GenAI Proof of Concepts (PoCs) will not move into production. This is a high failure rate, especially considering many of these PoCs would revolve around foundational projects like Chatbots. As use cases grow more complex, so will the complexity of reaching production.

Why are these patterns relevant?

Almost two years ago, Latent Space already defined 4 distinct software roles in the GenAI space:

Latent Space’s understanding of AI Engineers (latent.space)

1. ML Researcher / Research Scientist: are responsible for training efficient and performant LLMs. Although their role is fundamental for the growth of GenAI adoption, only a handful of companies will develop meaningful models.

2. ML Engineer / Data Scientist / Research Engineer: focus on collecting data and fine tuning models and enabling inference. If we continue to see improvements in the performance of LLMs, newer generic models will be able to outperform older fine tuned models. In the 20th century, a program could run twice as fast just by waiting two years thanks to Moore’s law. We can extrapolate this situation to LLMs, where just by waiting we will see better performance and faster inference times.

3. Full stack engineers: will continue to focus on providing stable platforms and incremental features to existing products. The productivity of engineers in this role will increase with GenAI coding agents, but they will not be leading the wave of innovation in the next years.

4. The AI Engineer

The 4th role described by Latent Space is where opportunity lies. AI Engineers are consumers of APIs from LLMs. Their goal is to create software that leverages the capabilities of LLMs by mastering the art of prompting, coordinating, monitoring and providing the appropriate tools to wield this new technology. AI Engineers bring the skills of software developers with the knowledge of how to use LLMs to deliver complex GenAI applications to production. AI Engineers take advantage of the three trends mentioned above:

1. They do not try to enter a monopoly market.
2. The capabilities of the software they can design will only improve as the models improve.
3. They bring software engineering knowledge to help customers design production ready solutions.

Moreover, AI Engineers can design software solutions covering higher program complexity. The paradigm shift of software 3.0 identifies that business logic, I/O control, and the data pre/post-processing will be partially or even entirely guided by the AI Agent, allowing for a use case far beyond the software 2.0 (traditional ML) and 1.0 paradigms (traditional programing).

This role is a growing industry trend (Microsoft has released an AI Engineer certification and O’Reily has a book on becoming an AI Engineer), and I believe more and more traditional software engineers will need to adapt their code to the indeterministic nature of LLMs.

What does this mean?

As I often do, I like to reference the Gartner hype cycle and again acknowledge that GenAI is clearly in the peak of inflated expectations. But, just like with the .com bubble, when the bubble bursts, the small percentage of companies that survive will have a real impact on the future of our world. AI Engineers will be the technical role leading this wave.

---

PD: Beyond the hype, AI Engineers are just Software Engineers that know how to prompt, work with non-deterministic responses, optimize token usage and deal with all the other quirks of calling LLMs with APIs. The skills for AI Engineers are not far from the ones already held by Software Engineers.