AI Engineering book: First Notes
Impressions, takeaways, and concepts from a quick look at Chip Huyen's AI Engineering.
I started reading AI Engineering by Chip Huyen. These are my notes on the concepts I found interesting. This is the first entry in an ongoing series where I document what I learn, chapter by chapter.
Chip opens with what surprised her about ChatGPT: a relatively small improvement in model quality led to an explosion of possibilities — new applications, new use cases, a whole ecosystem seemingly overnight. She reminds us that the seed of these technologies has been around for a while; the papers powering them were published as early as the 1950s.
The core goal of this book is to train us in how foundation models work so we can adapt them to solve real-world problems.
Regardless of how fast the tooling evolves, the best practices for working with AI models stay the same:
Systematic experimentation
Rigorous evaluation
Optimization toward cheaper and faster models
The book provides a framework for adapting foundation models — both language models (LLMs) and multi-modal models (LMMs) — and helps answer a set of practical questions that I found very interesting to think of:
Should I build this AI application?
How can I evaluate and measure my app?
Why do models hallucinate, and how can I prevent it?
How can I get the most out of prompt engineering?
What is RAG and how do I use it?
What is an agent? How do I build and evaluate one?
When should I fine-tune a model?
How do I make my model faster, cheaper, and safer?
How do I create a feedback loop for continuous improvement?
Beyond these questions, the book also covers model types, evaluation benchmarks, use cases, and AI application design patterns.
Chip also wrote Designing Machine Learning Systems (DMLS) and considers both books complementary. Some topics are more relevant to ML Engineering and get deeper coverage in DMLS.
AI Engineering is not a tutorial. It's about understanding the fundamentals of this role — the practical concepts needed to build AI applications that solve real-world problems.
Chip says you don't need deep Machine Learning knowledge to build AI applications, but it helps to be familiar with a few core concepts:
Probability: sampling, determinism, and distribution
Machine Learning: supervision, self-supervision, log-likelihood, gradient descent, backpropagation, loss functions, and hyperparameter tuning
Neural network architectures: feedforward, recurrent, and transformer
Metrics: accuracy, F1, precision, recall, cosine similarity, and cross entropy
All of these are explained throughout the book as they become relevant.
The book lists several reasons to read it. The ones that really resonate with me:
Identify underserved areas in AI engineering and better understand real use cases
Deeply understand what the role involves and what a career in AI engineering looks like
Understand how this technology works out of pure curiosity
Finally, here's an overview of what the book will cover chapter by chapter:
Use cases and the current state of the industry
How foundation models work under the hood
Evaluation techniques — measuring the behavior of AI applications
Prompt engineering and security
RAG (Retrieval-Augmented Generation)
Fine-tuning
Data — how to generate the best possible data for your application
Inference optimization
I'm really excited to start this new adventure. Onwards.
