Lessons from the frontlines of small-model development
Why small models are harder than they look, with Maxime Labonne
Something Maxime Labonne said during one of our earlier roundtables stuck with me because it runs counter to how people think about small models.
You can watch the video here!
The common assumption is that small models are simply the cheaper version of frontier models. Same idea, lower cost. But Maxime’s experience has been that they’re often harder to work with. Not because they’re worse, but because they expose problems that larger models can often hide.
When most people build AI systems today, they’re testing them with frontier models. Those models are smart enough to compensate for weak prompts, incomplete logic, or edge cases that nobody thought about. Small models don’t give you that luxury. As Maxime put it, they can fail on surprisingly basic tasks, and when they do, entire workflows can break.
That changes the engineering. You need stronger guardrails, better error handling, and less reliance on the model figuring things out for itself. In some ways, small models force you to build more robust systems because they expose weaknesses earlier.
The tradeoff is that you get something valuable in return. Small models can run on phones, laptops, and local infrastructure. They’re private, can work offline, and give you a level of control that isn’t always possible with centralized frontier models.
Maxime doesn’t see this gap lasting forever, however. He pointed to techniques like distillation and reinforcement learning as reasons small models are steadily improving, particularly when it comes to agent capabilities.
Agentic Engineering grew out of a desire to create something useful for people trying to make sense of agentic AI and beyond. If you’d like to support that direction, please tell your friends about us.