His career has moved fluidly between academia and industry, from teaching advanced AI courses at the American University of Beirut to advising teams on turning machine learning ideas into systems that can be trusted.

He is also known for his candid take on the current AI landscape, especially the gap between meaningful engineering and what he often calls AI slop.

In this conversation, Ammar challenges one of the most common assumptions in agent development: that a correct answer is evidence of a successful agent. He explains why reliability lies in the path an agent takes, not just in the result it produces, and why evaluation must evolve from output scoring to a discipline that measures behaviour and trustworthiness in production.

Most teams think they’re evaluating agents, but they’re actually not. Where do you see the biggest illusion of evaluation today?

The biggest illusion is that teams think they are evaluating an agent when they are only evaluating the final answer.

That works well for a chatbot. But an agent is different. It plans, chooses tools, passes arguments, reads observations, retries, stops, and sometimes takes action. A final-answer score hides most of the actual failure surface.

An agent can produce a good-looking answer after calling the wrong tool, wasting ten steps, misreading a tool result, or ignoring a failed call. From the outside, the answer may look acceptable. From a reliability perspective, the run is not acceptable.

So the illusion is: “the answer looked right, therefore the agent worked.” However, what you need to know is whether the path was valid, efficient, grounded, and safe.

You break evaluation into component, trajectory, outcome, and adversarial layers. Where do most teams underinvest, and what failures does that lead to?

Most teams underinvest in trajectory evaluation and adversarial evaluation.

Outcome evaluation is the easiest layer to reach for because the final answer is visible. Component evaluation is also fairly intuitive once tools are involved: did it choose the right tool, did it pass valid arguments, did the plan make sense?

Trajectory evaluation is harder because you need structured traces and assertions over the run itself. But this is where many production failures live: loops, duplicate calls, silent retries, no recovery after a tool failure, unnecessary detours, high latency, high token cost. Two agents can produce the same answer, but one gets there in four clean steps, and the other gets there through an expensive, brittle path. Output scoring treats them as equal. Production does not.

Adversarial evaluation is also underbuilt. Teams may try a few prompt-injection examples manually, but they rarely turn those attacks into a versioned regression suite. That leads to a false sense of safety. A guardrail can look very strong against the examples it was designed for, while still being fragile against slightly different payloads.

Agent failures only show up after deployment. What’s the hardest failure mode to catch early, even with a good evaluation setup?

The hardest failures are the ones that look like successful runs.

A tool returns something plausible but incomplete. The agent takes a reasonable-looking path. The final answer is fluent. No exception is thrown. But the answer is weakly grounded, the evidence is stale, or the agent skipped a recovery step after a bad observation.

These failures are hard because they do not announce themselves as failures. They show up later as retries, edits after the answer, escalations, user abandonment, or quiet loss of trust.

The other hard category is drift. A hosted model changes, a tool schema changes, retrieval content shifts, or user traffic moves into a different distribution. Nothing “breaks” in the traditional software sense, but the agent becomes less reliable. This is why offline evals need to connect to production monitoring. A test suite is necessary, but it is not the whole system.

LLM-as-a-judge is becoming a default pattern. Where does it actually work well, and where does it quietly break?

LLM-as-a-judge works well when the task is bounded, the rubric is explicit, and the judge has the evidence they need. It is useful for rubric-based scoring, regression checks, pairwise comparisons, and multi-dimensional outcome evaluation, especially when you calibrate it against human labels.

The important part is that the judge itself has to be evaluated. I would not trust a judge just because it is an LLM. I would look at correlation with human labels, agreement rates, mean absolute error, and performance by rubric dimension.

Where it quietly breaks is when teams use it as an uncalibrated oracle. Judges often reward verbosity, prefer answers in a certain style, miss missing citations, or give a strong score to an answer that is polished but not grounded. Overall scores can also hide weak dimensions. For example, a judge may be decent on safety or format, but poor on groundedness, which is often the dimension that matters most for a research or retrieval-heavy agent.

So I see LLM judges as useful evaluators, not authorities. They need rubrics, evidence, calibration, and periodic human audit.

If you had to audit an agent system in production with very limited time, what signals or metrics would you look at first to decide if it’s reliable?

Aggregate success rate is often the last metric I look at. I would start with the traces behind the failures and the production signals that users generate when the agent is not working.

The first signals I would inspect are abandonment rate, retry rate, escalation rate, clarification rate, thumbs down, and edit-after-answer rate. Those are often more honest than a dashboard success metric.

Then I would look at trace-level reliability: number of tool calls, duplicate calls, loop-like behaviour, failed tool calls, recovery after failure, latency, and token cost. A reliable agent should not only get the answer right; it should get there through a path that is stable and explainable.

I would also check whether offline evals are tied to production: are failed production examples converted into regression tests? Are adversarial cases versioned? Are judge scores calibrated against human labels? Are there no-go gates for safety, groundedness, cost, latency, and step count?

With limited time, I am looking for one thing: whether the team has connected offline evaluation, online monitoring, and regression gates. If those are disconnected, reliability is usually more assumed than measured.

As organizations continue to explore what AI can and should do, voices like Ammar’s help bring the discussion back to the questions that matter: What problem are we really solving? Can the system be trusted? And are we building something meaningful, or simply adding more noise to an already crowded field?

If you’d like to continue exploring these ideas, Ammar will be speaking at Agent Evals Bootcamp on June 27th, where he will turn these ideas into a hands-on framework for evaluating agents across tool use, planning, trajectories, outcomes, regressions, and adversarial failure modes before deployment.

Of course, this won’t do for us. What’s the point of building custom AI pipelines if they sit locked in a dark room? To make that logic useful, we need to connect it to the real world. We build user authentication, set up secure access controls, write adapters for communication apps like Telegram or Discord, and configure reliable task schedulers.

By combining LangChain with the OpenClaw ecosystem via the LangClaw framework, you can bridge this gap in an afternoon.

Why combine OpenClaw and LangGraph?

OpenClaw began as a configuration-first, out-of-the-box personal AI runtime. It provides developers with a pre-wired ecosystem that includes direct messaging connectors for platforms like WhatsApp and Telegram, persistent memory layers, and native browser automation tools.

Instead of writing code to manage how an agent communicates with a chat application or remembers previous conversations, users simply write markdown files (SKILL.md) to define functions. OpenClaw handles the operational runtime, API routing, and state storage automatically.

While this structure is ideal for standard automation, engineering teams often require deeper programmatic control over their agentic reasoning. They need complex, multi-step state machines, advanced semantic search pipelines, and conditional execution paths.

This is where LangChain excels, and it is exactly why the two systems are paired together.

LangClaw as the glue layer

LangClaw acts as the architectural bridge between these two worlds. It is a declarative, Pythonic framework that compiles complex LangChain reasoning loops directly into an OpenClaw-compatible runtime engine.

Instead of managing external configuration markdown files, you use clean Python decorators to build tools, establish role-based access controls, and schedule autonomous background tasks.

If you’re the kind of person who reads something like this and immediately wants to see the code, we’ve documented a complete LangGraph + OpenClaw build that turns these ideas into a working corporate intelligence agent, covering everything from subagent orchestration and scheduled research tasks to permissions, messaging, and token-efficient utility commands.

Get the playbook here

Let’s build one.

The first step is creating the primary agent and establishing its responsibilities through a system prompt.

import os
from datetime import datetime
from langclaw import Langclaw
from langclaw.gateway.commands import CommandContext
# Initialize the master agent application interface
app = Langclaw(
 system_prompt=(
 “## Corporate Intelligence Agent\n”
 “You are a corporate intelligence analyst. You track market trends “
 “and draft precise outreach sequences based on current events.\n”
 “Delegate deep multi-source research tasks to the web-researcher subagent.”
 ),
)

Tools allow the agent to access capabilities beyond language generation. In this example, the agent can retrieve market intelligence data.

# Custom Tool: Data Extraction Layer
@app.tool()
async def fetch_market_news(sector: str) -> dict:
 “”“Extract breaking market developments for a specific commercial sector.”“”
 # This acts as a standard tool available to your core reasoning models
 return {
 “timestamp”: datetime.now().isoformat(),
 “sector”: sector,
 “headline”: “Warehouse Automation Demands Surge Amid Labor Reshuffling”,
 “body”: “Logistics providers face margin pressures, driving immediate infrastructure upgrades.”
 }

Rather than placing all reasoning responsibilities inside a single agent, LangClaw allows specialised subagents to handle focused tasks.

# Subagent Delegation: Isolated Reasoning Context
app.subagent(
 “web-researcher”,
 description=”Performs deep multi-step web analysis and synthesis.”,
 system_prompt=”You are an investigative researcher. Identify technical pain points and list them.”,
 tools=[”web_fetch”, “web_search”],
 output=”channel”, # Streams output directly back to the active user channel
)

Role-based permissions make it possible to expose different capabilities to different groups of users.

# Role-Based Access Control: Security Gateways
app.role(”premium_analyst”, tools=[”*”])
app.role(”basic_user”, tools=[”fetch_market_news”])

Agents do not need to wait for user prompts. Using cron schedules, they can execute tasks automatically.

# Automated Execution: Background Scheduler
@app.cron(”0 8 * * 1-5”) # Executes Monday through Friday at 8:00 AM
async def automated_morning_brief():
 “”“Scans market trends autonomously and publishes digests to the user feed.”“”
 raw_data = await fetch_market_news(sector=”Logistics”)
 instruction = f”Synthesize this market report for the main channel: {raw_data[’body’]}”
 # Compile runtime structure down to LangGraph for invocation
 compiled_agent = app.compile()
 analysis = await compiled_agent.ainvoke(
 {”messages”: [{”role”: “user”, “content”: instruction}]},
 config={”configurable”: {”subagent”: “web-researcher”}}
 )
 # Broadcast the completed report directly to your active chat channel
 await app.gateway.send_message(
 channel=”management_feed”,
 text=f”🔔 **Automated Market Digest**\n\n{analysis[’messages’][-1].content}”
 )

Not every task requires an LLM. For predictable workflows, slash commands can bypass the model entirely.

# Slash Command: High-Speed Utility Routing
@app.command(”template”, description=”Generate static outreach copy without AI overhead”)
async def outreach_template_cmd(ctx: CommandContext) -> str:
 “”“Returns a structured outreach framework directly, bypassing the LLM entirely.”“”
 args = ctx.message.text.split(’”’)
 client_name = args[1] if len(args) > 1 else “Colleague”
 # Bypassing the LLM completely saves computation costs and provides sub-millisecond responses
 return (
 f”📩 **System Template Generated**\n\n”
 f”Hello {client_name},\n\n”
 f”I noted your team’s footprint in the logistics sector. Given recent automation trends, “
 f”I wanted to connect to share notes on how regional hubs are optimizing their margins.\n\n”
 f”Regards,\n[Internal System]”
 )

Once the components are assembled, launching the runtime is straightforward.

if __name__ == “__main__”:
 app.run()

Once the runtime is launched, the agent becomes more than a collection of functions and workflows. It becomes a persistent service that can receive requests, execute tasks, and communicate results through your chosen channels.

Let’s see how that works.

What happens when the agent goes live

When you invoke app.run(), LangClaw spins up a persistent asynchronous gateway server. It does not wait for a single terminal input; it establishes a permanent network listener.

Interaction happens entirely through your chosen deployment endpoints:

• Production chat channels: By adding tokens for Telegram, Discord, or Slack into your .env configuration file, your script logs into those platforms as a live bot user.

• LLM invocations: When a user types a natural language question into the chat, LangClaw pipes the string into your LangChain model, runs the necessary code tools, and formats the response.

• The slash command route: When a user types /template “Alex”, the gateway identifies the leading slash character. It isolates the instruction, skips the LLM reasoning step, runs the native Python function, and returns the text.

  ---

Hosting your agent for less than £5 per month

Deploying a traditional enterprise application often brings significant server management costs. However, because this framework relies on an event-driven architecture and asynchronous IO libraries, it requires minimal computing resources.

You can host this entire stack for under £5 per month using modern cloud infrastructure.

Virtual Private Servers (VPS)

Platforms like DigitalOcean, Hetzner, or Linode offer entry-level Linux instances with 1GB of RAM and 1 CPU core. Because LangClaw uses a non-blocking asyncio event loop, a single micro-instance can easily manage hundreds of concurrent chat messages and automated cron tasks without breaking a sweat.

Process Management

To ensure your agent runs continuously, you wrap the script using a Linux system process supervisor like systemd or Supervisor. If the server reboots or encounters an unhandled API timeout, the operating system brings the agent back online instantly, providing 99.9% uptime with zero manual maintenance.

We’ve set up an agent, we’ve talked about messaging and about the advantages of full control, about the hosting. Building your automation stack with this methodology yields three immediate operational improvements:

• Zero-token utilities: By routing routine structural actions through slash commands, you bypass LLM token fees entirely. This provides instantaneous responses while eliminating API runtime costs for predictable tasks.

• Decoupled architecture: Your core business logic remains completely separate from your communication channels. If your team decides to migrate from Telegram to Slack, you alter a single environment variable. The underlying tools, subagents, and schedules require no modifications.

• Production readiness: You move from an abstract local script to an enterprise-grade agent. The system manages its own user security roles, schedules its own data retrieval cycles, and operates autonomously around the clock.

Let us know how you get on!

You can watch the video here!

The common assumption is that small models are simply the cheaper version of frontier models. Same idea, lower cost. But Maxime’s experience has been that they’re often harder to work with. Not because they’re worse, but because they expose problems that larger models can often hide.

When most people build AI systems today, they’re testing them with frontier models. Those models are smart enough to compensate for weak prompts, incomplete logic, or edge cases that nobody thought about. Small models don’t give you that luxury. As Maxime put it, they can fail on surprisingly basic tasks, and when they do, entire workflows can break.

That changes the engineering. You need stronger guardrails, better error handling, and less reliance on the model figuring things out for itself. In some ways, small models force you to build more robust systems because they expose weaknesses earlier.

The tradeoff is that you get something valuable in return. Small models can run on phones, laptops, and local infrastructure. They’re private, can work offline, and give you a level of control that isn’t always possible with centralized frontier models.

Maxime doesn’t see this gap lasting forever, however. He pointed to techniques like distillation and reinforcement learning as reasons small models are steadily improving, particularly when it comes to agent capabilities.

Agentic Engineering grew out of a desire to create something useful for people trying to make sense of agentic AI and beyond. If you’d like to support that direction, please tell your friends about us.

Share Agentic Engineering

Those questions still matter as foundation models remain the foundation of everything happening in AI today. But it feels like the centre of gravity has shifted (slightly at first and then all at once).

The work happening today isn’t about making a model better at answering questions. It’s about figuring out how to build systems around those models that can reason, retrieve information, use tools, coordinate actions, and operate live workflows.

We had our own theories about where the AI conversation was heading, but theories are cheap. If Agentic Engineering was going to be built around the questions people actually care about, there was only one group worth asking.

So we turned to the people reading this

A few weeks ago, we asked you what you wanted more of from AI coverage. While the responses covered a wide range of interests, the overall message was consistent (and I’m not surprised).

You weren’t asking for more AI news.

You weren’t asking for more model rankings.

And you certainly weren’t asking for more hype.

The most common frustrations with AI newsletters today were too much hype, a lack of practical depth, and repetitive coverage that often feels disconnected from how AI is actually being used. What you wanted instead was real-world workflows, implementation advice, clear explanations of complex topics, and lessons from teams building with these technologies in practice.

The interesting part wasn’t that AI agents ranked first. Honestly, I expected that.

What caught my attention was everything sitting underneath it.

Open-source AI. Infrastructure. RAG systems. Multi-agent systems. Agent orchestration. Evaluation.

At some point, it stopped looking like interest in a technology category and started looking like interest in an engineering discipline.

Taken together, those responses point toward something bigger than a content preference. They point toward a shift in the conversation itself.

Once AI systems begin interacting with databases, applications, workflows, enterprise systems, and even other agents, an entirely new set of questions emerges.

• How should these systems be evaluated?

• How do you observe and debug them?

• What does governance look like when AI can take actions rather than generate outputs?

• How much autonomy is too much?

• How do you move from an impressive demo to something people can actually trust?

These aren’t purely research questions. They’re engineering questions that sit at the intersection of software engineering, AI engineering, operations, product design, governance, and systems thinking.

That’s why Agentic Engineering was born

Not because every workflow needs an agent. Not because every company should rush to deploy autonomous systems. And certainly not because we’re interested in chasing the latest buzzword.

Agentic Engineering exists because a distinct engineering discipline is beginning to emerge around how these systems are designed, evaluated, deployed, governed, and integrated into real-world environments.

Where we go from here

Over the coming months, we’ll be exploring that discipline through expert-written analysis, interviews with practitioners working close to the technology, research breakdowns, case studies, and conversations with the people building these systems every day.

The truth is that nobody has all the answers yet. The playbook for building reliable agentic systems is still being written. Best practices are emerging in real time. Some ideas will prove durable. Others won’t survive contact with production.

We’re watching a new engineering discipline take shape, and we’d like to document it as honestly as possible. The breakthroughs, the lessons, the contradictions, the failures, and the things that genuinely move the field forward.

If you’re trying to make sense of where agentic systems go from here, you’re in the right place 🙂

That’s what made this roundtable session so interesting for my co-host, Ali Abidi, and me.

We spent an hour digging into these questions with three people who’ve helped shape how the industry thinks about AI engineering: Chip Huyen, Paul Iusztin, and Maxime Labonne.

Chip Huyen brought the systems-level perspective she’s become known for through Designing Machine Learning Systems and AI Engineering: why reliability, orchestration, tooling, memory, evaluation, and infrastructure choices are quickly becoming more important than flashy demos.

Paul Iusztin and Maxime Labonne, co-authors of LLM Engineer’s Handbook, brought deeply practical perspectives from the frontlines of modern AI engineering, especially around modular architectures, production workflows, multimodal systems, post-training, fine-tuning, and the engineering discipline needed to move beyond PoC purgatory and into systems that can actually survive real-world use.

The conversation stayed honest throughout. Nobody pretended this space is “solved.” The discussions revolved around trade-offs, failure points, bottlenecks, orchestration chaos, and the messy engineering layer that starts appearing the moment agents interact with real systems and real users.

Huge thanks again to Chip, Paul, and Maxime for such a thoughtful conversation. Ali and I had an absolute blast hosting this one.

Thanks for reading!

Agentic Engineering grew out of a desire to create something useful for people trying to make sense of agentic AI and beyond. If you’d like to support that direction, please tell your friends about us.

Share Agentic Engineering

Once agents begin using tools, retrieving information, maintaining memory, looping through decisions, and interacting with external systems, things stop looking like a simple chatbot pretty quickly. You start ending up with something closer to an operational system that needs planning, structure, and supervision to keep functioning coherently.

And that’s the part of the conversation that still feels oddly underdiscussed.

A lot of attention goes toward models and autonomy, but much less toward the layer responsible for coordinating all of this.

That layer is the orchestrator. And the more complex agentic systems become, the more important the orchestration layer becomes.

Join Angentic Engineering

For builders navigating the move from models to agents

An AI orchestrator serves as the central hub that coordinates interactions between the model, tools, memory stores, APIs, and other external systems. It ensures that AI agents operate effectively and in a controlled manner.

AI orchestrators can help in the following:

• Managing complexity: AI workflows often involve multiple coordinated steps such as retrieval, reasoning, and action execution. Orchestrators automate and structure these processes, making systems easier to scale and maintain.

• Enhancing scalability: Orchestrators handle high loads by distributing tasks, caching responses, and parallelizing operations—critical for handling multiple users or token-intensive tasks.

• Ensuring context awareness: Since LLMs have limited memory, orchestrators integrate vector databases and memory systems to help agents retain information and deliver more coherent, personalized experiences.

• Facilitating tool integration: Orchestrators streamline the use of APIs, search engines, and databases by managing task execution and ensuring smooth interaction between LLMs and external tools.

• Improving reliability and monitoring: From logging to human-in-the-loop feedback, orchestrators offer tools to catch errors, prevent hallucinations, and ensure systems run securely and reliably.

To better understand the need for an AI orchestrator in the context of AI agents, we need to introduce three key features of AI agents: autonomy, abstraction, and modularity.

Subscribe now

Autonomy

Autonomy refers to an AI agent’s capacity to operate independently, making decisions and executing actions without human intervention. This self-directed behavior enables AI agents to perform tasks, adapt to new situations, and pursue goals based on their learned experiences.

The autonomy of an AI agent implies that the steps that the agent is going to take are not necessarily known in advance.

For example, let’s consider a non-agentic workflow as simple as the following:

Whenever we prompt an LLM, we are doing an API call, which is the only step that comprises this workflow. Even in the scenario of a retrieval augmented generation (RAG) workflow, steps are known in advance

Let’s now consider an autonomous agentic approach. Let’s say we have an agent with two tools:

• Weather tool: a function that takes two parameters: city and unit of measurement.

• Location tool: a function that takes the current position of the user, leveraging the GPS position. This function takes no parameters.

Both functions come with a natural language description, as per the AI agent’s anatomy. Our workflow design will let the agent decide which tool to invoke

Let’s say that a new user asks, “What’s the weather for tomorrow?” The following things will happen:

1. The agent will read through the descriptions of its tools and understand that it needs to invoke the weather tool. However, it’s missing the two parameters, but thanks to its autonomy, it can look around to retrieve them. It soon understands that it can leverage the location tool to get the first parameter: the output of that function will serve as the parameter of the weather tool.

1. For the second parameter, the agent cannot make it alone: it needs to ask the user. So it does, asking the user which kind of unit of measurement is needed. Once the user responds, the agent is able to properly invoke the tool with both parameters.

1. The agent observes the output of the weather tool and ascertains that it now knows the final answer for the user.

Can you imagine how many “if…else” statements would have been needed to replicate such a degree of autonomy in a standard Robotic Process Automation (RPA) process? And even if we could manage a similar scenario, what if the user asks something that has not been hardcoded in the process? Adaptability, self-critique, and self-adjustment are key features of agentic autonomy.

There are many degrees of autonomy we can provide our agents with: it all boils down to the workflow we set up and the planning strategy we instruct our agents to follow. Designing the proper agentic workflow is an architectural design conversation that is core when building your agentic state.

Abstraction and modularity

Abstraction refers to breaking down and simplifying complexity. It is what makes these systems comprehensible and scalable. ​But beyond simplification, it also enables modular design, a fundamental principle in building intelligent systems.

Modularity breaks down complex problems into smaller, reusable components, each handling a specific part of the challenge. This approach offers several advantages:

• Interchangeability: Components can be swapped, upgraded, or replaced without affecting the entire system

• Reusability: Well-designed modules can be repurposed across different projects, improving efficiency

• Scalability: Independent yet seamlessly integrated components make it easier to expand solutions

In multi-agent systems, abstraction and modularity allow for the creation of cooperative agents, each specializing in specific tasks while interacting dynamically. This mirrors human problem-solving, where we divide, delegate, and collaborate to tackle complexity effectively.

A great way to understand abstraction and modularity in agentic patterns is by looking at a multi-agent traffic management system in a busy metropolitan city, where different levels of agents handle different levels of abstraction, ensuring smooth operation without overwhelming any single entity.

At the most granular level, we have intersection controllers, which operate at a single traffic light or intersection. These agents rely on real-time data from cameras and sensors to adjust traffic signals based on vehicle congestion, pedestrian movement, and emergency vehicle priority.

They don’t worry about what’s happening in the next city block or the broader urban landscape; their only job is to optimize traffic flow at their specific location. If a sudden influx of cars appears at a junction, they may extend green-light durations to ease congestion.

Zooming out, we have district-level traffic coordinators. These agents don’t micromanage individual traffic lights but instead analyze traffic flow across multiple intersections within a neighborhood or district.

They use data from intersection controllers, GPS tracking, and public transport systems to identify congestion patterns, reroute vehicles, and balance the flow of cars across the area. If they detect excessive delays in one zone, they adjust how much time a light is on for the traffic signal for multiple intersections rather than just one.

More importantly, they direct intersection-level agents, ensuring their adjustments align with broader district-wide traffic goals.

At the highest level, we have the citywide traffic management system, responsible for optimizing the flow of millions of vehicles across the entire metropolitan area. This agent doesn’t focus on specific traffic lights or individual congestion points; instead, it allocates resources, predicts long-term patterns, and makes strategic adjustments.

Using data from weather reports, major event schedules, accidents, and public transportation networks, this agent might reroute entire roads, coordinate construction schedules to minimize disruption, or implement city-wide emergency response plans in case of major incidents.

If an accident occurs on a major highway, the citywide system redirects district-level agents to adjust traffic patterns, which in turn instructs intersection controllers to reroute vehicles efficiently.

This layered structure demonstrates the power of abstraction and modularity in multi-agent systems:

• Intersection agents handle local, real-time decisions, adjusting traffic lights and prioritizing immediate flow

• District-level agents analyze and coordinate groups of intersections, optimizing traffic across a wider region

• Citywide agents focus on the big picture, planning for long-term efficiency, emergency responses, and systemic optimizations

This mirrors how software architectures, AI systems, and even corporate structures function in the real world. Whether it’s frontline workers executing tasks, middle managers coordinating efforts, or executives setting the overall vision, abstraction enables complex systems to remain scalable, efficient, and resilient.

By designing multi-agent AI architectures with this layered approach, we ensure each agent focuses only on what it needs to handle, preventing system overload and enabling adaptive, real-time decision-making at scale, just like a smart traffic system managing a bustling city.

If this doesn’t sound like a real thing to you, let’s have a look at OpenAI’s tool called Operator, which acts as an autonomous agent, capable of performing tasks in a web browser, such as booking tickets or filling online orders.

OpenAI’s Operator follows a hierarchical multi-agent approach similar to the traffic management system. Each agent operates at a different level of abstraction, ensuring efficiency and adaptability without overwhelming any single component.

• Web controllers (low-level agents): These agents handle execution: moving the mouse, clicking buttons, and entering text. They don’t analyze or plan—they simply follow commands.

• Vision and reasoning (mid-level agents): These agents interpret the web interface. The Vision Agent processes screenshots, detecting relevant elements, while the Reasoning Agent determines the next action (clicking, typing, or scrolling). This layer abstracts away the details of execution, focusing on understanding and decision-making.

• The planner/orchestrator (high-level agent): The top-level agent oversees the entire system, ensuring that web interactions align with broader goals—whether it’s searching for information or filling out a form. It delegates tasks to mid-level agents, ensuring smooth and strategic navigation.

This structured approach highlights why abstraction matters so much in multi-agent design. Low-level agents can focus purely on execution without needing to reason about broader objectives, while mid-level agents handle interpretation, coordination, and planning across tasks. Above them, high-level agents manage overall strategy and orchestration without getting pulled into implementation details. Separating responsibilities this way allows complex systems to remain scalable, adaptable, and manageable without overwhelming any single layer of the architecture.

By leveraging this modular design, OpenAI’s Operator adapts dynamically, handling different websites without requiring manual programming. This scalable and generalizable architecture is a prime example of how multi-agent systems drive real-world AI applications.

From an architectural perspective, all these components – agents, skills, plugins – can be seen as repeatable assets in your organization. In this context, AI orchestrators ensure that these components work together without being tightly coupled, preventing complexity from overwhelming the system.

Following the preceding hierarchical example, with an AI orchestrator, you can easily define the following:

• Execution agents (low-level): These handle raw tasks such as API calls, database queries, or web scraping, executing commands without decision-making

• Reasoning agents (mid-level): They analyze data, determine actions, and select the right tools, abstracting execution details

• Orchestration and planning (high-level): The orchestrator oversees workflows, breaking down tasks, distributing them across agents, and adapting dynamically

By structuring AI systems this way, orchestrators enable adaptive, generalizable intelligence, ensuring seamless interaction between components without manual intervention.

This excerpt was adapted from AI Agents in Practice by Valentina Alto, which does an excellent job of grounding these ideas in real architectural patterns instead of treating agents as abstract autonomous magic. The book goes deep into orchestration frameworks, multi-agent systems, memory architectures, tooling, production deployment, and the operational realities of building AI agents that can function beyond demos.

If you’ve been trying to separate what’s structurally important in the agent ecosystem, this book is worth spending time with.

Explore the book

And I think that’s why the conversation around agents sometimes feels simultaneously overcomplicated and vague. In reality, though, these systems are operating at very different levels of autonomy.

So, after spending the last few articles looking at how agents are structured and orchestrated beneath the surface, I wanted to take a new angle: the different kinds of agents organizations are building, and how capabilities change as systems move from retrieval into action and eventually toward autonomy.

Join Angentic Engineering

For builders navigating the move from models to agents

Looking at agents through this lens, the differences between these systems start becoming much easier to spot.

Some are primarily designed to retrieve and synthesize information more intelligently. Others are built around executing workflows and operational tasks. And some are beginning to operate with much higher levels of autonomy, coordinating decisions and adapting dynamically as conditions change.

Broadly speaking, most AI agents today fall into three categories: retrieval agents, task agents, and autonomous agents.

Retrieval agents

Retrieval AI agents build upon the foundations of RAG but incorporate advanced agentic behaviors, making them more autonomous and adaptive. In fact, we are adding to a standard RAG pipeline an additional layer of intelligence and planning that allows the agent to “strategize” on how to retrieve the most relevant pieces of information.

Let’s consider the following example.

Let’s say we want to build an AI assistant for a doctor to quickly retrieve information about treatments. Given the doctor asks: “What are the latest treatments for Type 2 diabetes?”, let’s see how the two approaches compare:

Traditional RAG approach:

• The RAG system retrieves the top three relevant articles from the database.

• The model extracts relevant text from those articles and generates a response summarizing key treatments.

• If the retrieved documents do not fully answer the doctor’s question, the model cannot refine the search unless the doctor manually submits a new query.

Retrieval AI agent approach

• The agent retrieves an initial set of documents and analyzes them.

• It detects that some retrieved studies are outdated, so it refines its search criteria and retrieves more recent publications.

• It recognizes a gap in information regarding a specific drug and fetches a dedicated study on that drug.

• Finally, it synthesizes all retrieved sources into a comprehensive answer, ensuring relevance and completeness.

In conclusion, agentic RAG can lead to several improvements over traditional RAG. Unlike traditional RAG systems that typically rely on a single retrieval pass, retrieval AI agents can iteratively refine their searches by breaking complex queries into multiple steps and progressively improving the relevance of the information they gather. They also maintain contextual awareness through memory of previous interactions, allowing them to ask clarifying questions, adjust retrieval strategies dynamically, and respond more intelligently over time.

Another important difference is their ability to interact directly with tools such as APIs, databases, and vector search engines, enabling them to fetch structured and real-time information when needed. Rather than pulling information from a single static source, retrieval agents can also augment knowledge adaptively by combining and synthesizing information across multiple sources depending on the context of the query. In many cases, these systems can even make autonomous decisions about when additional retrieval is necessary, which sources are most relevant, and how search results should be refined to improve the overall quality of the response.

Retrieval agents are the simplest form of AI agents, yet the extra layer of intelligence is already demonstrating great improvements for the overall user experience. However, the real power of an AI agent comes to life when it can combine retrieval skills with actionable tasks.

Task agents

Task agents go beyond information retrieval by performing specific actions. These agents are designed to automate workflows and replace repetitive tasks for users. Unlike retrieval agents, they execute predefined actions in response to user commands or external triggers.

Let’s consider once again an example in the healthcare domain, this time from the perspective of the general practitioner’s office receptionist, John.

John manages a high volume of appointment requests. Patients book visits through various channels: phone calls, emails, and an online booking system. Managing last-minute cancellations and rescheduling requests is time-consuming and often leads to gaps in the schedule.

A typical process in John’s day might look like the following:

1. John receives an email from Patient X to book an appointment and shares some preferences in terms of date and time

2. John checks the availability of the specialist practitioner required and tries to match the earliest slot possible with Patient X’s preferences

3. John doesn’t find any match, hence goes back to Patient X to find alternatives

4. Finally, John and Patient X agree on a slot and the appointment is scheduled

If you think about the above steps, they are nothing but tasks that John is meant to perform to achieve the goal – scheduling the appointment in an optimized slot for both the practitioner and the patient.

Whenever we want to map and enhance a business process with an AI agent – more specifically, a task agent – a good practice is that of transposing the human tasks into agentic tasks. Let’s see, for example, how a task agent can assist John:

1. The AI agent automatically scans the email received from Patient X. It extracts key details like the Patient’s name and contact details, preferred date and time, and the specialist practitioner required.

2. The AI agent checks the availability, invoking the plugin (the tool we equip our agent with) in the clinic’s scheduling system. It matches Patient X’s preferences with the earliest available slots for the specialist practitioner. If there’s a match, it proceeds to Step 5.

3. The AI agent finds no match. Since no match is found, the AI agent generates a list of the next best available slots based on the specialist’s schedule. It also drafts a response email to Patient X, leveraging a writing skill, with suggested alternatives, but John reviews and approves it before sending.

4. Patient X responds with a new preference and either:

   • Accepts one of them and proceeds to Step 5

   • Requests new options, then the AI agent repeats Step 3

5. Once John and Patient X agree on a slot, the AI agent automatically schedules the appointment in the system, leveraging the same plugin as above. Plus, it sends a confirmation email to Patient X with the details, leveraging an email plugin. Finally, it updates the specialist’s calendar and notifies them of the booking.

As you can see, the AI agent acts as John’s assistant, handling repetitive scheduling tasks while he focuses on in-person patient interactions.

Autonomous agents

Autonomous agents represent the most advanced category of AI agents. Unlike retrieval and task agents, which operate within predefined boundaries, autonomous agents strategically orchestrate multiple tasks and retrieval processes, making real-time decisions to optimize workflows. These agents exhibit a high degree of independence, adaptability, and contextual awareness, allowing them to perform complex operations with minimal human intervention.

The core distinction of autonomous agents lies in their ability to:

1. Combine retrieval and action: They can both find information (like a retrieval agent) and act on it (like a task agent).

2. Plan and self-adjust: They dynamically adapt based on new information or changing constraints.

3. Perform multi-step workflows: They break down complex tasks into subtasks, execute them iteratively, and adjust based on results. Let’s continue with John’s clinic example. As the clinic gets busier, managing appointments, cancellations, and reschedules becomes overwhelming. A task agent helped streamline individual actions, but now an autonomous agent takes over the end-to-end scheduling process with minimal supervision. Here is how it works, step by step:

   • Intake and prioritization: The agent monitors all channels (email, portal, phone transcripts), extracts patient preferences, urgency, and specialist needs, and ranks requests based on priority. For example, a canceled appointment opens a slot, and the agent immediately matches it to Patient X, who’s been waiting for a similar time.

   • Planning and optimization: It reviews the full daily schedule, identifies conflicts or idle gaps, and builds an optimized plan—shuffling low-priority visits to make room for urgent ones.

   • Execution with feedback: The agent messages patients with options, updates calendars, books appointments, and sends confirmations—all automatically. If preferences change, it loops back, refining its actions.

   • Real-time adaptation: A doctor calls in sick. The agent halts new bookings, reschedules affected patients, and notifies staff—handling all steps autonomously unless human input is needed.

   • Continuous learning: At day’s end, it analyzes outcomes, updates patient preferences, and adjusts future prioritization logic.

The autonomous agent can plan, retrieve, decide, act, adapt, and learn—all without relying on predefined workflows. John now focuses on edge cases, while the agent intelligently handles the rest.

Autonomous agents represent the next step in AI-driven process automation. By merging retrieval AI capabilities (context awareness, real-time query refinement) with task execution skills (appointment scheduling, automated notifications), autonomous agents can fundamentally reshape business processes and daily operations.

There are different degrees of autonomy we can provide our agents with, and the decision is based upon the business scenario as well as the level of confidence we have in the accuracy of the solution.

This piece borrows from AI Agents in Practice by Valentina Alto, which I found myself bookmarking constantly while working through these recent pieces on agents.

What stayed with me most wasn’t just the frameworks or implementation details, but how clearly it separates systems that retrieve, systems that execute, and systems that can independently coordinate decisions across workflows. That sounds obvious when written out plainly, yet a surprising amount of the current ecosystem still collapses all three into the same conversation. The more separated those ideas become, the easier it gets to reason about what these systems are capable of doing.

Expore the book

But while putting that piece together, I kept noticing something interesting. Many of these systems sound new on the surface, yet a lot of the underlying ideas have existed for much longer than the current wave of agent frameworks.

Once you start looking at older work around autonomous systems and classical AI architectures, you begin noticing familiar tensions everywhere. Systems that prioritize planning versus systems that prioritize responsiveness. Architectures built around reasoning versus architectures built around reaction. Trade-offs between control, adaptability, speed, and autonomy.

So it made sense to follow up the previous piece with this one.

I’m not saying modern LLM agents map perfectly onto these older paradigms. Understanding deliberative, reactive, and hybrid architectures, however, makes today’s agent ecosystem feel a little less chaotic and a lot more interpretable.

This article explores those architectural foundations and how many of those ideas continue to echo through the systems being built today.

Join Angentic Engineering

For builders navigating the move from models to agents

If you strip things back, agentic systems are just different ways of organizing how an autonomous system perceives its environment, reasons about decisions, responds to changing conditions, and carries out actions toward a goal.

The interesting part is that there’s no single architectural approach for doing this well. Some systems prioritize planning and structured reasoning. Others are optimized for fast reactions and adaptability. And many modern systems attempt to balance both.

Broadly speaking, most agentic architectures tend to fall into three categories: deliberative, reactive, and hybrid architectures. Each one reflects a different philosophy around how intelligent systems should behave, make decisions, and interact with the world around them.

Let’s walk through them.

Deliberative architectures

Also known as knowledge-based or symbolic architectures, deliberative architectures rely on the use of explicit representations of knowledge and reasoning mechanisms to reach decisions. They typically follow a sense-plan-act cycle, where they first perceive information about the environment, then make a plan of action according to that perception and the knowledge base, and finally execute such plans of action.

The key advantage of deliberative architectures is their ability to handle tasks that involve complex reasoning, such as planning, problem-solving, and decision-making. These architectures leverage techniques such as rule-based reasoning, constraint satisfaction, and heuristic search to navigate through intricate problem spaces and formulate appropriate courses of action.

One of the critical components of a deliberative architecture is the knowledge base that stores symbolic representations of the environment, goals, constraints, and domain-specific knowledge. This knowledge base is typically encoded using formal language or logic, enabling the system to perform logical inference and reasoning. The sense-plan-act cycle in deliberative architectures typically involves the following steps:

1. Sensing: The agent perceives and acquires information about the environment through various sensors or input mechanisms.

2. Knowledge updating: The perceived information is used to update the agent’s internal knowledge base, ensuring that it maintains an accurate representation of the current state of the environment.

3. Planning and reasoning: Based on the updated knowledge base, the agent employs reasoning techniques and algorithms to formulate plans and make decisions. This may involve techniques such as constraint satisfaction, logical inference, search algorithms, or heuristic-based planning.

4. Plan execution: Once a plan or course of action has been determined, the agent executes the corresponding actions in the environment, potentially modifying the environment or achieving specific goals.

Deliberative architectures excel in handling tasks that require complex reasoning, planning, and decision-making in well-defined environments. They can effectively handle uncertainty and ambiguity through techniques such as probabilistic reasoning, fuzzy logic, or belief revision mechanisms.

However, deliberative architectures also have some disadvantages. One significant challenge is the computational cost associated with maintaining and reasoning over complex knowledge bases, which can limit real-time responsiveness in dynamic environments. Additionally, the explicit representation of knowledge can be challenging in domains where knowledge is difficult to formalize or is constantly evolving.

To address these limitations, deliberative architectures are often combined with reactive or behavior-based components in hybrid architectures, allowing both complex reasoning and rapid response to environmental changes.

Despite their limitations, deliberative architectures remain a crucial component in many intelligent systems, particularly in domains where complex decision-making, planning, and reasoning are essential, such as robotics, decision support systems, and intelligent tutoring systems.

Reactive architectures

Reactive architectures, also known as behavior-based or stimulus-response architectures, aim to provide immediate responses to stimuli from the environment. Unlike deliberative architectures, reactive architectures do not rely on explicit models of the world or complex reasoning processes. Instead, these systems directly map perceptions onto actions, typically using simple condition-action rules or neural networks as depicted in the following figure:

Some of the key properties and characteristics of reactive architectures include the following:

• Speed and responsiveness: Reactive architectures are designed to react rapidly to changes in the environment. By directly coupling perceptions to actions, they can bypass time-consuming deliberative reasoning processes, enabling swift and timely responses.

• Robustness and fault tolerance: These architectures are generally robust and less susceptible to noise or incomplete information. Their simple, standalone nature makes them less prone to catastrophic failures, as individual components or behaviors can compensate for or mitigate the effects of faulty or missing input, especially when used within a deliberative architecture.

• Handling uncertainty: Reactive architectures can effectively handle uncertainty in dynamic environments. Their ability to respond directly to environmental stimuli allows them to adapt and adjust their actions based on the current situation, without relying on precise or complete models of the entire world.

• Parallel and distributed processing: Reactive architectures often employ parallel and distributed processing using multiple reactive modules, where multiple modules operate simultaneously and independently. This decentralized approach enables efficient handling of complex tasks and provides inherent scalability and modularity.

• Emergence of complex behavior: Despite the simplicity of individual behaviors or rules, the interaction and coordination of multiple reactive components can lead to the emergence of complex, intelligent-like behavior at the system level.

While reactive architectures offer advantages in terms of speed, robustness, and handling uncertainty, they also have limitations as highlighted:

• Lack of long-term planning: Reactive architectures cannot generally plan or reason about long-term consequences. Their focus is on immediate responses to environmental stimuli, making it difficult to pursue complex, multi-step goals or strategies.

• Limited reasoning and abstraction: These architectures may struggle with tasks that require abstract reasoning, generalization, or the manipulation of symbolic representations. They are primarily designed to operate at a lower, stimulus-response level.

• Limited learning capabilities: Many reactive architectures cannot learn from experience or adapt their behavior over time. Their fixed set of rules or behaviors may not be suitable for dynamic environments or tasks that require continuous learning and adaptation.

Despite these limitations, reactive architectures are widely used in applications where real-time responsiveness, robustness, and the ability to handle uncertainty are essential, such as in robotics, video games built with AI, and control systems. Additionally, reactive architectures often serve as components within more complex hybrid architectures, complementing deliberative or learning-based systems to achieve desired levels of performance and adaptability.

Hybrid architectures

Researchers have recognized the strengths and limitations of both deliberative and reactive architectures, leading to the development of hybrid architectures that aim to exploit the advantages of both approaches. Such hybrid architectures typically employ a layered structure, consisting of the following:

• A reactive layer for fast and low-level responses. The reactive layer is responsible for handling real-time interactions with the environment, providing rapid and situationally appropriate responses to external stimuli. This layer is designed to be highly responsive, fault-tolerant, and capable of handling uncertainty, leveraging the strengths of reactive architectures.

• A deliberative layer for high-level reasoning and planning. The deliberative layer is dedicated to higher-level reasoning, planning, and decision-making processes. This layer can maintain a more comprehensive representation of the environment, goals, and constraints, enabling it to formulate complex strategies, reason about abstract concepts, and plan long-term courses of action.

The interaction between these two layers is crucial for enabling agentic systems to respond effectively to dynamic environmental contexts while maintaining the capability to plan actions and reason about them. The reactive layer can provide real-time feedback and situational awareness to the deliberative layer, informing its decision-making processes. Conversely, the deliberative layer can guide and influence the reactive layer’s behavior by providing high-level plans, goals, and constraints.

To achieve complex goals and leverage the strengths of both layers, hybrid architectures often employ the following techniques:

• Task decomposition: Break down complex tasks into subtasks that can be handled by the appropriate layer, with the reactive layer handling low-level, time-critical tasks and the deliberative layer focusing on higher-level planning and coordination

• Multiplan selection: The deliberative layer can generate multiple potential plans or strategies, and the reactive layer can dynamically select and execute the most suitable plan based on the current environmental conditions

• Planning with external modules: The deliberative layer can incorporate external modules or specialized algorithms for tasks such as path planning, resource allocation, or scheduling, leveraging domain-specific knowledge and techniques

• Reflection and refinement: The deliberative layer can reflect on the outcomes of executed plans, learn from experience, and refine its reasoning and planning processes accordingly, enabling continuous improvement and adaptation

• Memory-augmented planning: The deliberative layer can maintain a memory or history of past experiences, decisions, and outcomes, enabling it to leverage this knowledge in future planning and reasoning processes

By combining the strengths of both deliberative and reactive approaches, hybrid architectures seek to balance responsiveness and reasoning, enabling the development of more robust, autonomous, and adaptable agentic systems. These architectures leverage the power of both approaches, providing the ability to respond rapidly to dynamic environments while maintaining the capability for complex planning, reasoning, and decision-making.

The design and implementation of effective hybrid architectures remain an active area of research, as researchers strive to develop architectures that can seamlessly integrate and coordinate the deliberative and reactive components, enabling the creation of highly capable and intelligent agentic systems.

Selecting the appropriate architectural pattern for an agentic system is contingent upon the specific requirements of the application, encompassing factors such as task complexity, environmental uncertainty, and the necessity for real-time responsiveness. Deliberative architectures excel in scenarios that demand intricate reasoning and decision-making processes, while reactive architectures thrive in dynamic environments that necessitate swift and adaptive responses. Hybrid architectures strike a harmonious balance by judiciously leveraging the strengths of both paradigms, resulting in the development of more capable and adaptable agentic systems that can seamlessly navigate the complexities of their operating environments.

This article was adapted from Building Agentic AI Systems by Anjanava Biswas and Wrick Talukdar.

Reading through it, what stood out to me was how much of agent design ultimately comes back to timeless systems questions, like how decisions are made, how tasks are decomposed, how coordination happens, and how systems adapt under changing conditions.

That feels relevant right now as current conversations around agents still focus more on tooling abstractions than the systems underneath them. If you enjoy thinking about agents as systems-design problems rather than just autonomous demos, this book is worth exploring further.

Explore the book

However, most agentic systems tend to follow a small number of recurring design patterns. The differences usually come down to how an agent reasons, how it plans tasks, how it interacts with tools, and whether it operates alone or alongside other specialized agents.

Once you recognize these patterns, a lot of the ecosystem starts making more sense. You can begin to see why some systems are better suited for short reasoning tasks while others are designed for long-running workflows, delegation, or multi-step execution.

This article breaks down some of the most common patterns used in modern AI agents and how they differ.

Join Angentic Engineering

For builders navigating the move from models to agents

All AI agent design patterns contain the core three components, but have variations in how they implement the components. In particular, they differ based on how they control the control logic framework and its tooling sophistication. These include ReAct, CoT, planner-executor, and hierarchical/multi-agent.

CoT

CoT simply encourages the model to produce a step-by-step reasoning trace before generating a final answer. A literal interpretation of it would be to tell someone to think about a problem, create a plan on how they are going to solve it, and then take the steps to solve it. One limitation of CoT is that it’s unable to take action and, more importantly, unable to adapt its plan based on the results of actions.

ReAct (Reasoning + Acting)

ReAct is a pattern that we mentioned and described in pseudocode earlier. This design pattern enforces an AI agent to iteratively reason about the problem and choose a tool or action to take, feeding the result back into the continuous loop until its goal is achieved. The difference between ReAct and CoT is in their ability to act and then adjust their plan based on the results of that action. This is best for real-world agents that need to reason and interact with tools to solve complex tasks. This pattern is most often the go-to for most traditional AI agents.

Planner-execution

The planner-execution pattern separates the system performing the planning from the system performing the execution. An agent first generates a high-level plan, and then another agent (or multiple agents) executes each task in that plan. This pattern works well for very long-term complex tasks or tasks that can easily be modularized and delegated to sub-agents.

Hierarchical/multi-agent

A hierarchical/multi-agent pattern divides tasks among multiple specialized agents that are experts in that particular domain or task. This pattern can also perform work in a distributed manner, meaning that the system can finish tasks simultaneously rather than sequentially. This system is akin to a normal company, which contains a manager (a CEO) who delegates tasks to specialized workers with their own domain (CMO, COO, CFO, etc.).

For example, if we build an AI agent that resolves customer complaints, there may be an agent that receives the complaint and produces an action plan, and then specialized agents to perform those tasks, each with their own model, tooling infrastructure, and memory and knowledge (such as a customer agent that can look up customer information, a compliance agent that reviews policies, and a response agent that drafts a reply to the customer).

These patterns are not mutually exclusive. In fact, most real-world agents, especially those built with the frameworks briefly discussed next, tend to blend these approaches depending on the complexity of the task and the tools available. The OpenAI Agents SDK framework, for example, combined ReAct and hierarchical/multi-agent patterns.

In addition to the design patterns, companies have built different frameworks to develop and deploy AI agents, which differ based on the design pattern used and based on their feature set. These include the OpenAI Agents SDK, LangChain, LangGraph, AutoGen, AutoGPT, and Crew AI.

One thing that becomes clear when building agents is that there’s no single correct architecture.

That’s also part of why the agent space feels so noisy right now. Many of the frameworks being discussed often solve similar problems underneath, just with different abstractions and orchestration layers, as well as developer experiences wrapped around them.

Understanding the underlying patterns matters more than memorising framework names. Once you understand how reasoning, planning, tooling, and delegation fit together, it becomes much easier to evaluate new agent stacks without getting distracted by surface-level differentiation.

Thanks for reading!

Agentic Engineering grew out of a desire to create something grounded and useful for people trying to make sense of agentic AI beyond the weekly framework and abstraction noise.

If you’d like to support that direction, you may pledge your support below.

Pledge your support

That’s part of the reason I wanted to start this newsletter here, with fundamentals. Before the frameworks, abstractions, and “10x autonomous employee” headlines, it helps to understand the actual moving parts underneath these systems.

Join Angentic Engineering

For builders navigating the move from models to agents

AI agents follow a typical pattern and can be broken down into three fundamental components: model, tooling interface, and memory and knowledge.

Production-grade agents depend heavily on how well these components coordinate with one another. A capable model paired with poor tooling or weak memory management will, without a doubt, produce unreliable systems.

Model

The model is still the brain of the agent. It’s the part doing the reasoning, interpreting inputs, deciding what to do next, and trying to recover when things go sideways.

But the more you look at production systems, the harder it becomes to talk about models in simple leaderboard terms. The “best” model usually depends on what kind of compromises you’re willing to live with.

Some models are fast enough to feel seamless, but start struggling once tasks become more complex. Others reason well but move slowly enough to make the whole experience feel heavy. Some are great at coding, some handle multimodal workflows better, and some begin falling apart the moment you push them into long-running tasks with too much context.

And then there’s the strange reality that models don’t just differ in capability. They differ in personality almost. Different blind spots, different reasoning habits, different confidence levels, different failure modes. Some hallucinate more aggressively. Some become overly cautious. Some feel oddly persuasive even when they’re wrong.

Most AI agents are not powered by entirely different “agent models.” Many of them rely on the same families of frontier models, whether that’s GPT-class, Gemini-class, or Claude-class systems. What differs between these applications is not the model but the instructions provided to the model and how the model interacts with other components.

These “instructions” are provided to the model by defining a system prompt, which essentially defines the model’s role inside the system. When you really get down into it, a surprising amount of agent development involves refining prompts, adjusting tool descriptions, and reshaping interaction flows to make systems behave more reliably.

Control logic framework

Another important part of an AI agent is the control logic framework: the loop that lets the system observe what’s happening, reason about what to do next, take action, and then repeat that process until it reaches a goal.

What’s interesting is that this loop often does not happen “inside” the model itself. In many cases, it’s the surrounding agent framework that keeps the cycle running, feeding outputs back into the model, deciding when tools should be invoked, and determining whether another reasoning step is needed.

The process looks something like this:

Read user’s goal and create action plan

For each step in action plan:

Create action inputs

Execute action

Get result

Add result to memory

Modify action plan if necessary or if goal not achieved

If goal is achieved:

Return output to user

Different agent frameworks implement this loop differently, but most follow some variation of planning, acting, evaluating, and repeating. Patterns like CoT and ReAct show up constantly underneath modern agent systems.

Some frameworks even get a little strange with it. One pass generates the answer, and another pass comes back to critique the reasoning like an internal reviewer hovering over the system’s shoulder.

Tooling interface

One of the biggest differences between a normal LLM and an AI agent is that the agent can actually interact with the outside world instead of just generating tokens inside a chat window. That capability comes from the tooling interface, which gives the agent a structured way to decide when and how actions should be executed.

An email assistant, for example, might be equipped with actions that look something like this:

Action #1: Send an email

Description: Sends an email to a user

Parameters: to_email_address, email_subject, email_body

Action #2: List all emails

Description: Lists all emails, including email_id

Parameters: search_term (optional)

Action #3: Read an email

Description: Returns the content of an email

Parameter: email_id

Once these tools are available, the agent’s model and control logic framework decide when they should be used and what inputs they need.

An email assistant, for example, might search for emails related to a “coffee expansion proposal,” read through the relevant threads, generate a summary, and then send that summary to a manager. Underneath, the agent is simply chaining together a series of tool calls and feeding the outputs back into the reasoning loop.

The mechanics differ slightly depending on the framework being used, but the overall pattern stays fairly consistent. The agent needs awareness that the tools exist, a way to execute them, and a way to pass the results back into the model.

Memory and knowledge

Memory and knowledge are another part of what makes agents intelligent instead of just reactive. The two are related but slightly different.

Memory is about what the system remembers from interactions and previous context, while knowledge is about what it can retrieve from external sources when needed. In both cases, the goal is to give the model enough relevant context to make better decisions and produce more useful responses.

Let’s talk about each concept and how it’s used to provide more context to the AI agent. There are typically two types of memory: working and long-term.

Working memory

Working memory is basically the agent’s short-term memory: the running context from the current interaction.

It’s what allows systems like ChatGPT to understand that when you ask “How big is it?” after asking about the sun, the word “it” still refers to the sun instead of something completely unrelated.

Underneath, this usually works by continuously feeding recent conversation history back into the model as context. That’s what makes follow-up questions and multi-step conversations possible in the first place.

But this memory is limited. Context windows eventually fill up, older information starts dropping out, and frameworks have to decide what information is worth keeping around. A surprising amount of agent design ends up revolving around that question alone.

Long-term memory

Long-term memory is what allows an agent to carry information across sessions instead of treating every interaction like a completely fresh start. Without long-term memory, an agent is basically stateless: every request exists in isolation. With memory, the system can begin recalling preferences, past interactions, recurring tasks, or relevant context from earlier conversations.

This usually means storing information in some external database or memory layer that the agent can retrieve later through tools. So an email assistant, for example, might remember how you typically write messages, who you communicate with most often, or the kinds of follow-ups you usually prefer. Your AI agent may have the following tooling, which enables it to store and recall memories:

Action #1: Store information

Description: Stores important information about the user

Parameters: information

Action #2: Read information

Description: Retrieves important information about the user

Working memory is usually the easy part. Most systems already know how to keep track of the current conversation. Things get more interesting once agents need to remember information across sessions or retrieve context from external knowledge sources in real time. That’s usually where extra infrastructure starts creeping in: vector databases, retrieval pipelines, memory stores, ranking systems, all the machinery needed to stop the agent from feeling like it has permanent short-term memory loss.

Knowledge is slightly different from memory, though. Memory is about recalling previous interactions. Knowledge is about pulling in relevant information from outside sources like documents, databases, file repositories, or internal company systems.

And the moment agents start operating inside real workflows instead of isolated chats, that distinction starts mattering a lot more.

Training knowledge

Training knowledge refers to the information already baked into the model through its training data. It’s why an LLM can answer something like “How big is the sun?” immediately without needing to search the web or retrieve a document first.

That capability is still the thing that makes these systems feel a little unreal sometimes. A model can absorb enormous amounts of general knowledge and then reshape it fluidly around whatever the user is asking.

But once you move into AI agents, that baked-in knowledge starts becoming less useful on its own.

Agents usually need information that’s contextual, recent, proprietary, or tied to a specific workflow. And the model’s training knowledge can’t help much there. It doesn’t know what happened after its cut-off date, it can’t see your internal documents, and it definitely doesn’t know why your finance team suddenly renamed twelve folders last Thursday.

Which is why modern agents now depend on retrieving information dynamically instead of relying purely on what the model already knows.

Retrieved knowledge

Retrieved knowledge is where agents start becoming genuinely useful instead of just broadly knowledgeable. Unlike training knowledge, which is frozen at training time, retrieved knowledge happens in real time. The agent pulls relevant information from documents, databases, APIs, internal systems, or the web, depending on what the situation requires.

An HR agent is only useful if it can retrieve actual company policies instead of confidently inventing them. A sales agent needs live CRM context, not generic training data. The intelligence of the system comes less from what the model already “knows” and more from how well it can retrieve and use external context while operating.

Once you break agents down into these moving parts, the current ecosystem starts feeling less like a race toward “AI autonomy” and more like an ongoing experiment in coordinating increasingly complicated systems.

This article was adapted from Building Agents with OpenAI Agents SDK by Henry Habib, a practical guide to designing and building real-world agentic systems using the OpenAI Agents SDK.

The book goes much deeper into:

• tools and MCP integrations,

• memory and RAG architectures,

• multi-agent orchestration and handoffs,

• tracing and guardrails,

• production considerations,

• and hands-on projects that evolve from simple workflows into full agentic systems

If you’re trying to move beyond experimenting with agents and toward actually shipping them, it’s a genuinely useful resource.

Get the book

Over the past few months, we’ve had a growing suspicion that the AI conversation was changing underneath everyone’s feet. Your responses to last week’s survey confirmed it.

You told us what’s missing. What feels overhyped. What kinds of conversations actually matter. And across hundreds of responses, one thing became clear: you want more grounded coverage of AI agents and the systems emerging around them.

Let’s be real for a minute. “Agentic AI” is suddenly everywhere, but genuinely thoughtful coverage of the space feels strangely fragmented. There still isn’t a single place that consistently treats agentic systems as an engineering and operational transformation rather than just another trend cycle.

So with that, we’re announcing the rebrand of AI_Distilled to Agentic Engineering.

The AI conversation is maturing

This isn’t just a name change. It reflects where the AI landscape itself is heading and where many of you told us you already are. The broader “AI” lens made sense over the past few years while the ecosystem was still orienting itself. But the conversation is maturing now. The deeper builder audience emerging around AI infrastructure, orchestration, evaluation, memory, tooling, and autonomous workflows needs something more focused, more practical, and far less interested in demo culture.

We believe agents are becoming the next layer of software abstraction, and we want this publication to evolve alongside that reality.

What Agentic Engineering will focus on

With Agentic Engineering, we’ll still make the space accessible to curious newcomers, but we’ll also go much deeper into the real engineering trade-offs, debates, architectures, failures, workflows, and systems shaping this next phase of AI.

And importantly, we’re taking everything we learned from building AI_Distilled into this next chapter, including what not to do.

What happens next

So over the next few days, you’ll start seeing things change around here. And the next time this publication lands in your inbox, it won’t be from AI_Distilled.

It’ll be from Agentic Engineering.

Over the past few months, AI_Distilled has grown into a community of readers coming from very different parts of the AI ecosystem.

As the space continues evolving, we’ve been thinking carefully about what this publication should become going forward and how we can make it more genuinely useful for the people reading it.

So we’ve put together a short survey to understand what readers want more of, what feels missing from current AI media, and where we should take AI Distilled next.

It should take less than 4 minutes to complete, and every response will directly help shape the next phase of the publication.

Take the Survey

We really appreciate you being part of the AI Distilled community.

Sincerely,

Packt team