When we talk about building AI agents, the conversation often focuses on capability: can the agent research, write, analyze, plan, or execute the task?

But capability is only part of the problem.

Because LLM-based agents are probabilistic, their outputs are not perfectly consistent. The same agent, given the same type of task, can produce a strong result one time, an acceptable result the next, and a poor result after that.

That changes the agent engineer’s role.

The objective is not simply to build an agent that can perform a task. It is to engineer the system so that a sufficiently large proportion of its outputs consistently fall within the range the business considers acceptable.

In this installment of our Building in Public series, we look at agent performance as a distribution of possible outcomes, and at the engineering practices that can shift more of that distribution into an organization’s acceptable operating range.

Stochastic Nature of Agents

LLM-powered agents are probabilistic systems.  That means the same agent can produce an excellent result on one run, an acceptable result on the next, and a poor result after that.

So the real engineering challenge is not simply:  “Can the agent do the task?” It is:  “How often does the agent produce an output the business can actually accept?”

Think of agent performance as a distribution.

• Some outputs will be weak.
• Some will be strong.
• Most will fall somewhere in between.

The job of agent engineering is not to make the model produce its best possible output. It is to increase the probability that every execution produces an output inside the organization’s acceptable operating envelope.

Acceptable Outputs

Every organization has an acceptable operating range that indicates the outcomes the corporation is willing to accept as evidence that the agent completed its task successfully.

Run an agent 1000 times, and you will likely see some form of normal distribution.

Agent engineering is largely about shaping the distribution of outcomes so that a greater proportion falls inside the organization’s acceptable operating range.

Agentic Engineering

The job of agent engineering is to move more of the output distribution into that range.

That can mean:

• shifting average quality upward through better context, prompts, tools, and grounding
• reducing variability through SOPs, structured inputs, and clearer constraints
• catching the remaining failures through evaluation, retries, escalation, and human approval

This changes how we should think about agent quality.  The goal is not to produce the best possible output once.  The goal is to increase the probability that every execution produces an acceptable output.

In production AI, consistency matters as much as capability.