Building an agentic team that operates autonomously while still producing results within an acceptable range of outputs is neither straightforward nor easy. Today, we want to share one agent pattern we have found useful. This, of course, is not the only way to do this, and each platform you use may have capabilities that allow for different strategies. The takeaway is the capabilities we’ve identified that work for us.
- Planning, orchestration, and adaptation.
- Standard Operating Procedures.
- Production Guidance.
- Release Management
- Corporate Knowledge
Acceptable Range of Outputs
In a previous article, we covered how agents can produce a range of outputs from very poor to excellent, and our job as designers is to ensure that most of the range of outputs, if not all, lie in an acceptable range.
The design pattern we will cover today is one technique we use to shift the distribution into a well-engineered system. You’ll see how the agents have well-defined responsibilities to keep the team on task and ensure the output meets corporate standards.
Is it perfect? No. Is it easy? Also no. But I think you will see how it is an idea worth exploring and worth the time to implement with your teams.
Overview of the Pattern
The pattern abstracts several functions an agentic team needs to run effectively. The diagram below shows the agents used with the pattern. This diagram shows only the agents in for our discussion. Other agents that provide the business/operational component would also reside under the Project Manager Agent, but are not shown.
- Project Director: Reviews the agent team’s capabilities, builds a plan, and updates and executes it to complete the assigned task.
- SOP Manager: Guides the Project Director’s plan by providing the Standard Operating Procedures used by the company.
- Production Advisor: Guides agents, allowing them to complete assigned tasks without interrupting a human for trivial issues. Escalates when the issue justifies human involvement.
- Company Knowledge Anchor: Centralizes company knowledge (ICP, value proposition, case studies, market support data) in a single agent to ensure that all agents can ground their output in the latest up-to-date information.
- Production Manager: Reviews all outgoing content to ensure it meets corporate policies and guidelines. Returns content for rework when it does not meet standards. Release Manager might be a strong name.
Together, these agents let agentic teams run autonomously while keeping actions within corporate boundaries and ensuring only relevant escalations are sent for human review. With this agent pattern, an architect controls how the agent team operates, including how much autonomy it has.
Now you might implement this as a document or knowledge base that each agent can access, and that is a valid option. In fact, this was our initial design. However, by giving an agent responsibility for serving up this information, we found we could determine the best way to use it and identify gaps. As we all know, agents are well known to fail silently. With this pattern, we can view the interactions and make adjustments as needed. Testing was easier, and overall the agents behaved more predictably and reliably; their output distribution shifted into the acceptable zone.
Our objective with this pattern is to separate concerns so we can reuse the agents in many work contexts.
Project Director
The project director leads a team of agents and is accountable for completing the assigned task. This agent orchestrates, plans, adjusts work, and engages the correct specialist to perform the work. The director evaluates the specialist agent’s output to judge how it advances progress toward completing the task.
This agent must be transparent about its plan, the rationale for selecting specialist agents, how it evaluates the agent’s output, and how it updates its plan accordingly.
The first step this agent performs is to determine if one or more standard operating procedures apply to the task at hand. This is critical to keep the agent grounded in corporate policies and prevent rogue behaviour.
SOP Manager
When writing agents, a common trap is distributing or duplicating corporate procedures across multiple agents. A centralized SOP manager clarifies which procedures apply and makes changes easy.
To be clear, an SOP is a high-level procedure for how the business completes work, not the specific steps a specialist performs. For example, a procurement SOP might define the rules for procurement requests and approvals. It doesn’t list which agents perform these tasks or how they do them; that is the agent’s territory.
The SOP manager centralizes the corporate Standard Operating Procedures.
Production Advisor
During request processing, agents can get stuck and may not be able to move forward. They may need specific information and not know where to get it, or they may have to choose from a set of options.
Of course, agents can be written to manage this situation themselves, but they may fail silently, and the designer may never realize there is a weakness in the system.
Centralizing this in a production advisor offers the following benefits.
- The weakness is exposed, as the engineer can see the request and take steps to improve the system.
- The decision-making process is centralized, ensuring decisions are made consistently.
- Updates to the decision-making process are made quickly.
Corporate Knowledge Anchor
A similar pattern is repeated with the corporate knowledge anchor. Company information (ICP, messaging, value proposition, mission, vision, marketing validation data, etc.) is maintained within a single agent. Agents that need corporate information request it from the Corporate Knowledge Anchor.
The same benefits and patterns repeat: centralized information, gap and weakness identification, and reliable, transparent information processing.
Production Manager
The production manager reviews all outgoing content to ensure it aligns with corporate policies and is grounded in corporate knowledge. This agent recommends what is needed to gain approval. Only after approval is the content released.
Model Diversity
If you recall from previous posts, selecting the right LLM model can improve agent reliability. Here we highlight an adjacent idea: model diversity produces better, more reliable outputs across a multi-agent ecosystem.
Using models from multiple vendors appears to improve performance. My theory is that an agent using an OpenAI model is checked by an agent using an Anthropic model, correcting for model-specific variation.