Agentic workflow design is the work of deciding which steps in a process a language model may decide for itself, which steps software controls, and where a human signs off. The best designs use the least autonomy that gets the job done: a fixed chain where the path is known, an agent loop only where the path genuinely depends on what the model finds along the way. Below I cover the patterns, how to choose between one agent and many, a build process you can follow, and how to test the result.
Key Takeaways
- Start with the least agentic design that works. A fixed sequence of model calls beats an autonomous agent when the steps are known in advance.
- Add more agents only for real boundaries: separate permissions, separate context, true parallelism or independent verification. Distinct job titles alone are not a reason.
- Decide in advance where model decisions stop and ordinary code takes over. Put human approval on anything irreversible, sensitive or costly.
- Build evaluations before the agent gets live access, and track cost per successful task, not just accuracy.
- Ship narrowly, watch traces, and widen autonomy only when production evidence supports it.
Table of Contents
- What makes a workflow agentic
- Workflow, agent or both
- Orchestration patterns and when to use each
- One agent or several?
- A design process you can follow
- Tools, data and human review
- Evaluating quality, cost and failure risk
- Perspective
- Work with me on your agentic workflow
- FAQ
- Sources
What makes a workflow agentic
A traditional workflow follows rules you wrote down. Step one, step two, a branch, an end. An agentic workflow decides at runtime. Microsoft's documentation says agentic workflows can adapt when unexpected events happen, choose tools based on the prompt and available data, and handle unstructured data. Non-agentic workflows, by contrast, work best in stable environments with predefined rules and repetitive tasks.
IBM describes agentic workflows as AI-driven processes where agents make decisions, take actions and coordinate tasks with minimal human intervention, using reasoning, planning and tool use. That is the contrast with classic robotic process automation, which follows predefined rules.
Under the hood, a single agent is a loop. Anthropic's guide describes it as perceiving the environment, deciding the next step and acting, repeating until the task is done or a stopping condition such as "pause for human review" is reached. Neo4j breaks the same idea into four parts: planning, tool use, reflection and orchestration.
That loop is the part to design carefully. A model that can choose its own next action is useful, but only inside limits you set. If you are planning content or research work, an orchestration-first planning workflow shows the same cycle applied to a different domain.
Workflow, agent or both
Most real systems are hybrids. A useful rule: if you can draw the flowchart before the model runs, draw it and let code run it. Use the model for the nodes where input is messy or the next step truly depends on context.
Wayfair's engineering team documented this in their supplier-ticket system. They used a deterministic workflow where one node was itself a ReAct agent that looked up the supplier ID. That gave guaranteed ordering of operations plus flexible error handling where it mattered. Their reported results were 93% accuracy on question-type identification versus 75% for humans across 81 categories, 98% on language and 88% on supplier ID.
The lesson for your own process: the agent is one node, not the whole building.
Orchestration patterns and when to use each
Stanford's CS 224G lecture names seven building blocks. Here they are with the situation each one fits.
| Pattern | How it works | Use it when |
|---|---|---|
| Prompt chaining | A fixed sequence of model calls | The steps are known and ordered |
| Routing | Classify the input, send it to a specialist | Different inputs need different handling |
| Parallelization | Independent tasks run at the same time | Subtasks do not depend on each other |
| Orchestrator-workers | A lead model delegates dynamically | You can't know the subtasks upfront |
| Evaluator-optimizer | Generate, test, feed back, retry | You have a clear pass/fail check |
| ReAct | Reason, act, observe, repeat | The path depends on tool results |
| ReWOO | Plan upfront, execute in parallel | The plan is stable and tools are slow |
The evaluator-optimizer pattern is the easiest to picture. In the Stanford material, a coding agent writes code and the evaluator runs unit tests. If the tests fail, the feedback goes back to the generator for another try. It works best when the check is objective.
Multi-agent shapes
OpenAI's practical guide splits orchestration into single-agent systems, where one model with tools runs in a loop, and multi-agent systems, where work is spread across coordinated agents. It names two multi-agent designs:
- Manager: a central agent calls specialist agents as tools and keeps the conversation.
- Decentralized: peer agents hand tasks to one another based on specialty.
Microsoft's Azure Architecture Center describes the handoff pattern as a fit when the number or order of agents can't be set in advance, or when expertise needs emerge during processing. Full control passes from one agent to the next.
The difference between manager and handoff matters in practice. Airbyte notes that in a manager pattern the delegating agent owns the final response and treats specialist output as input, while in a true handoff the specialist takes over. Mix the two up and you get two agents who each believe they are answering the user. Write a handoff contract for every transfer: what context moves, who owns the reply, and what counts as done.
One agent or several?
This is the decision most teams get wrong, usually in the direction of too many agents. The guidance from the major vendors is consistent.
Microsoft's Cloud Adoption Framework says to use multiple agents when security or compliance boundaries, separate teams managing knowledge areas, or planned growth beyond three to five distinct functions require separation. Distinct roles such as planner, reviewer and executor do not justify it on their own. Prototype with one agent first, and move on only when testing shows limits that prompting, retrieval or policy controls can't fix.
Arize frames it the same way: one agent is the baseline, and you add coordination only when there is a real permission, context, parallelism or verification boundary one agent should not own alone. It also names the failure modes. Multi-agent systems tend to lose context, duplicate work, or let a failure cascade across handoffs. A single agent's typical failure is one unchecked mistake.
There are cases where multiple agents win clearly. Anthropic reports that its multi-agent research system, with a lead agent and subagents, outperformed a single agent by 90.2% on internal research evaluations. It also consumed roughly 15 times more tokens than chat. That trade only makes sense for high-value tasks with heavy parallelization or information that exceeds one context window.
A practical test, with a few questions to ask before you split anything:
- Does any step need different permissions or data access than the rest?
- Are there subtasks that can truly run in parallel?
- Does the context of one step pollute the others?
- Is the tool set too large for one agent to pick from reliably?
- Does the value of the task justify several times the token cost?
If every answer is no, stay with one agent and a good set of tools. It is cheaper, faster and easier to debug.
A design process you can follow
The published frameworks differ in naming but agree on the order. This is the sequence I use, drawn from them.
1. Define the outcome and the boundaries
Write down what the agent should do, what it must never do, who uses it and how it is invoked. Glean's design guidance says a clear brief prevents scope creep, and that success criteria, such as accuracy on the ten most common use cases, should be set before building starts.
2. Map the real process
Walk through how a person does the work today, including the exceptions. Mark the steps that are fixed and the steps that need judgment. Only the second group is a candidate for agent behavior.
3. Pick the smallest architecture
Choose from the pattern table above, starting at the top. Prompt chaining first, routing next, an agent loop only if the path truly varies.
4. Define tools as contracts
Each tool gets a narrow purpose, typed inputs, clear errors and the minimum permissions. Read-only wherever possible. If your agents browse the live web, they will hit anti-bot defenses, and proxies for AI agents are one practical way to keep that tool reliable.
5. Add approval and stop rules
Set retry limits, budget limits and conditions that force a pause. More on this below.
6. Build evaluations before live access
The Bardeen.ai walkthrough puts evaluations ahead of live access, then a narrow deployment with measurement. I agree with that order.
7. Roll out in stages
One published lifecycle suggests a progression: offline evals with mocked tools, internal use with read-only tools, a limited pilot with human approval on side effects, a canary with tight budgets, then broader rollout, with higher autonomy only after production evidence justifies it.
Tools, data and human review
A model with write access can do real damage, so the boundary between model and software is where design effort pays off.
OpenAI's guide recommends human intervention when an agent exceeds failure thresholds such as retry or action limits, and when actions are high-risk, sensitive, irreversible or high-stakes. Stanford's lecture gives concrete cases: sending emails, committing code and financial transactions. The implementation is checkpointing: pause, show the action to a person, resume on approval.
Security controls matter more than the number of agents. A peer-reviewed study of capability-fenced orchestration reduced indirect prompt injection success from 71% to 11% and excessive-agency violations from 23 to 2 per 100 tasks. The cost was about 2.7 times the spend and 3.7 times the latency of a single-agent baseline. The authors attribute the gains to specific runtime controls such as capability fencing, validation gates and verify-before-commit, not to the multi-agent structure itself. Removing fencing pushed attack success back up to 57%.
So the takeaway is practical: fence what each agent can touch, validate outputs before they commit, and verify before side effects. For a hands-on view of the approval step, see my write-up on shipping a human-in-the-loop MVP. For wider safeguards, the five AI guardrails for SaaS founders cover what to put in place first, and keeping raw PII out of prompts matters whenever agents see customer data.
If you serve EU customers, architecture choices also carry legal weight. My EU AI Act checklist for founders is a starting point.
Evaluating quality, cost and failure risk
An agent that works in a demo can still fail on the tenth run. Label Studio reports GPT-4-based agents falling from a 60% success rate on a single run to a 25% consistency rate across eight runs. It also reports 50x cost variation between agents with identical accuracy, from $0.10 to $5.00 per task. Measuring accuracy alone, it says, can yield agents 4.4 to 10.8 times more expensive than cost-aware alternatives.
Four things to measure, following Kesho Partners' framework:
- Whether the task outcome is correct
- Whether the execution trace and tool calls were valid
- Safety and security under adversarial and repeated attempts
- Reliable, economical production performance
Their cost metric is simple and useful: automated cost plus human operating cost, divided by correctly completed tasks. A cheap agent that needs constant human cleanup is not cheap.
Treat evaluation as a loop, not a gate. Every serious incident should leave behind a test case, and a model upgrade, prompt rewrite or tool change should run the regression suite before rollout, as Turitto's lifecycle notes argue. Arize says the same of coordinator changes: compare against a baseline on identical cases and turn confirmed coordination mistakes into regression tests.
For the practical details, see my pieces on AI evaluation metrics for production LLMs, production monitoring and reducing hallucinations beyond retrieval.
Perspective
I build software for founders and owner-operated businesses, and the same pattern repeats with agents. People arrive wanting a swarm of autonomous agents, when the process they need to fix has four known steps and one messy one.
My advice is to be boring on purpose. Write the flow as ordinary code. Give the model the one step where judgment is genuinely needed. Put a person on the actions that can't be undone. That design is cheaper to run, easier to test and easy for the next developer to pick up, which is why I work in widely known tools like TypeScript, Node and SQL.
I also think the evaluation work should be scoped and priced upfront like the build itself, not left as an open-ended extra. A fixed scope forces the useful questions early: what counts as success, what must the agent never do, and who approves what.
If a plain automation or an off-the-shelf tool fits your case, I will tell you so before you spend anything.
Hanad Kubat
Work with me on your agentic workflow
If you have a process you want to hand to an agent, I design and build it end to end as one senior engineer, at a fixed price with the code yours from the first commit. Projects start from €12,000, and you see working progress before you pay. Take a look at what I build and tell me which workflow is eating your time.
FAQ
What is agentic workflow design?
It is the practice of deciding which parts of a process an AI model controls, which parts fixed code controls, and where humans approve. It covers the pattern, the tools, the limits and the tests. The goal is a system that adapts where it must and stays predictable everywhere else.
How do I build an agentic workflow?
Define the outcome and boundaries, map the real process, and choose the smallest architecture that works. Then define your tools, add approval and stop rules, and write evaluations before the agent has live access. Roll out narrowly and widen autonomy as production evidence builds.
When should I use multiple agents?
Use them when you have real boundaries: different permissions, separate context, true parallel work or independent verification. Microsoft advises prototyping with a single agent first. Distinct roles alone do not justify the extra cost and coordination.
What are examples of agentic workflows?
Supplier-ticket triage is a documented one: a fixed workflow with a ReAct agent handling the supplier-ID lookup. Others include research assistants that split a question across parallel subagents, and coding agents that write code and run tests until they pass. In every case a person approves the risky actions.
Sources
- What are AI Agentic Workflows? (Microsoft Learn)
- What are Agentic Workflows? (IBM)
- Building Effective AI Agents (Anthropic)
- What are agentic workflows? (Neo4j)
- Automating Supplier Ticket Management with LLM Agents (Wayfair)
- Agent Orchestration & Workflow Design (Stanford CS 224G)
- A Practical Guide to Building AI Agents (OpenAI)
- AI Agent Orchestration Patterns (Azure Architecture Center)
- Multi-Agent Workflows: Architecture and Shared Context (Airbyte)
- Single-agent or multiple agents (Microsoft Cloud Adoption Framework)
- Multi-agent orchestration: how to evaluate agent teams (Arize)
- Building a Multi-Agent Research System (ZenML summary of Anthropic)
- Plan and design your agent (Glean)
- How to Build an AI Agent in 9 Steps (Bardeen.ai)
- Agent Development Lifecycle (Agentic Systems Patterns)
- Capability-Fenced Orchestration of LLM Multi-Agent Systems
- Do you need an agent evaluation framework? (Label Studio)
- How to Evaluate AI Agents in Production (Kesho Partners)
- Proxies for AI agents