Microsoft Foundry agents vs. custom orchestration: how I choose
Foundry's agent framework removes a lot of plumbing — but not every AI feature needs an agent. Here's the decision framework I use before reaching for either.
Since Microsoft Foundry made it straightforward to spin up agents with tools, memory, and orchestration baked in, I keep getting asked the same question: "should we just build everything as an agent now?" The honest answer is no — and knowing when not to reach for an agent is the more valuable skill.
Start by asking: does this task have a fixed shape?
If your workflow is "take input A, call tool B, transform with model C, return D" — every single time, in that order — you don't need an agent. You need a pipeline. A deterministic chain of calls is easier to test, cheaper to run (no wasted reasoning steps), and far easier to debug when something goes wrong at 2am.
Agents earn their keep when the path through the task varies based on what the model discovers along the way — when step 2 depends on what step 1 returned, and you can't enumerate every branch in advance.
What Foundry's agent framework actually buys you
When an agentic approach is the right call, Foundry removes meaningful plumbing:
- Tool registration and calling — defining functions/tools once and letting the model select and chain them.
- Thread-based memory — conversation state persisted without you wiring up your own store.
- Built-in connectors — Azure AI Search, Bing grounding, code interpreter, without custom integration code.
- Tracing — every tool call and reasoning step is visible in the portal, which matters enormously when you're debugging why an agent did something unexpected.
That last point is underrated. The hardest part of shipping agents isn't building them — it's understanding their behavior in production. Foundry's tracing turns "the agent did something weird yesterday" into an actual investigation.
Where I still build custom orchestration
- High-throughput, latency-sensitive paths. Agent reasoning loops add latency and token cost per request. If you're processing thousands of items a minute, a tuned pipeline beats an agent on cost and speed.
- Anything safety-critical with a known failure mode. If a wrong answer has real consequences (financial, medical, legal), I want a deterministic chain with explicit validation steps — not a model deciding what to do next.
- Where the "tools" are really just simple API calls in sequence. Wrapping three sequential REST calls in an agent adds reasoning overhead for no benefit.
A rule of thumb that's served me well
Build the simplest version first — a plain pipeline. Run it against real traffic. Only reach for an agent when you can point to specific cases where the fixed pipeline fails because the right next step genuinely depends on context the model needs to reason about. That gives you a concrete justification (and a test set) rather than "agents are the new thing, let's use them."
Closing thought
Foundry makes building agents dramatically easier — which is exactly why it's worth being more deliberate, not less, about whether to build one. The framework removing friction from agent development doesn't change whether your problem needs an agent. Match the tool to the shape of the problem, not the other way round.