Azure AI Foundry: my honest first impressions

Azure AI Foundry is Microsoft's umbrella platform for building, deploying, and operating AI applications on Azure. It pulls together what used to be Azure OpenAI Studio, Azure ML, AI Search integration, and a model catalog into a single workspace.

I've been shipping projects on it for the past few months. Here's the honest take.

What it is, in plain terms

Foundry is three things in one:

A model catalog. OpenAI, Llama, Mistral, Microsoft Phi, and many others accessible through one inference endpoint pattern, deployed into your Azure subscription with proper governance.
A workspace for building agents. Tools, orchestration, evaluation, tracing all in one UI.
An ops surface for AI. Deployments, costs, content safety, monitoring.

If you've used Azure OpenAI Studio in 2024, Foundry is its grown-up sibling.

What's genuinely good

The model catalog is excellent for evaluating alternatives. Switching between GPT-4o, Llama, and Mistral is configuration, not refactoring. For teams trying to find the right model-task fit, this is real leverage.

Tracing works without being annoying. Foundry's tracing for agentic workflows captures the full conversation tool calls, retrievals, intermediate reasoning in a UI that's actually navigable. I've built three internal agents on it and the tracing has paid for itself debugging non-deterministic failures.

Built-in evaluators. A library of metric implementations (groundedness, fluency, similarity, custom rubrics) you can run on a dataset without writing them yourself. For teams just getting into evaluations, this is a faster on-ramp than building from scratch.

Identity story is solid. Managed identities, Entra ID-only auth, RBAC. Foundry handles the security model the same way the rest of Azure does. No bespoke API keys.

Pricing is transparent (mostly). You pay per Foundry resource, per model deployment, per inference call. No surprise platform fees on top.

What's frustrating

The UI is busy. There's a lot in Foundry, and the navigation hasn't quite caught up. New users routinely click "deploy" expecting one thing and getting another. The information architecture needs a pass.

Project sprawl. "Hub" vs "Project" vs "Workspace". These are distinct concepts, and the difference isn't always clear in the docs. I've ended up with abandoned hubs more than once when refactoring.

Documentation lags features. Foundry ships fast, which is great, but the docs sometimes describe a slightly older UI. Cross-reference with Microsoft Learn videos and the GitHub samples repo.

The "agents" abstraction is opinionated. Foundry has its own way of expressing agents, tools, and orchestration. If you've already built on LangChain or Semantic Kernel, mapping concepts isn't free. Don't assume the migration is mechanical.

Cost visibility per-project is awkward. You can see total Foundry cost easily; per-project attribution requires tagging discipline. Worth setting up early.

Where Foundry shines vs. doesn't

Use Foundry when:

You're already on Azure and want a unified AI platform.
You're shipping enterprise AI with compliance requirements (data residency, content safety, audit).
You need to compare and switch between models frequently.
You're building agents that benefit from Foundry's built-in tracing.
Your team already knows Azure ML or Azure OpenAI.

Don't use Foundry when:

You're prototyping on a small budget: direct Azure OpenAI calls (or OpenAI direct) are simpler.
Your stack is heavily multi-cloud: Foundry is Azure-shaped.
Your team is fully invested in another framework (e.g., a deeply customized LangGraph setup) and the migration cost outweighs the benefit.

The integration story

Foundry connects naturally with:

Azure AI Search for RAG. Configure once, point a project at it, retrieve.
Azure Storage / Cosmos DB for grounding data and chat history.
Application Insights telemetry plugs in.
GitHub / Azure DevOps Foundry projects can be configured via code (the full IaC story is improving fast but isn't complete yet).

Production checklist before shipping a Foundry app

Model deployment is using a stable, dated model version (not "latest").
Content safety filters are tuned for your domain.
Customer-managed keys are configured if compliance requires it.
Private Endpoints are on for the Foundry resources.
Managed identity is used for everything; no API keys.
Cost alerts are configured at the resource level.
Evaluations are running in CI on every prompt or pipeline change.

The summary

Foundry is good. Not perfect, but firmly past "early adopter only." For Azure-native organizations building real AI products, it's the path I'd choose today. The rough edges are real, and they're being filed down month by month.

If you're starting an AI project on Azure in 2026, look at Foundry first.