What is Microsoft Foundry?
A practical guide for engineering teams
Microsoft Foundry (previously Azure AI Foundry / Azure AI Studio) is Microsoft's unified Azure platform for building, testing, deploying and governing generative-AI applications and AI agents. Rather than being a single AI model, it is an engineering platform that brings models, agent runtimes, tools, evaluation, monitoring, identity and governance into one managed environment.
Why engineering teams should care
A prototype that calls an LLM API is relatively easy to build. The harder engineering problem is turning that prototype into a production service: choosing and changing models, securely connecting company data, controlling access, measuring output quality, tracing failures, managing deployments and applying enterprise security policies. Foundry is intended to provide that surrounding infrastructure.
The main building blocks
Models. Foundry provides a model catalog covering Microsoft and third-party models, including models from OpenAI, Anthropic, Meta and others. Teams can compare models and select an appropriate model for cost, latency, capability and deployment requirements rather than designing the application around one provider.
Agents. Foundry Agent Service supports AI systems that do more than return text. An agent combines a model with instructions and tools so it can retrieve information, call functions and perform multi-step tasks. Teams can use managed prompt agents or deploy their own code as hosted agents when greater control is required.
Tools and enterprise knowledge. Applications and agents can be connected to tools, retrieval systems and business data. This is the basis for patterns such as retrieval-augmented generation (RAG), where a model answers using approved internal documents rather than relying only on its training data.
Evaluation and observability. Foundry includes facilities for evaluating AI applications and tracing their behaviour. This matters because conventional unit tests are not sufficient for probabilistic AI output. Teams can establish test datasets, compare versions, measure quality and investigate which model or tool calls produced a result.
Security and governance. Because Foundry is integrated with Azure, engineering teams can use Microsoft Entra identity, role-based access control (RBAC), network isolation, content-safety controls and Azure Policy. This is particularly useful when AI systems need access to confidential engineering or business information.
What does a typical solution look like?
Consider an engineering assistant that helps developers diagnose test failures. A user asks a question through a web application. The application sends the request to a Foundry project, where an agent uses an appropriate language model. The agent may search approved design documents, query a test-results service or call an internal API. It then returns an answer with the relevant context. Foundry can provide the identity, model endpoint, tool integration, tracing and evaluation around that workflow.
Foundry versus simply using Azure OpenAI
If an application only needs to send a prompt to a model and receive a response, a direct model call may be all that is required. Foundry becomes more valuable as the system grows: multiple models, agents, retrieval, tools, evaluation, observability and governance can be managed as parts of the same platform. Microsoft also supports upgrading existing Azure OpenAI resources to the newer Foundry resource model while preserving existing endpoints and state.
A sensible adoption path
- Start with one concrete engineering use case and make a simple model call.
- Create a small, representative evaluation set before adding complexity.
- Add company knowledge through retrieval only where the use case needs it.
- Introduce an agent when the application genuinely needs tools or multi-step orchestration.
- Add tracing, monitoring, RBAC and network controls before moving into production.
- Treat prompts, agent instructions and evaluation datasets as versioned engineering assets and deploy them through a repeatable CI/CD process.
Where Foundry fits - and where it does not
Foundry is most useful when an organisation wants to build production AI applications on Azure and needs a managed path from experimentation to operation. It does not remove the need for normal software architecture: the application still needs well-designed APIs, data ownership, authentication, testing, logging and cost controls. Likewise, an agent should not be added simply because it is available; deterministic application code is usually better for tasks that have fixed rules and predictable inputs.
Summary
For an engineering team, the simplest way to think about Microsoft Foundry is as an AI application platform rather than an AI model. It gives developers access to models and agents, but its real value is the production framework around them: tools and knowledge integration, evaluation, observability, security and governance. A good starting point is a narrow use case with measurable value, followed by incremental addition of retrieval, tools and agents only when they solve a real engineering requirement.
Further reading
- Microsoft Learn - What is Microsoft Foundry?
- Microsoft Learn - Foundry product and capability map