What It Is
A reference implementation showing how to deploy Ollama and Open WebUI on Azure Kubernetes Service (AKS). Built for Azure organizations, this implementation demonstrates infrastructure-as-code patterns for running open source AI models in your own cloud environment, using the same basic architecture we've implemented with our customers.
Available now on GitHub.
Why We're Open Sourcing It
Our perspective on open source AI infrastructure:
Practical Learning: Running AI infrastructure requires working through many practical considerations. Reference implementations help teams understand these challenges directly.
Infrastructure Patterns: While many organizations are exploring open source AI models, there's a gap between model availability and production deployment patterns. This reference shows one approach to bridging that gap.
Market Development: We believe LLMs will become commoditized and permissively licensed open source solutions will become standard. Organizations need examples of how to own their AI infrastructure and intellectual property.
Starting Point: This implementation represents one point in time in a rapidly changing landscape. Teams can use it as a foundation and adapt it as their needs and the technology evolve.
How It Works
The implementation uses:
- OpenTofu: Declarative infrastructure-as-code
- Azure Kubernetes Service (AKS): Managed Kubernetes platform
- Kubernetes: Container orchestration for the application components
The templates demonstrate automated provisioning and configuration of the necessary infrastructure components.
Getting Started
Getting started is simple and requires only a few steps:
git clone https://github.com/fmops/azure-private-ai-template
cd tofu && tofu apply
echo "$(tofu output kube_config)" > ../kubeconfig
cd .. && kubectl apply -R -f k8s/See the project readme on GitHub for details.
Implementation Notes
- Built following standard Azure and Kubernetes patterns
- Demonstrates basic security configurations
- Shows common scaling approaches
- Uses infrastructure-as-code for repeatability
- Implements resource management via Kubernetes
This reference implementation is intended as a starting point for teams exploring private AI infrastructure deployment patterns.
Need Help on Your AI Journey?
BlueTeam AI helps organizations build and operate private AI infrastructure. From initial deployment to production scaling, our team of Azure and AI infrastructure experts can guide your implementation. Contact us to learn more about our services.