Solution
AT&T deployed a highly scalable AI platform built on Azure to handle millions of daily interactions across customer care and internal workflows. The system ingests customer context, historical chat histories, and operational data, storing long-term records in Azure DocumentDB and maintaining real-time responsiveness through Azure Cache for Redis. Multiple specialized agents collaborate via Microsoft Agent Framework, orchestrated through Azure Kubernetes Service, with Azure OpenAI in Foundry Models providing foundational AI capabilities. Azure AI Search retrieves verified customer knowledge, while agents autonomously update call notes and summarize context. Governance is enforced through Azure API Management for containerized agents, Azure AI Content Safety for safety guardrails, and comprehensive monitoring via Azure Monitor and Log Analytics. All compliance requirements are met through audit trails and transparency controls.
Data flow
Customer interactions and operational data are ingested from millions of daily touchpoints. Azure AI Search retrieves verified knowledge about customer accounts and issues. Multiple specialized agents collaborate via Microsoft Agent Framework, with one handling customer context retrieval, another summarizing information, and a third autonomously updating call notes. High-performance Azure Cosmos DB ensures near-real-time responsiveness, while Azure DocumentDB stores long-term chat history for future analysis. Azure Storage maintains records for governance and compliance. All agent decisions and outputs are monitored in real time via Azure Monitor and Log Analytics.
Solution architecture
8 components · 5 layers - Compute
- Azure OpenAI in Foundry Models Provides foundational LLM capabilities and model orchestration for specialized AI agents across customer care and operational domains.
- Storage
- Azure AI Search Retrieves verified customer content and contextual information to enable grounded, accurate agent responses.
- Azure Cosmos DB & Azure DocumentDB Optimized data layer providing high-performance chat interactions and long-term configuration storage with disaster recovery.
- Azure Cache for Redis Provides real-time responsiveness and low-latency data access for interactive agent conversations across millions of daily sessions.
- Orchestration
- Azure AI Foundry Orchestrates multi-agent workflows where specialized agents collaborate and communicate through Microsoft Agent Framework.
- Azure Kubernetes Service Manages containerized agent deployments and orchestrates enterprise-scale operations across distributed infrastructure.
- Governance
- Azure API Management Ensures governance, monitoring, and compliance controls across all agent API endpoints with traceability and performance metrics.
- Uncategorized
- Azure Storage Stores long-term records for governance, analytics, and compliance audit trails required by telecommunications regulations.
- Azure AI Content Safety filters, data governance policies, and model protection controls
- Azure API Management for governance and monitoring of containerized agents
- Azure Cosmos DB chosen for high-performance chat/agent apps
- Azure DocumentDB stores long-term chat history and master configurations
- Azure Kubernetes Service for container orchestration of agents
- Azure Monitor + Log Analytics + OpenTelemetry-compliant Arize AI for tracing and observability
- Legal, security, and finance review before launching every agent
- Multi-agent system orchestrated with Microsoft Foundry and Azure OpenAI
Approximately 9 billion tokens ... processed daily ... Over two times the free cash flow benefits saved annually through efficiency gains and automation.
Digital coworkers surface answers faster than the old systems, reducing the time spent searching for information during calls by 33% and cutting average handle time.
We chose Azure Cosmos DB because it is optimized for high-performance chat and agent apps, super-responsive user experiences, and scalability.
With Microsoft, we’ve deployed 71 unique generative AI solutions serving over 100,000 employees—every one of them governed, trusted, and delivering ROI.
