Indiana Pacers achieve 1.14% caption error rate with Azure AI live arena captioning
Pacers Sports & Entertainment built the NBA's first live in-arena captioning service using Azure AI Foundry, cutting speech-to-text error rates by 87% and reducing model training time from five days to two minutes.
Background
Traditional speech-to-text tools failed to handle basketball's high pace, unscripted announcer commentary, and specialized vocabulary (player names, play calls, sponsor mentions). No NBA or WNBA team had solved the problem of delivering live, accurate captions to fans at scale across both mobile and in-arena displays. PS&E saw it as both an accessibility imperative and a competitive differentiator.
What Was Implemented
- Custom Azure AI Speech model trained on hundreds of hours of Pacers and Fever broadcasts within Microsoft Azure AI Foundry
- Training dataset enriched with structured text files tagging player, coach, and official names
- Custom script compressing per-game audio preparation from five days to two minutes
- Event-driven real-time delivery architecture: Azure Functions + Azure SignalR Service
- Simultaneous caption delivery to mobile app (standard WebView) and arena screens at Gainbridge Fieldhouse
- Language expansion: English → Spanish → 12 additional languages (14 total)
- Built-in content moderation filters to prevent inappropriate or misinterpreted output from reaching screens
Results
Caption error rate: 1.14% (original target was 10%; 87% better than baseline). Model preparation time: compressed from 5 days to 2 minutes ( 99.97% reduction ). Languages supported: 14 . Captions live on Pacers, Fever, and All-Star games; architecture ready to scale to concerts, comedy, and family events.
Lessons
- Domain-specific custom model training is essential for AI speech in specialized vocabulary environments; generic speech-to-text cannot match broadcast-quality accuracy for sports
- Starting with English and iterating to additional languages is a viable and sustainable expansion path
- Serverless, event-driven architecture (Azure Functions + SignalR) enables low-latency synchronization across multiple display surfaces from a single endpoint
- Building for accessibility can simultaneously deliver a competitive fan-experience differentiator