Technology Overview

In 2025, implementing AI in retail has become less about algorithms and more about infrastructure. The difference between success and failure often comes down to mundane details: how quickly you could process customer data, whether your models could respond in under 100 milliseconds, and how much you were willing to spend on GPUs. The modern AI stack has evolved into a complex assembly of specialized components, each solving problems that didn’t exist five years earlier.

Every AI system started with data, but retail data was particularly messy. Product descriptions mixed languages, measurements, and marketing-speak. Customer reviews contained emojis, sarcasm, and spelling errors. Inventory systems used different SKU formats across warehouses. Before any model could learn, this chaos needed structure.

Instacart faced this challenge at massive scale in 2021. The delivery service processed 50 million grocery products from thousands of stores, each with different naming conventions. A simple item like bananas might be listed as “Bananas - Yellow,” “Organic Bananas (bunch),” “banana’s,” or “BANANA FRUIT EA.” Instacart’s data pipeline, built on Apache Beam, standardized these variations in real-time, processing 2 billion events daily.

The pre-processing layer had become sophisticated enough to handle edge cases that would have broken earlier systems. Home Depot’s 2022 pipeline could process a customer uploading a photo of a broken faucet part, extract text from any visible labels using optical character recognition (OCR), identify the manufacturer through logo detection, and cross-reference against 3D computer-aided design (CAD) models to find exact replacement parts. This entire chain completed in under three seconds.

Training modern AI models required computational resources that would have been unimaginable a decade earlier. In 2012, the AlexNet model that revolutionized computer vision was trained on two GPUs over six days. By 2025, training a retail foundation model required thousands of GPUs running for months.

NVIDIA’s A100 GPU, released in 2020, became the workhorse of AI training. Each chip could perform 19.5 trillion floating-point operations per second. But single GPUs weren’t enough. Amazon’s 2023 product understanding model was trained on a cluster of 4,000 A100s, consuming 15 megawatts of power, enough to supply 11,000 homes.

Google had taken a different path with its Tensor Processing Units (TPUs). Designed specifically for machine learning, TPUs sacrificed flexibility for speed. Where GPUs could handle any parallel computation, TPUs were optimized for the matrix multiplications at the heart of neural networks. Shopify trained its 2023 merchant assistance model on Google’s TPU v4 pods, achieving 40% faster training than equivalent GPU clusters at 30% lower cost.

Training models was just the beginning. Serving predictions to millions of customers in real-time required different infrastructure entirely. GPUs, essential for training, were often overkill for inference. NVIDIA’s 2022 Jetson Orin NX brought AI to the edge, enabling Kroger to run product recognition directly on smart shopping carts. Each cart could identify products as customers added them, suggest recipes, and track spending, all running on a chip that consumed just 15 watts.

CPUs remained relevant through clever optimization. Intel’s 2021 OpenVINO toolkit enabled Macy’s to run its style recommendation model on standard Xeon processors, achieving 90% of GPU performance at 20% of the cost for inference workloads. The key was quantization: reducing model precision from 32-bit to 8-bit integers, which CPUs handled efficiently.

Neural processing units (NPUs) represented the next evolution. Apple’s M1 chip, with its 16-core Neural Engine, enabled on-device AI that never left the customer’s phone. The Shopify mobile app used this for visual search, processing images locally without sending them to servers, addressing privacy concerns while eliminating network latency.

The modern AI stack included an arsenal of optimization techniques that made seemingly impossible deployments practical. Caching was the simplest but often most effective. eBay’s 2023 search system cached embeddings for its 1.7 billion active listings, reducing computation by 85% for common queries.