Technology Overview

Mixture of Experts

Google’s 2021 paper “Switch Transformers: Scaling to Trillion Parameter Models” introduced a clever solution to the size problem. Instead of every request using every parameter in a massive model, Mixture-of-Experts (MoE) models activated only relevant sections. Think of it as a department store where customers were automatically routed to the right specialist rather than consulting every employee.

The architecture was elegantly simple. A small “router” network looked at each input and decided which expert networks to activate. When Shopify implemented MoE in 2023, its model had 128 experts but only used eight for any given request. A query about “vintage wedding dresses” activated experts specializing in formal wear, vintage fashion, and wedding planning, while ignoring experts for electronics, home goods, and children’s products.

This selective activation changed the economics of large models. Alibaba’s M6-MoE model had 10 trillion parameters but only used 100 billion per request, achieving the quality of a massive model at the cost of a modest one. During its 2023 Singles Day sale, the system handled 500,000 queries per second by distributing different experts across different servers, with the router ensuring requests went to machines with the relevant specialists already loaded in memory.

The approach worked particularly well for retail’s diverse needs. Best Buy’s 2024 model had distinct experts for technical specifications, compatibility checking, installation requirements, and warranty terms. A customer asking, “Will this graphics card work with my Dell?” activated only the compatibility expert, reducing response time from 200 milliseconds to 30 milliseconds.

Retrieval Augmented Generation (RAG)

Foundation models had a fundamental limitation: Their knowledge was frozen at training time. A model trained in January couldn’t know about products released in February. RAG solved this by combining language models with real-time database lookups.

The concept was straightforward. Instead of trying to encode all knowledge in model parameters, RAG systems retrieved relevant information from databases, then used language models to synthesize responses. When Target implemented RAG in 2023, its customer service bot could answer questions about products added that morning, store hours changed that afternoon, and promotions launching that evening.

The implementation details mattered enormously. Walmart’s 2024 RAG system maintained separate vector databases for products, policies, store information, and customer history. When a customer asked, “Is that TV you advertised on Sunday still on sale at my store?” the system performed four parallel lookups: ad content from Sunday, current promotions, store inventory, and the customer’s location. The language model then synthesized these into a coherent response.

RAG also solved the hallucination problem that plagued pure language models. Instead of inventing product features, the model could only describe what was in the database. Sephora’s implementation included confidence scoring: If retrieved documents didn’t strongly match the query, the system would acknowledge uncertainty rather than fabricating an answer.

Agents

The evolution from assistants to agents marked a fundamental shift. Assistants responded to requests. Agents took independent action to achieve goals. The difference became clear in 2024 when Instacart launched an autonomous shopping agent. Instead of waiting for customers to search for items, the agent proactively monitored pantry staples, tracked consumption patterns, and automatically added items to carts before they ran out.

These agents combined multiple AI techniques. Perception (understanding the current state), planning (determining actions needed), execution (carrying out those actions), and monitoring (verifying success). Chewy’s 2024 pet supply agent tracked pet food consumption rates, monitored price changes across suppliers, applied available coupons, and timed orders to arrive just before supplies ran out.

The technical challenge was maintaining coherence over extended interactions. Adobe Commerce’s agent for B2B purchasing could negotiate prices over multiple emails, remembering context from weeks earlier. It maintained state, kept track of a conversation or activity, using a combination of vector databases for semantic memory and structured databases for factual details like quoted prices and delivery terms.

On-Device Models

Running AI models on customer devices rather than servers offered advantages in privacy, latency, and cost. Apple’s 2023 implementation of on-device product recognition in the Apple Store app could identify any product in Apple’s catalog using just the iPhone’s camera and neural engine, no internet connection required.

The challenge was model size. A phone couldn’t run a 175-billion-parameter model. Qualcomm’s 2024 Snapdragon chips could handle models up to 7 billion parameters, enough for sophisticated tasks but requiring careful optimization. Target’s mobile app used hierarchical inference: A small on-device model handled common requests while escalating complex queries to server-side models.

5.1.6 Looking Forward

The journey from the 1956 Dartmouth Conference to 2025’s AI-powered retail landscape spans nearly 70 years of alternating breakthrough and disappointment. What began as abstract mathematical theory has become infrastructure as essential to modern retail as credit card processing or inventory management.

The J. Lyons company that pioneered commercial computing in 1951 would recognize today’s fundamental challenge: using technology to serve customers better and run operations more efficiently. What they wouldn’t recognize is the scale and sophistication of modern AI systems that can understand natural language, recognize products in photos, predict demand across millions of SKUs, and generate human-quality text in milliseconds.

The pattern across this history is clear. Each breakthrough enabled the next. Expert systems proved computers could encode business logic. Machine learning showed they could discover patterns. Deep learning demonstrated they could perceive and understand. Transformers revealed they could reason and communicate. Foundation models proved they could generalize across domains. And emerging techniques like RAG and MoE are making these capabilities practical at retail scale. But equally important are the limitations that persist.

Understanding this technological foundation matters because retail AI in 2025 isn’t about choosing whether to use these technologies but how to combine them effectively. A modern retailer might use computer vision for visual search, transformers for customer service, RAG for product information, traditional machine learning for demand forecasting, and rule-based systems for pricing constraints, all orchestrated through complex infrastructure spanning edge devices to cloud clusters. The retailers succeeding with AI aren’t those with the most advanced technology but those who understand both the capabilities and limitations of each component in the stack.