Technology Overview
The AI technology stack for commerce — foundational models, agentic AI, LLMs, privacy & security, investment costs, and what comes next — based on Book Part 5 of the AI Best Practices for Commerce reference.
Investment & Cost Considerations
The economics of making AI work sustainably
Understanding Token Costs and Model Consumption
One of the most defining features of modern AI economics is token-based pricing. Large language models break text into discrete units called tokens, and pricing is calculated based on the number of tokens processed during input and output. This creates a consumption model where costs correlate directly with usage patterns, prompt complexity, and output verbosity.
For organizations adopting generative AI, understanding token behavior becomes essential. The length of prompts used internally can dramatically influence costs; verbose prompt engineering or unnecessary context retrieval may multiply token consumption. Processes that rely on multiple sequential prompts—such as agents or multi-step workflows—compound these costs. Because token usage scales directly with user activity, organizations must build governance frameworks that monitor prompt design, enforce usage guidelines, and optimize interactions.
Fine-tuning and embedding generation also carry token-based costs. Fine-tuning requires feeding large volumes of training examples into a model, consuming a substantial quantity of tokens during preprocessing and ingestion.
Embedding operations used in retrieval-augmented generation depend on vectorization, where every document or chunk of text must be converted into numerical space—a process priced per token. Organizations with large knowledge bases or extensive document repositories must evaluate the long-term cost of maintaining embeddings and updating vector stores.
Even inference—the execution of the model to generate output—carries computational costs that scale with complexity. Larger models generally offer more capabilities but require more tokens per operation. Organizations must balance performance and cost, determining when to use small, medium, or large models based on task requirements. The shift toward model “size portfolios” reflects the growing need for intelligent routing mechanisms that direct requests to the most cost-effective model capable of satisfying the task.
Token economics also influence business model design. Companies building AI-powered products must determine how to price their offerings when underlying costs fluctuate with usage. This introduces challenges in forecasting, margin protection, and scalability planning. Understanding tokens becomes not only a technical skill but a financial necessity.
- Prompt length: verbose prompt engineering multiplies input tokens significantly
- Multi-step workflows: agent loops compound token consumption — each tool call adds context
- Embeddings & RAG: cost per document adds up across large product catalogs
- Output volume: long generated descriptions cost 3-10× more than input tokens
- Fine-tuning: large volumes of training examples consumed during preprocessing
| Training Approach | Compute Required | Typical Cost | When to Use |
|---|---|---|---|
| Train from scratch | 16,000+ H100 GPUs | $400M+ (Meta Llama 3 405B) | Never for retailers |
| Full fine-tuning | 1 A100 GPU, few hours | Hundreds of dollars | Domain adaptation with custom data |
| LoRA fine-tuning | Single GPU, <1hr | Tens of dollars | Task specialization, multiple variants |
| Prompt engineering | No training compute | $0 (only inference) | First-pass improvement, always start here |
Last updated: March 12, 2026