Retail analytics and personalization

The challenge

A national retail chain with more than 200 stores and a growing e-commerce arm came to us with a familiar problem: they had plenty of customer data and almost no ability to use it. Purchase history lived in the POS systems, browsing behavior in the e-commerce stack, and campaign data in the marketing tools, with no shared identity connecting them. A customer who browsed online and bought in store looked like two different people.

The consequences were predictable. Marketing ran broad, untargeted campaigns with low engagement and poor ROI. Nothing reacted to customer behavior as it happened, so the moment someone showed intent was the moment the business could do nothing about it. Demand planning suffered from the same blindness, with stockouts and overstock that better forecasting could have avoided.

What we built

The diagram above shows the shape of the system: three source systems feeding one event pipeline, a unified customer view in the middle, a model layer on top of it, and three delivery surfaces at the end. Everything flows left to right, and everything downstream reads from one customer record.

Event pipeline and unified customer view

We started by treating every customer touchpoint as an event. POS transactions, web and mobile e-commerce activity, and email and CRM interactions all land in a single behavioral event pipeline built on Spark with Redis as the hot path, processing over 5M interactions a day.

The pipeline feeds an identity stitching layer that resolves online and offline activity to one customer record, stored in MongoDB with Elasticsearch for query and search. We made identity resolution a pipeline concern rather than a per-application one deliberately: once stitching happens in one place, every model and every delivery channel inherits the same view of the customer, and we never have to reconcile conflicting profiles downstream.

Segmentation and demand models

The unified view exposes features to a model layer with three components. Segmentation models group customers into behavioral cohorts based on what they actually do rather than static demographic buckets, so segments update as behavior changes. Demand forecasting works at SKU by location granularity, which is the level merchandising teams actually plan at; a chain-wide forecast would have told them nothing about which store needed which product. The personalization models sit alongside these, scoring recommendations and offers per customer.

We kept these as separate models over a shared feature layer rather than one monolith because they evolve at different speeds. Forecasting retrains on a slower cadence than recommendation scoring, and decoupling them meant we could ship and tune each independently.

Personalization delivery across channels

Model decisions flow out to three surfaces: on-site recommendations on the web storefront, targeted email campaigns, and in-store offers and stock guidance. All three read from the same decision layer, so a customer sees consistent treatment whether they open an email or walk into a store. The system serves real-time personalization for more than 2M active users.

How it was delivered

A team of seven delivered the platform over five months. We sequenced the work to de-risk it: data integration and the streaming pipeline first, then the unified profiles and segmentation, then the recommendation models with an A/B testing framework before any decisioning went live, and finally performance tuning at production scale and handover to the client’s team. Email personalization shipped early, which gave stakeholders measurable results months before the full platform was complete. The stack was React, Node.js, and Python on AWS, with Spark, MongoDB, Redis, and Elasticsearch doing the data work.

What shipped

• Behavioral event pipeline processing 5M+ customer interactions daily
• One customer view stitching POS, e-commerce, and marketing data across online and offline
• Real-time personalization for 2M+ active users across web, email, and in-store channels
• SKU by location demand forecasting and behavioral segmentation
• 35% increase in conversion, 28% higher cart value, 42% better retention, and a 60% lift in email campaign effectiveness

The client’s team now operates the platform themselves, and the architecture has room to grow: new delivery channels plug into the existing decision layer, and new models plug into the existing feature layer.