# Case study: E-commerce Product Data Platform

> An internal platform that turns product data from brands into accurate, complete e-commerce content for each customer, across 152,000+ products: jobs and batches with two QA stages, a data model built from configurable fields, queued bulk work, and an API shaped by each customer's template.

- URL: https://www.happen.software/case-studies/ecommerce-product-data-platform
- Publisher: Happen Software (https://www.happen.software/)

## At a glance

- **Client**: Retail data and insights company
- **Category**: Retail & eCommerce · Platforms & portals
- **Running**: Since 2021 · 5 years
- **Scale**: 152,000+ products managed on the platform
- **Engagement**: Build and evolve, since 2021
- **Users**: The client's capture, creative, QA, and admin teams
- **Data in**: Product loadsheets, brand owner imports over an API, and product images by barcode
- **Data out**: A client API shaped per customer by export templates, plus queued exports
- **Data model**: 19 field types, plus aggregate, inherited, and transform fields

## Outcomes

- **152,000+**: Products managed on the platform for the client's customers
- **No code**: Needed to add a new kind of product data: fields are set up in the portal, not built by developers
- **2 QA stages**: On every batch, after capture and after creative, before it goes to review

## Context

Our client prepares product data for e-commerce: names, descriptions, classifications, pack sizes, nutritional details, images, and whatever else each of its customers needs. Much of it starts with the brands that make the products, and it has to be right before it reaches a customer.

We built the platform its teams use to do that work. A customer's products arrive as a job, are checked against what's already known, captured, enriched, and passed through QA, then published through an API that returns each customer's own fields in its own shape. It is a large system, with most of its data model configurable rather than fixed in code.

## The challenge

- **Each customer's data is different**: Each customer wants its own fields, classifications, and formats, so the data model can't be fixed in code.
- **Brands' data has to be right**: Product data arrives from brands and customers, and mismatched codes, duplicates, or inconsistent names must be caught before the data is used.
- **Many products, many hands**: Products pass through capture, creative, and checking teams, and bulk changes touch many products at once, so the work needs structure.

## Before and after

| Area | Without it | With the platform |
|---|---|---|
| New fields | A new field for a customer means a code change. | Fields set up in the portal from 19 types, with dictionaries, units of measure, and classifications. |
| Incoming data | Problems in a loadsheet found partway through the work. | Duplicate GTINs and missing required cells caught when the loadsheet is loaded. |
| Matching | Products checked against existing records by eye. | Each product matched to its baseline, with code mismatches and duplicates flagged as errors. |
| Work in progress | Progress tracked outside the system. | Batches move through capture, capture QA, creative, creative QA, and review. |
| Bulk changes | Large updates applied by hand. | Six kinds of bulk import queued, processed in the background, and recorded. |
| Delivery | One data shape for every customer. | Each customer's systems read their products through an API shaped by their own export template. |

## A product, from loadsheet to API

1. **Load**: A customer's products arrive on a loadsheet and become a job. Duplicate GTINs and missing required cells are flagged as it's loaded.
2. **Match**: Each product is matched to its baseline record. SKU mismatches and duplicates are errors to resolve, and brand or sub-brand names that differ from the standard are warnings.
3. **Capture**: Products are split into batches for capture. AI suggests descriptions, categories, features and benefits, and keywords, and reads text and nutritional tables from product images.
4. **Check**: Each batch passes capture QA, then creative and creative QA, before it goes to review.
5. **Create**: Reviewed batches wait for their products to be created, then the batch is completed and the job's progress moves on.
6. **Publish**: The customer's systems read the products through the client API, with each API key returning the fields and layout of that customer's export template.

## What we built

- **Jobs, batches & QA**: Loadsheets become jobs, jobs are split into batches, and batches move through capture, capture QA, creative, creative QA, and review, so everyone can see where each product stands.
- **Custom-field data model**: Fields are configured, not coded: 19 types from text, numbers, and dates to GTINs, SKUs, and taxonomy levels, owned by the customer, the brand owner, or the client.
- **Derived fields**: Aggregate fields combine several fields into one. Transform fields change case, trim or replace with patterns, or apply standard brand names. Inherited fields take their value from an ordered list of sources, chosen by filter rules.
- **Accuracy checks**: Loadsheet checks, baseline matching, code mismatch and duplicate errors, brand name warnings, and a daily check for possible duplicate products.
- **Template-driven client API**: Export templates define the fields and layout each customer receives. API keys can be tied to a template, and each key sees only the values it's allowed to.
- **Brand owner imports**: Brand owners send product data over their own API. It's validated, then queued for import against their products.
- **AI-assisted content**: AI drafts descriptions, categories, features and benefits, and keywords during capture, and reads text and nutritional tables from product images.
- **Queued bulk work**: Imports, exports, product updates, and image fetches run in background queues. Six import types, from linking GTINs to pack configuration set-up, are processed and recorded.

## What each team gets

### Capture & creative teams

- Batches of products ready to work on
- AI-drafted descriptions and keywords to review
- Text and nutritional tables read from images

### QA & review

- A QA stage after capture and after creative
- Errors and warnings listed against each product
- Value history and audits on product changes

### Admin team

- Fields, dictionaries, and templates set up in the portal
- Bulk imports queued instead of done by hand
- Sign-in with their company accounts

### Customers & brand owners

- Product data through an API in their own shape
- API keys limited to their own data
- An API for brand owners to send product data in

## How it was delivered

- **2021, Foundation**: Jobs, loadsheets, capture and QA stages from the first months, and the client API by July.
- **2022–2023, Templates & brands**: Export templates to shape each customer's data, an API for brand owners, and AI-assisted content from 2023.
- **2024–2025, Completeness**: Completion metrics per customer and an internal API that publishes the import schema brand owners build against.
- **2026, Modernised**: An AI data assistant added for the client's teams.

## Engineering

- Business rules written as plain functions over loaded data, so the API's rules on who sees what are tested without a database.
- The client API returns a value only when the rules say a caller may see it, so anything unexpected stays hidden rather than shown.
- Pattern-based transforms are guarded against invalid patterns and time limits, so one bad rule can't hold up the rest.
- One definition of required fields drives both the import checks and the schema brand owners build against, so the two can't drift apart.
- Bulk work runs in queues with a status per item and a recorded snapshot of what changed, and image fetches retry before giving up.
- About 430 automated tests, and the roughly 300 that need no database run before every deployment to the development environment, stopping it if any fail.
- Kept current with regular technology, library, and framework updates, along with ongoing performance and security improvements.

Stack: .NET, ASP.NET Core, SQL Server, Angular, Active Directory, GitHub Actions

## Get in touch

Managing product data from many brands for many customers? We'll talk through how configurable fields, checks, and QA stages could keep your data accurate, and how each customer could get it in the shape they need. Book a free 30-minute call with one of our engineers.

Contact: https://www.happen.software/contact or sales@happen.software