# Case study: Broadsheet Deal Capture

> A rewrite of an old Windows capture tool into a system that has captured 40,500+ retailers' broadsheet adverts and more than 3.8 million standardised, checked deals, with AI to find and read each deal and a capturer reviewing every result before it reaches the client-facing reports.

- URL: https://www.happen.software/case-studies/broadsheet-deal-capture
- Publisher: Happen Software (https://www.happen.software/)

## At a glance

- **Client**: Retail data and insights company
- **Category**: Retail & eCommerce · Rescue & rebuild
- **Running**: Since 2020 · 6 years
- **Volume**: 40,500+ adverts, 3.8 million+ deals, and 84,000+ products captured
- **Engagement**: Rewrite of an old Windows tool in 2020, then run and evolve since
- **Users**: Capturers, capture managers, and administrators, signing in with company accounts
- **Sources**: Retailers' broadsheet adverts by publication, region, and date
- **AI**: Deals found and read by AI on request, with every result reviewed by a capturer
- **Output**: Standardised deal data synced to the client-facing reports we also built

## Outcomes

- **3.8 million+**: Deals captured, each in the same standard shape whatever the retailer or region
- **40,500+**: Broadsheet adverts captured since the rewrite in 2020
- **84,000+**: Products captured, each linked to its brand so it can be recognised across adverts

## Context

Our client collects the deals retailers advertise in their broadsheets and turns them into pricing data for its own clients. Every advert holds dozens of deals, each with products, pack sizes, prices, and a promotion that every retailer lays out in its own way.

The capture used to run on a very basic Windows application where deals went in as free text: no fixed deal types, no product list, no price checks, and no QA step, so accuracy was poor and the data was hard to report on. We rewrote it in 2020 as a web-based system built around two aims: accuracy, and deal data in one standard, verified shape across retailers, regions, and publications. Since 2024 it has used an AI service we built and host to find and read the deals on each page, and its completed deals feed the client-facing pricing reports we also built.

## The challenge

- **Free text, poor accuracy**: Deals were typed into a very basic Windows tool as free text, with nothing to check them, so mistakes and inconsistencies went straight through.
- **Every retailer is different**: Each retailer lays out its deals, names its products, and runs regional editions its own way, but the data has to come out the same.
- **Data people rely on**: The deals feed pricing reports for the client's own clients, so a wrong price or product has to be caught before it goes out.

## Before and after

| Area | Without it | With the system |
|---|---|---|
| Reading deals | Every deal typed in from the page as free text. | AI can find each deal on the page and read its products and prices, for a capturer to review. |
| Standard shape | Free text recorded the way each retailer printed it, with no structure. | Every deal captured as one of 4 deal types and 8 promotional mechanics, against one category tree. |
| Products & brands | The same product spelt differently from one advert to the next. | New products staged to be manually checked and captured, and new spellings mapped to the right brand and product. |
| Price checks | No verification or QA, so a mistyped price went straight through. | Prices checked against the product's reference price and recent average, with warnings and blocks. |
| Quality assurance | No review step before data went out. | Every advert QA'd: a capturer finishes it, and a separate user with completion rights signs it off. |
| Regional editions | The same deal captured again for each region. | Matching deals in other editions copied from the completed original, ready to confirm. |
| Reporting | Data moved across to reports by hand. | Completed deals synced to the reporting database 3 times a day and checked every evening. |

## An advert, end to end

1. **Set up**: An advert is created with its retailer, publication, region, and dates. Its pages are uploaded as images or PDFs, put in order, and kept together as one PDF, and it's assigned to a capturer with a priority and due date.
2. **Read with AI**: Users with permission send the advert to the AI service. It finds each deal on the page and reads the products, sizes, prices, and deal type, and the results come back with each deal outlined on the image.
3. **Capture**: The capturer works through each deal: deal type, promotional mechanic, products, pack sizes, and prices. Products the AI read must be reviewed, and known spellings are mapped to the right brand and product.
4. **Check**: Required fields are enforced, prices are compared with the product's reference price and its recent average in the same currency, and category rules for the retailer type apply. New products wait in staging.
5. **Complete**: The capturer marks the advert done, and a user with completion rights completes it. Matching deals in other regions' editions can be copied from the completed original for review.
6. **Report**: Completed deals are synced to the reporting database at 06:00, 12:00, and 17:30, with a check at 20:00 that every eligible deal is there. From there they feed the client-facing reports.

## What we built

- **AI deal extraction**: A custom-trained vision model finds the deals on each broadsheet page, and AI reads their products, sizes, prices, and promotions. Results are drawn on the page so the capturer can see where each deal came from.
- **Structured deal capture**: Every deal is captured as straight, bulk, combo, or multibuy, with one of 8 promotional mechanics and a product configuration such as single, pre-pack, banded, or mixed. Capturers can flag a query or a missing brand.
- **Price checks**: Each price is compared with the product's reference price, with a warning from 15% and a block from 20% by default, and with its 3-month average in the same currency. A capturer can override the average check when a price really has changed.
- **Brand & product mapping**: New products are held in staging until they're approved. Spellings from reviewed AI results are learned as aliases of the right brand and product, and a blacklist keeps wrong matches from coming back.
- **One standard structure**: Retailers with groups and types, regions with their currencies, publications, brands and brand owners, and a 4-level category tree, so every deal is described the same way.
- **Regional editions**: When a retailer runs the same deal in several regions' broadsheets, matching deals are recognised by their content and copied from the completed original, ready for a capturer to confirm.
- **Reporting sync**: Completed deals, with a link to the advert's PDF, are synced to the reporting database three times a day. An evening check confirms every eligible deal arrived and repairs any gaps.
- **Workload & progress**: A dashboard of capture figures, top capturers, each capturer's workload, and incomplete adverts, an hourly capture report on weekdays, and a morning email listing older adverts still incomplete.

## What each team gets

### Capturers

- Adverts assigned with a priority and due date
- AI-read deals to review instead of typing
- A warning before a doubtful price is saved

### Capture managers

- Each capturer's workload on a dashboard
- An hourly capture report on weekdays
- The final say on completing each advert

### Administrators

- Retailers, regions, publications, and categories in one place
- Brand and product mappings they can correct
- Permissions set section by section

### Report users

- Deals in one shape across retailers and regions
- Data refreshed three times a day
- A link back to the advert each deal came from

## How it was delivered

- **2020, Rewrite**: Adverts, deals, products, reference prices, and sign-in with company accounts, built from June and set up for production by October.
- **2021–2023, Reporting**: Reference price rework and a dedicated reporting database synced through the day.
- **2024, AI extraction**: AI deal extraction tested against captured adverts, then connected to capture through a separate service.
- **2025–2026, Fewer repeats**: Regional deal copying, learned brand and product mappings, and one PDF per advert.

## Engineering

- AI as an assist, not the last word: products it reads have to be reviewed before a deal can be completed, and sending adverts to it is a permission.
- Two-step completion and an audit trail: a capturer finishes an advert, and a separate permission completes it.
- Regional copies matched by a hash of the deal's content, only from completed deals for the same retailer within 30 days.
- A sync built for volume: deals are sent in chunks that never split an advert, source data is loaded in batches, and an evening reconciliation repairs any gaps.
- Large images and PDFs uploaded in parts, and each advert's PDF rebuilt whenever its pages are added, removed, or reordered.
- Kept current with regular technology, library, and framework updates, along with ongoing performance and security improvements.

Stack: .NET, ASP.NET Core, SQL Server, Angular, Active Directory, Computer vision, AI, GitHub Actions

## Get in touch

Turning documents into data you can rely on? We'll talk through how AI could read the documents, what checks keep the data accurate, and how to get it into one standard shape. Book a free 30-minute call with one of our engineers.

Contact: https://www.happen.software/contact or sales@happen.software