# Case study: Deal Extractor

> Our own product: a vision model we trained crops every deal off a retail broadsheet, AI reads each crop on its own, and rules in code check the result before it's returned as structured JSON. It extracts more than 100,000 deals from broadsheets every month.

- URL: https://www.happen.software/case-studies/deal-extractor
- Publisher: Happen Software (https://www.happen.software/)

## At a glance

- **Client**: Happen Software (our own product)
- **Category**: Retail & eCommerce · AI & automation
- **Running**: Since 2024 · 2 years
- **Volume**: 100,000+ deals extracted from broadsheets every month
- **In production**: Since 2024, finding and reading deals for a client's capture team
- **Input**: Broadsheet, circular, and catalogue page images, sent over a REST API
- **Output**: JSON for every deal, plus the advert's key number, dates, promotional text, and terms
- **AI**: A custom-trained vision model to find deals, and AI to read each one
- **Quality**: 149 test cases, scored over repeated live runs before a change ships

## Outcomes

- **100,000+**: Deals extracted from retail broadsheets every month
- **90%+**: Accuracy on the deals it extracts, with every result checked by rules in code
- **Since 2024**: In production, finding and reading deals for a client's capture team
- **149**: Cases in the evaluation suite that prompt and model changes are scored against, including 50 deals sampled from real pages

## Context

Retailers publish their promotions as broadsheets, circulars, and catalogues: images, not data. Teams that track prices and promotions need every deal as a record, and on a crowded page that means reading and typing each one. The wording is easy to get wrong, too: "6x400g" is six separate items, while "6-pack" is one pack of six, and that changes the price per item.

We built Deal Extractor to do that one job, and we design, build, and run it ourselves. A vision model we trained finds the deals, AI reads each one from its own crop, and rules in code check the result. It has been in production since 2024, where a client's capture team uses it to find and read deals for review. In 2026 we grew its tests into a 149-case evaluation suite of hand-checked pages, so prompt and model changes are measured rather than judged by eye.

## The challenge

- **Crowded pages**: Deals sit edge to edge and price flashes overlap the next product, so reading a whole page at once attaches prices to the wrong deal.
- **Small wording, big difference**: "2 for", "6x400g", "6-pack", and "3 for 2" each mean a different deal type and pack size, and one misreading spoils every price comparison built on it.
- **Accuracy you can prove**: AI output varies from run to run and small prompt edits move results, so quality has to be measured on real pages, not judged from a few good examples.

## Before and after

| Area | Without it | With Deal Extractor |
|---|---|---|
| Finding deals | Someone works out where each deal starts and ends on a crowded page. | A custom-trained vision model boxes every deal and returns its pixel coordinates. |
| Reading deals | Every product, size, and price typed in by hand. | AI reads each deal from its own crop, so a price can't be taken from the deal next door. |
| Deal types & pack sizes | Recorded however each person reads the wording. | 4 deal types and one quantity x pack x size notation, applied by the same rules every time. |
| Regional editions | The same promotion captured again for every edition. | A match key built from what the deal says, the same across editions with different artwork. |
| Advert details | Dates and terms copied from the fine print. | Key number, validity dates, promotional text, and terms read from up to 5 pages of one advert. |
| Quality | Accuracy judged from spot checks. | Prompt and model changes scored against 149 test cases over repeated runs. |

## A broadsheet page, end to end

1. **Submit**: The client's system sends a page image to the API with its own ID for the page. The file type is checked, an ID that's already been used is turned away, and the page joins the queue.
2. **Find the deals**: Every 30 seconds, queued pages go through our vision model, which draws a box around each deal. If it finds none, the whole page is read as one deal rather than dropped.
3. **Crop**: Each box is cut out of the page, and all the crops from one page are read in parallel.
4. **Read**: AI reads each crop into a fixed JSON schema: every offer, product, pack size, price, saving, and flag, copying only what's printed. A crop that fails is tried up to 3 times.
5. **Check**: Code checks the deal type is one of four, corrects unknown units, splits size alternatives into separate offers, removes repeated products, and builds the match key from the products and price.
6. **Collect**: The client polls for status and percentage complete, or fetches many pages in one call. A finished page returns every valid deal with its coordinates, crop key, and match key.

## What we built

- **Custom-trained deal detector**: A vision model we trained from scratch on labelled broadsheet pages to recognise one thing: a deal. It runs inside the service, with no external AI call, and returns each deal's pixel coordinates on the original page.
- **One deal per AI read**: Each crop is read on its own, at a low temperature and into a fixed JSON schema. The instructions say to copy only what's printed: no guessed sizes, and savings, loyalty pricing, and "as seen on TV" only when shown.
- **Four deal types**: Straight, Bulk, Combo, and Multibuy, decided in a fixed order, with every product's pack size written as quantity x pack x size, such as 1x6x1L for one six-pack or 6x1x400g for six separate items.
- **Rules checked in code**: The AI's answer is checked before it's saved: deal types and units are validated, size alternatives become separate offers, and a crop with several prices becomes separate deals that share one crop key.
- **Matching across editions**: Each deal carries a match key built from its products and price, ignoring order, casing, punctuation, and how words are split between brand and name, so the same promotion matches across regional artwork.
- **Advert details**: The advert's key number, validity dates, promotional text, terms, websites, and phone numbers, from up to 5 pages read as one advert. Text on the page is treated as data, never as instructions.
- **REST API & queue**: Five endpoints to submit pages, read advert details, check status, fetch results in bulk, and resubmit a failed page. Each client has its own API key, and work runs through a background queue with retries.
- **Usage & cost tracking**: An admin portal for clients and their API keys, AI token usage by operation, and Excel reports of deals read per client per day, counting only crops that were read successfully.

## What each team gets

### Capture teams

- Deals found and read before anyone types
- Each record linked to the crop it came from
- Advert dates and terms from the same pages

### Developers

- A plain REST API returning JSON in a fixed schema
- Status checks, or results for many pages in one call
- A resubmit call for pages that failed

### Analysts

- Pack sizes in one notation, ready to compare
- A match key to remove repeats across editions
- Deal type, savings, and loyalty pricing as fields

### Our team

- AI token usage by operation in the portal
- Daily deal counts per client in Excel
- Errors alerted to our engineers as they happen

## How it was delivered

- **2024, First release**: Project set up in May, the deal detector trained and a first version deployed by June, then set up for production in September.
- **2024–2025, Deal rules**: Four deal types with rules for each, crops read in parallel, test suites for every deal type, and crops holding several deals split into separate records.
- **2025, Advert details & matching**: Key numbers, validity dates, promotional text, and terms read from single and multi-page adverts, and a match key for the same deal across editions.
- **2026, Measured & tuned**: Per-deal retries, token usage tracking, a match key with no second AI call, and a 149-case evaluation suite behind a rewritten prompt.

## Engineering

- Find first, then read: the vision model crops each deal, so the AI only ever sees one deal and can't borrow a price or product from its neighbour.
- Accuracy measured, not eyeballed: 149 test cases, including 50 deal crops sampled across a year of real pages and checked by hand against the images, where one wrong field fails the whole deal.
- Every change compared over at least three runs, because small wording edits move the results. A rewritten prompt at under half the length passed more of the hand-checked deals than the longer one it replaced.
- Models chosen on evidence: several AI models were compared on the same suite for accuracy and measured cost, and cheaper options that lost accuracy were turned down.
- The AI never has the last word: JSON schema output, deal types and units checked in code, size alternatives split, and page text treated as data, never as instructions.
- The match key is computed from the extracted products and price rather than from the image, which removed a second AI call for every deal in 2026.
- Kept current with regular technology, library, and framework updates, along with ongoing performance and security improvements.

Stack: .NET, ASP.NET Core, SQL Server, Angular, Python, OpenCV, Computer vision, AI, GitHub Actions

## Get in touch

Need data out of documents that only exist as images? Whether you want Deal Extractor for your own broadsheets or document AI built around your pages, we'll talk through how it would read them and how we'd measure the accuracy. Book a free 30-minute call with one of our engineers.

Contact: https://www.happen.software/contact or sales@happen.software