# Case study: Zondo Explorer: State Capture Commission Search for News24

> Zondo Explorer made the State Capture Commission's reports, transcripts, and statements searchable by the people and organisations named in them. Starting in 2021, we ran OCR and traditional natural language processing over more than 250,000 pages, matched tens of thousands of entities and their relationships, and built the public site and editors' portal for News24.

- URL: https://www.happen.software/case-studies/zondo-explorer
- Publisher: Happen Software (https://www.happen.software/)

## At a glance

- **Client**: News24
- **Category**: Media · AI & automation
- **Running**: 2021–2024
- **Scale**: 250,000+ pages, tens of thousands of entities, and relationships between thousands of them
- **Documents**: The commission's final report volumes, hearing transcripts, and statements submitted as evidence
- **Engagement**: From 2021 to 2024: extraction, matching, portal, and the public site
- **Users**: The public, researchers, and journalists, with News24's editors curating in the portal
- **Platforms**: A public explorer, an editors' portal, and tools for loading new source material

## Outcomes

- **250,000+**: Pages of commission reports, transcripts, and statements processed
- **10,000+**: People and organisations identified, tens of thousands in all, each linked to every page that mentions them
- **1,000s**: Of entities connected by relationships the models found, each tied to the page it came from
- **100%**: Of mentions traceable to the original document page, which readers can open or download

## Context

The Judicial Commission of Inquiry into State Capture, chaired by now Chief Justice Raymond Zondo, sat from 2018 to 2022. It published its final report in volumes, alongside hearing transcripts and the statements and evidence submitted to it: more than a quarter of a million pages, much of it scanned.

News24 wanted South Africans to be able to find every person and organisation that surfaced in the proceedings, and how they connect. We ran OCR and natural language models over every page to find the people, organisations, and relationships in them, built the matching that turned raw mentions into clean entities, and built the portal News24's editors curated it in and the Explorer the public used.

The extraction work predates ChatGPT and today's widely used large language model tools. We used OCR and traditional NLP models for entity and relationship extraction, with matching and editorial review turning the results into a searchable archive.

## The challenge

- **Scanned pages**: Much of the material was scanned, so names arrived with OCR errors: letters swapped, split, or misread.
- **One name, many spellings**: The same company or person appears with and without titles, in different word orders, as acronyms, and in possessive forms.
- **Evidence, not opinion**: Every connection the site shows had to lead back to the exact page it came from, so readers could check it themselves.

## Before and after

| Area | Without it | With Zondo Explorer |
|---|---|---|
| Finding a name | Searching volume by volume, PDF by PDF. | One search across every report, transcript, and statement, by person or organisation. |
| Spellings and OCR errors | Each variant searched separately, and misread names missed. | Variants and OCR errors merged into one entity, with its aliases listed. |
| Where they appear | No overview of how much a name features, or where. | A breakdown of every document and page an entity is mentioned in, with charts. |
| Connections | Relationships pieced together by reading. | Relationships between people and organisations drawn as a diagram, grouped by type. |
| Checking the source | Finding the page again in the original. | Every mention links to its page, and each document can be viewed and downloaded. |

## From scanned page to explorer, end to end

1. **Read**: Each document is split into pages and run through OCR. Page images and thumbnails are stored so readers can see the original later.
2. **Extract**: Traditional NLP identifies people, organisations, places, and relationships on each page, ready for matching and editorial review.
3. **Clean**: Irrelevant fragments and reading errors are cleaned up so people and organisations have useful, consistent records.
4. **Match**: Alternative spellings and OCR errors are brought together into one record, with its aliases and links to every page where it appears.
5. **Curate**: Editors review people, organisations, and relationships in the portal, merge duplicate records, and add roles, photos, and items of interest.
6. **Explore**: Readers search the Explorer, open an entity, see where it appears and what it's connected to, and click through to the page itself.

## What we built

- **OCR & entity extraction**: OCR and traditional NLP identify people, organisations, and places across the archive, each linked to the page that mentions it.
- **Entity matching**: Different spellings, acronyms, and OCR errors are brought into one record so readers can find the same person or organisation consistently.
- **Relationships**: Readers can explore relationships between people and organisations and follow each connection back to its source page.
- **Search**: Search across every person and organisation, with people and organisations of interest featured on the home page alongside the documents and key relationships.
- **Entity pages**: For each person or organisation: role, photo, a breakdown of every document they're mentioned in with charts, and their relationships grouped by type and drawn as a diagram.
- **Document pages**: Each report volume, transcript, and statement with the entities mentioned in it and the most-mentioned charted, page images to view, and the document to download.
- **Editors' portal**: Entities, aliases, alias page references, relationships, and documents managed by News24's editors, who can link stray entities together, add roles and photos, and flag entities of interest.
- **Audited changes**: Every edit in the portal audited with who made it, with portal users and permissions per section, and imports run from the portal as new material came in.

## What each team gets

### Readers

- Every person and organisation in the commission's documents, in one search
- Where each appears, and how often
- The original page, a click away

### Journalists & researchers

- Connections between people and organisations, drawn as a diagram
- Spellings and OCR errors already merged
- Pages and documents to cite

### News24 editors

- A portal to review and correct entities
- People and organisations of interest featured on the site
- Roles, photos, and links added where the models fell short

## How it was delivered

- **2021–2022, OCR & extraction**: OCR and traditional NLP made the commission's growing collection of documents searchable by the people and organisations they mentioned, with work beginning before ChatGPT and today's LLM tools.
- **Late 2022, Matching & the Explorer**: Consistent entity records, source-document imports, the editors' portal, and the public Explorer brought the archive into a searchable service.
- **2023, Relationships & refinement**: Improved document handling, entity roles and photos, clear page references, and refined relationships made the archive easier to explore and verify.
- **2024, Support & maintenance**: Supported the Explorer for readers and researchers.

## Built to last

- Matching brings spelling variations and OCR errors together so readers can find all references to the same person or organisation.
- People and organisations have consistent records with their alternative names visible.
- Quality checks and editorial review help keep irrelevant or incorrect mentions out of the public explorer.
- Every mention remains linked to its original document and page so readers can verify the evidence.
- Relationships are reviewed and linked to their source material, supporting research without losing context.
- Search and entity pages remain responsive across more than 250,000 pages of source material.

Technology: .NET, ASP.NET Core, SQL Server, Angular, NLP, OCR

## Get in touch

Sitting on a mountain of documents? We'll talk through how OCR and natural language processing could pull the people, organisations, and connections out of your archive, and how to put them in front of readers or investigators with every claim traceable to its page. Book a free 30-minute call with one of our engineers.

Contact: https://www.happen.software/contact or sales@happen.software