Manas Mishra
Résumé

Case study Datavio · Document AI

Reading hundreds of debit notes so nobody has to

Debit notes arrive as PDFs in a vendor portal, and every line has to be tied back to an invoice and a SKU. The pipeline extracts, matches and reconciles them — rules first, GPT only for what the rules can't place.

Role
Designed & built
Context
Datavio · marketplace finance data
Stack
Azure Document Intelligence · GPT · Python
Pattern
Resumable, checkpointed phases
GPT calls after moving to rules-first matching — down from one per debit note
486 → 2
Of 486 debit notes split cleanly down to SKU lines
94.9%
Debit-note PDFs fetched in a single resumable run
489
Concurrent downloads — fast, without hammering the portal
5
checkpoint after every phase phase · fetch phase · extract phase · match no match portal fetch ×5 doc intel rules matched gpt debit notes pdfs fields + tables invoice + sku long tail
Every PDF flows through fetch, extraction and matching. Rules handle the clear-cut matches; anything they can't place detours through GPT. Raw items (amber) turn green once matched.
TL;DR Debit-note PDFs are pulled from a vendor portal, parsed with Azure Document Intelligence, and their line items matched to invoice lines and SKUs — deterministic rules first, GPT for the leftovers. The pipeline runs in resumable phases with a checkpoint file, so a failure halfway through a large batch picks up where it stopped. One run fetched 489 PDFs, five at a time; across 486 debit notes, 94.9% split cleanly to SKU lines and GPT calls fell from 486 to 2.

01 · The problemDeductions hide in PDFs

Marketplaces raise debit notes against a brand's invoices — deductions that quietly reduce what the brand gets paid. They arrive as PDFs in a vendor portal, and each line has to be tied back to the right invoice and SKU before anyone can accept or dispute it.

Doing that by hand, across hundreds of documents, is slow, tedious and easy to get wrong.

02 · The intuitionRules for the obvious, a model for the ambiguous

A lot of matching is boring in a good way: codes, quantities and amounts that line up exactly. Rules handle those deterministically — fast, free and explainable. The LLM is reserved for what the rules genuinely can't place.

Use rules for what's certain and a model for what's ambiguous — never the other way round.

You get results you can trust, and a model bill you can predict.

03 · The solutionFetch, extract, match — and remember progress

  1. Fetch. The portal connector downloads debit-note PDFs, five at a time.
  2. Extract. Azure Document Intelligence turns each PDF into structured fields and line-item tables.
  3. Match with rules. Deterministic rules map line items to invoice lines and SKUs wherever the data lines up; every split must agree to ±₹0.01 after a GST-rate check.
  4. Fall back to GPT. Items the rules can't place go to GPT instead of to a person.
  5. Checkpoint every phase. Progress is written to a checkpoint file, so re-running skips work that's already done.

04 · ImplementationBuilt to survive a bad afternoon

Phases, not one long script

Large batches fail in the middle — a portal times out, a PDF is malformed, a rate limit kicks in. Splitting the pipeline into phases with a checkpoint after each means a failure only costs the work in progress, not the whole run.

state = load_checkpoint("run.json")      # which documents finished which phase

for phase in (fetch, extract, match):
    for doc in pending(state, phase):
        phase(doc)
        mark_done(state, phase, doc)
        save_checkpoint("run.json", state)  # a crash resumes from here

A simplified sketch of the checkpointing pattern.

Concurrency with manners

Five concurrent downloads kept a 489-PDF run moving without hammering the portal. On someone else's platform, a concurrency limit is part of being a good citizen.

Keeping GPT on a short leash

The model only sees the items the rules couldn't place — a narrow, well-defined question instead of an open-ended one. That keeps its answers easier to check and its cost proportional to the genuinely hard cases.

The payoffBefore rules-first matching, every one of 486 debit notes went through GPT. After it, the rules split 94.9% of them cleanly and only 2 needed the model.

05 · ResultsWhat the pipeline delivers

Before

Someone opens each PDF, reads every line and hunts down the matching invoice and SKU by hand.

After

PDFs are fetched, parsed and matched automatically; people review results instead of doing the lookup.
  • GPT calls cut from 486 to 2 by putting deterministic rules first.
  • 94.9% of 486 debit notes split cleanly down to SKU lines, each agreeing to ±₹0.01.
  • 489 PDFs fetched in a single resumable run, five at a time.
  • The split debit notes feed the penny-level payment reconciliation.
  • Failures resume from the last checkpoint instead of restarting the batch.

06 · TakeawaysPatterns worth reusing

  • In pipelines that have to be right, LLMs work best as the fallback, not the first resort.
  • Long batch jobs need checkpoints. Assume the middle will fail.
  • Concurrency limits are part of being a good citizen on someone else's platform.

Turning messy documents into reliable data? I'd enjoy hearing about it.