Financial Statement Table Extraction with Accounting Consistency Checks

The problem analysts have with annual report PDFs

Listed companies publish their financial statements as PDF annual reports. Investment analysts, credit officers at banks and microfinance lenders, auditors in training and finance students all need those numbers in a spreadsheet, and in many markets the only way to get them is to copy the tables by hand. On exchanges such as the Pakistan Stock Exchange, smaller companies' reports are often scanned, inconsistently formatted, with notes, brackets for negative numbers and figures in thousands. Commercial data terminals cover large companies well and small ones poorly, and they are priced for institutions.

Copying by hand is slow, and a single wrong digit silently distorts every ratio calculated afterwards.

What already exists and the gap

• AWS Textract and Azure AI Document Intelligence extract tables from documents generically and charge per page; they do not know that a balance sheet must balance.
• Docsumo and similar document AI tools offer financial statement extraction for lenders as a paid service.
• Microsoft's Table Transformer and the PubTables-1M and FinTabNet datasets provide strong open models and benchmarks for table structure recognition.

The gap is domain validation. Generic extractors return tables; they cannot tell you the extracted numbers are wrong. Financial statements carry their own checks, and a system that uses them can know which cells to distrust.

What you build

A pipeline and a review app. The pipeline takes an annual report PDF, finds the statement pages, detects tables and their structure, reads the cells (text layer when present, OCR when scanned), and maps each row to a standard line item with the correct sign and unit. The consistency checker then runs accounting identities across the extracted data. Everything that passes becomes structured output; anything that fails is sent to the review app, where the analyst sees the original page with the suspect cells highlighted.

The technically interesting part

Using accounting structure as a free error detector, and measuring how good it is. Each identity involves specific cells, so a failed check narrows the error to a few candidates. Go further: when one identity fails and exactly one cell's digits could plausibly have been misread (a 3 for an 8, a dropped digit), propose the correction that satisfies all identities and ask the reviewer to confirm. Your evaluation should answer questions evaluators care about: what share of real extraction errors do the checks catch, how many false alarms do they raise, and how much reviewer time do they save compared with checking every cell?

The second challenge is structure recognition on financial tables with multi-level headers, indented subtotals and note references. Report accuracy separately for text PDFs and scans.

How to evaluate it

• Page classification accuracy for locating the three primary statements.
• Table structure quality with the GriTS metric on FinTabNet.c and on your gold set.
• Cell-level exact-match accuracy of numbers after mapping, by sector and by PDF type (text or scanned).
• Error detection recall and false alarm rate of the consistency checks on the extraction errors that occur.
• Reviewer time per statement with highlighted checks versus full manual verification, measured with at least three finance students.
• Cost and time per report for the open pipeline and, if used, the vision-language fallback.

Suggested plan

1. Month 1: collect reports, define the standard chart of accounts, annotate the first gold statements.
2. Month 2: page classification and table detection.
3. Month 3: structure recognition, OCR path, number parsing with units and signs.
4. Month 4: line item mapping and consistency checks with correction suggestions.
5. Month 5: review app and vision-language fallback experiment.
6. Month 6: full evaluation, reviewer timing study, write-up.

Path to a product

The first customers are small brokerage research desks, microfinance and SME lenders who analyse borrower financials, and university finance departments that want clean data for teaching. Offer a searchable dataset of listed companies' statements on subscription, and an upload service priced per document for private company financials. Accuracy reports and visible checks become the reason to trust it over hand-copied spreadsheets.

Scope warning

Do not attempt notes to the accounts, narrative sections or every table in the report; the three primary statements are enough. Do not build a stock analysis or recommendation tool. Respect the terms under which reports are published and do not redistribute reports themselves.

Learn what you need on Sefism

• Machine Learning roadmap : detection models, fine-tuning and evaluation methodology.
• AI Engineering roadmap : when and how to use a vision-language model as a fallback and cost it.
• Data Analytics cheat sheet : structuring the output for analysis and ratio calculations.
• PostgreSQL Documentation : storing statements, line items and review history.
• Google Colab : free GPU time for fine-tuning table models.

Get help with this idea

The key choices are your chart of accounts, your gold dataset and which identities you check. A mentor can review them early so the evaluation holds up. Bring your draft scope, the dataset you plan to use and your evaluation table, and we can look at them together. The build is yours; the guidance is on scope, stack and how to measure it. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session