When we started Tabular, we thought document extraction would be the least of our problems – surely after decades of PDF invoicing, plug-and-play solutions would exist. We couldn’t have been more wrong.

Take line item extraction: the baseline accuracy of conventional solutions, like Azure Cognitive Services, on our expert-labelled dataset of >500k line items was 81%. Building a truly autonomous accounting system requires an accuracy of 99.9%. While more modern, LLM based, providers proved better, they came nowhere close to our ambition of near perfect extraction.

Why do line items matter?

Line items are the list of goods or services a company purchased with the invoice. They matter because each item may have to be treated differently by the accountant. An incorrect item can lead to wrong tax fillings, which a bookkeeper is liable for. This means that the bar for line items is brutal – if an invoice is processed without human review, extraction must be perfect.

Why is line item extraction even so hard?

Complex & unique formats – One might be forgiven for believing that every supplier pours their creative energies into the design of a custom invoice format. Layouts differ so wildly, and are so context-dependent, that even a human expert may need minutes to decipher them. The most pristine PDF can pose a formidable extraction challenge if it is unclear what the relevant line items actually are. Run out of Sudokus? Give this invoice a shot…

• Full of noise: the invoice is densely packed with mostly irrelevant tabular data, making it hard to identify the actual line items. The entire upper half of the document is noise concerning the general pricing structure of the agreement and the quality of milk products delivered.
• Mixed hierarchy levels: there are both individual line items and summary items in the tables. Take the ‘MUNLV-Abzug’ of +10,11 EUR and the below ‘Gesamt Betrag’ of -10,11 EUR. These constitute just one line item, which is negative, since it is listed in a table called ‘Deductions’.
• Confusing labels: The invoice is full of labels like ‘Gross Amount’, ‘Net Amount’, ‘Payment Amount’, ‘Milk Money’, ‘Yearly Amount’, ‘Milk Payment Statement’ and ‘Payment 11.8.2025’; some of these represent line items, but which ones?

Poor image quality – In Europe, SMBs still heavily rely on printed documents; e-invoices are coming, but so far remain rare (<10%). Until then we must make due with grainy scans and hasty pictures of crumbled receipts. This is particularly bad for line item extraction: unlike a total amount or supplier name, we’re not extracting a single value but a composite list. Each line item consists of names, amounts, quantities, tax rates etc. The only way they’re connected is through their relative position in the invoice (e.g. name appears next to amount on the same horizontal axis). Noisy OCR text, slanted images and crumbled documents obscure this layout information – next thing you know, amount 1 appears to be associated with line item 2, and the entire extraction becomes scrambled.

All or nothing – We discovered that getting 49 out of 50 line items right is about as useful as not getting a single one. Because where is the missing line item? At the bottom of page 7 or was it page 3? Watching users take out a physical calculator and press their noses to the screen in search of a missing item that was never to be found quickly cured us of the belief that partial results were of any use. It was almost always easier for the user to start from scratch.