Parsing multi-page PDF receipts and folios

A hotel folio or airline invoice that spans several pages breaks single-page extraction in two silent ways: the repeated column header on page two is parsed as a phantom line item, and a charge whose description wraps across the page boundary is split into two half-rows that never reconcile against the printed grand total. This guide handles the stitching problem under the parent pdfplumber Line-Item Parsing guide, inside the broader Receipt Ingestion & OCR Data Extraction framework, where documents routinely arrive as multi-page folios rather than tidy single receipts.

The goal is a single reconciled line-item set per document whose sum matches the folio total, so that a multi-page stay reaches Automated Policy Validation & Anomaly Flagging as one coherent record instead of a scatter of orphaned page fragments.

Why Standard Approaches Fail

Concatenating page.extract_text() across pages and parsing the blob treats a folio as one flat document, which corrupts exactly the totals an audit depends on. Three named failure modes account for most of the damage.

Repeated-header phantom rows. Multi-page folios reprint the column header — Date Description Amount — and often a “carried forward” subtotal on every page. Parse each page independently and page two contributes a header row and a running-balance row as if they were charges, inflating the line-item count and, when the balance line parses as an amount, the total.
Boundary-split charges. A charge whose description wraps, or whose amount sits one line below its label, can straddle the page break: the description lands at the bottom of page one and the amount at the top of page two. Independent parsing emits a description with no amount and an amount with no description, and neither reconciles against the printed grand total.
Per-page subtotal double-counting. Hotel folios print a per-night or per-department subtotal that repeats and rolls up. Summing every numeric row adds those subtotals on top of the line items they summarize, so the computed total overshoots the real charge and the receipt is wrongly flagged — or, worse, quietly accepted at the inflated figure.

The remedy is to treat the folio as an ordered sequence of pages sharing one header and footer template, strip the repeats, reattach rows split at the boundary, and reconcile the stitched items against the document’s own printed total before anything posts.

Architecture & Algorithm

The stitcher below reads all pages, detects the header and footer bands by finding text that repeats at the same vertical position across pages, drops those bands, reattaches a trailing description on one page to the leading amount on the next, and accumulates the surviving rows into one ordered set with a Decimal running total. It then reconciles that total against the grand-total line so a mismatch is flagged rather than posted. Pages are processed in order and released one at a time, so resident memory stays flat regardless of folio length.

from __future__ import annotations

import json
import logging
import re
from collections import Counter
from dataclasses import dataclass, field
from decimal import Decimal, InvalidOperation
from typing import List, Optional

import pdfplumber

logger = logging.getLogger("expense_audit.multipage_folio")
if not logger.handlers:
    logger.setLevel(logging.INFO)
    _handler = logging.StreamHandler()
    _handler.setFormatter(
        logging.Formatter('{"ts":"%(asctime)s","lvl":"%(levelname)s","msg":"%(message)s"}')
    )
    logger.addHandler(_handler)

_AMOUNT_RE = re.compile(r"(\d[\d,]*\.\d{2})")
_GRAND_TOTAL_RE = re.compile(r"(?i)\b(?:grand\s+total|balance\s+due|amount\s+due|total)\b")
_SUBTOTAL_RE = re.compile(r"(?i)\b(?:subtotal|carried\s+forward|balance\s+forward)\b")


@dataclass(frozen=True)
class FolioLine:
    """One stitched charge line with the page it originated on."""

    description: str
    amount: Decimal
    source_page: int


@dataclass
class FolioResult:
    """Reconciled multi-page extraction with an explicit audit verdict."""

    correlation_id: str
    line_items: List[FolioLine]
    computed_total: Decimal
    stated_total: Optional[Decimal]
    reconciled: bool
    page_count: int
    flags: List[str] = field(default_factory=list)


def _parse_amount(text: str) -> Optional[Decimal]:
    match = _AMOUNT_RE.search(text)
    if not match:
        return None
    try:
        return Decimal(match.group(1).replace(",", ""))
    except InvalidOperation:
        return None


def _repeated_band_lines(pages_text: List[List[str]], min_pages: int = 2) -> set:
    """Return line strings that repeat verbatim across pages (header/footer)."""
    counter: Counter = Counter()
    for lines in pages_text:
        # A repeated header/footer line appears once per page; dedupe within page.
        for line in set(l.strip() for l in lines if l.strip()):
            counter[line] += 1
    return {line for line, n in counter.items() if n >= min_pages}


def _page_lines(page) -> List[str]:
    text = page.extract_text() or ""
    return [ln for ln in text.splitlines() if ln.strip()]


def parse_folio(pdf_path: str, correlation_id: str) -> FolioResult:
    """Stitch a multi-page folio into one reconciled line-item set."""
    with pdfplumber.open(pdf_path) as pdf:
        pages_text = [_page_lines(page) for page in pdf.pages]

    page_count = len(pages_text)
    repeated = _repeated_band_lines(pages_text)

    items: List[FolioLine] = []
    stated_total: Optional[Decimal] = None
    pending_description: str = ""      # carries a split row across a page break

    for page_idx, lines in enumerate(pages_text):
        for line in lines:
            stripped = line.strip()
            if stripped in repeated:
                continue                      # drop repeated header/footer band
            if _SUBTOTAL_RE.search(stripped):
                continue                      # never sum a subtotal into the items
            if _GRAND_TOTAL_RE.search(stripped):
                stated_total = _parse_amount(stripped) or stated_total
                continue

            amount = _parse_amount(stripped)
            label = _AMOUNT_RE.sub("", stripped).strip(" .-")
            if amount is None:
                # No amount yet: hold the label as a possible split row.
                pending_description = f"{pending_description} {label}".strip()
                continue

            description = f"{pending_description} {label}".strip() or "(unlabeled charge)"
            pending_description = ""
            items.append(FolioLine(description=description, amount=amount, source_page=page_idx))

    computed = sum((i.amount for i in items), Decimal("0"))
    reconciled = stated_total is not None and computed == stated_total

    flags: List[str] = []
    if stated_total is None:
        flags.append("NO_STATED_TOTAL")
    elif not reconciled:
        flags.append("TOTAL_MISMATCH")

    result = FolioResult(
        correlation_id=correlation_id,
        line_items=items,
        computed_total=computed,
        stated_total=stated_total,
        reconciled=reconciled,
        page_count=page_count,
        flags=flags,
    )
    logger.info(json.dumps({
        "correlation_id": correlation_id, "pages": page_count,
        "items": len(items), "computed": str(computed),
        "stated": str(stated_total), "reconciled": reconciled, "flags": flags,
    }))
    return result

Reconciliation is the audit anchor: parse_folio never trusts its own sum, it checks the stitched line items against the folio’s printed grand total and raises TOTAL_MISMATCH when they disagree. The pending_description carry handles the boundary split — a label with no amount is held and joined to the next amount it precedes, even across a page break — and subtotal lines are dropped so they can never be double-counted.

Step-by-Step Integration

Confirm the document is genuinely multi-page. Route only documents with more than one page through this stitcher; single receipts stay on the standard parser. A one-page folio sent through repeated-band detection loses nothing, but the check keeps the fast path fast.
Detect header and footer bands from repetition, not hard-coded coordinates. Vendors move their headers, so identify the repeated band by finding lines that appear on multiple pages rather than assuming the top N points. Pin the pdfplumber version so extract_text line breaks stay stable:
```
pip install "pdfplumber==0.11.4" "pdfminer.six==20231228"
```

Carry split rows across the boundary explicitly. Keep a pending description when a line has a label but no amount, and attach it to the next amount. Verify a boundary-split charge stitches back into one line:

result = parse_folio("fixtures/hotel_folio_3pages.pdf", "EXP-7742")
assert result.page_count == 3
assert all(item.amount > 0 for item in result.line_items)
assert "(unlabeled charge)" not in {i.description for i in result.line_items}

Reconcile against the printed grand total before posting. Treat the folio’s own total as ground truth and fail loudly on a mismatch:

assert result.reconciled, (
    f"folio did not reconcile: items sum {result.computed_total} "
    f"vs stated {result.stated_total}, flags {result.flags}"
)

Feed reconciled line items to cross-checks, not just the total. Hand the stitched item set to duplicate receipt detection so a folio re-submitted across overlapping expense reports is caught on its line items, and reuse the per-vendor grids from Improving table extraction accuracy in pdfplumber when a folio page is a ruled table rather than free text.
Route mismatches to review. Send any document carrying TOTAL_MISMATCH or NO_STATED_TOTAL to the queue owned by Receipt Error Categorization with both totals attached, rather than posting an unreconciled figure.

Edge Cases & Gotchas

Edge condition	What breaks	Mitigation
Header text shifts a few points per page	Coordinate-based header stripping misses it	Detect repeated bands by text repetition across pages, not fixed y
Charge split across the page break	Description and amount become two orphan rows	Carry a pending description and join it to the next amount
Per-night or department subtotal lines	Summing all numbers double-counts the charges	Skip lines matching the subtotal pattern before accumulation
Multiple grand-total candidates	The wrong figure is treated as the total	Prefer the last grand-total match on the final page; log all candidates
Currency changes mid-folio (FX stay)	One Decimal sum mixes currencies	Accumulate per currency; reconcile each currency separately
Blank continuation page	Repeated-band detection over-counts a blank line	Filter empty lines before band detection (handled in `_page_lines`)
Folio with a native table on some pages	Free-text stitching under-reads the ruled page	Route ruled pages through the table extractor, then merge item sets

Because an unreconciled or double-counted total would undermine SOX Section 404 control expectations, the computed total, the stated total, and every flag are logged per document so the reconciliation verdict is reproducible from the pinned configuration.

FAQ

How do I stop repeated page headers from becoming line items?

Detect the header and footer as bands of text that repeat across pages rather than assuming a fixed region at the top and bottom. Collect the distinct lines on each page, count how many pages each line appears on, and drop any line that recurs on two or more pages before you parse charges. This survives vendors that nudge their header position between pages, which coordinate-based stripping does not.

A charge is split across the page break — how do I stitch it back?

Carry the incomplete part forward. When a line has a description but no amount, hold that description as pending instead of emitting a broken row, and when the next amount appears, join the pending description to it. Because the carry persists across the page boundary, a description at the foot of page one reunites with its amount at the head of page two into a single line item.

Why do my folio totals come out too high?

Almost always because per-night or per-department subtotal lines are being summed alongside the individual charges they already summarize. Match and skip subtotal, carried-forward, and balance-forward lines before accumulation, and reconcile the remaining line items against the printed grand total. If the sum still overshoots, look for a running-balance column being parsed as a charge amount.

Should I trust my computed total or the printed grand total?

Treat the printed grand total as ground truth and use your computed sum only to verify it. Extraction can miss or double a line, so a computed total that matches the stated total is strong evidence the stitch is complete, and a mismatch is a signal to hold the document for review. Never overwrite the stated total with your own figure — flag the discrepancy and route it out.

How do I keep memory flat on very long folios?

Read pages in order and release each one as you go rather than materializing every page image at once. The stitcher only needs each page’s text lines and a small pending-description carry, so resident memory is a function of one page plus the accumulated item list, not of the total page count. For high volume, run the whole parse behind the async batch queue so a long folio never blocks a request path.

pdfplumber Line-Item Parsing — the parent guide this multi-page stitcher extends
Improving table extraction accuracy in pdfplumber — tuning table settings for ruled folio pages
Extracting line items from scanned PDFs with pdfplumber — the OCR-overlay path for image-only folio pages
Duplicate receipt detection — line-item cross-checks that catch re-submitted folios
Receipt Error Categorization — triage queue for unreconciled documents