Hierarchical Document Structure Extraction for Complex Multi-Section Documents

Hierarchical document structure extraction for complex multi-section documents must recover the relationships as well as read the characters. Production-ready pipelines must preserve heading ancestry, contextual links, and cross-page continuity so that agents can navigate and reason over the output. Page-level accuracy establishes whether content was read correctly. It does not establish whether that content still belongs to the right section, table, field, or record.

Why character recognition is the wrong place to look when hierarchical extraction fails

Accuracy numbers can look strong on a dashboard while an agent encounters a document it cannot navigate reliably. A parser may return every heading verbatim yet assign a subsection to the wrong parent. The text is right, but its scope is wrong. Because that assignment governs the content beneath the heading, the mistake changes how retrieval and reasoning use otherwise accurate passages. Improving character recognition does not repair that relationship.

This distinction separates perceptual errors from relational ones. A missing character changes what a string says, while a missing parent-child link loses the tie to what the surrounding content belongs to. Both need to be tested, but each needs a different test.

What relational recovery actually requires that page-level parsing does not

TOC hierarchy recovery reconstructs a complete, ordered heading tree across pages, with the correct parent assigned to every node. Two headings can occupy the same level while governing different branches, so recovering their order without their parents leaves sectional scope unresolved. LongDocBench annotates thousands of heading nodes across multiple depth levels in financial reports, textbooks, and academic papers to test this distinction.

Contextual relationship recovery identifies typed links from tables and figures to their captions, notes, and sources. Those links can be one-to-many and span substantial distances. Extraction may keep both a table and its source text but lose the link between them. The benchmark annotates 3,258 contextual relationships across thousands of table and figure objects, making the link itself an object of evaluation.

Table-form documents add a related structural task. Their semantic sections may contain separate local grids, with checkboxes and widgets whose meaning depends on their field labels. FormStruct-Bench treats recovery of this multilevel organization as distinct from reading visible content or recognizing a conventional grid. When evaluation measures these relationships only indirectly, or not at all, pipelines receive little incentive to preserve them.

How benchmark design has obscured the relational gap until recently

Page-level benchmarks serve a legitimate and necessary purpose. They evaluate text, tables, and layout through measures that include edit distance, detection metrics, and formula-structure similarity. These tests establish whether a parser recognizes the elements on a page. They do not directly establish whether a heading on one page governs content on another. Even multi-page protocols can obscure the distinction when hierarchy and cross-page merging contribute only to a broader composite score.

LongDocBench, from Ant Group, isolates TOC hierarchy and contextual relationship recovery in a reproducible testbed with human-verified annotations. FormStruct-Bench supplies a complementary diagnostic view for table-form documents. It uses five primary metrics and three structure-specific diagnostics across page, schema, and component levels, with slices over difficulty, structural constraints, and visual degradation. This makes it possible to trace an aggregate result back to a particular structural weakness.

ExtractBench extends the evidence to schema-guided enterprise extraction by evaluating value accuracy, record completeness at scale, and source grounding. Together, these benchmarks complement the older tests by showing relationships that a page score cannot fully describe. Rather, they distinguish a necessary condition for reliable extraction from the additional conditions that document-level use imposes.

What the benchmark results reveal about where current parsers actually fail

LongDocBench finds that five representative parsers score above 93.5 at the page level, yet the best tested result for TOC hierarchy recovery reaches only 0.55 Macro TEDS. Contextual relationship recovery reaches at most 0.63 Macro. The contrast is substantial within the tested systems. Strong recognition of page elements does not translate into strong recovery of their document-level organization, and heading ancestry is particularly difficult.

The same benchmark also tests the downstream value of those structures. Human-verified TOC hierarchies and contextual relationships together improve question-answering accuracy over a fixed retrieval baseline. Their benefits are complementary. This supports treating heading navigation and contextual linkage as separate capabilities, rather than assuming that improvement in one will cover the other.

FormStruct-Bench finds a similarly wide gap in table-form documents, where the best document-level score is high but the best fine-grained structural score is still below 18%. ExtractBench identifies a different expression of the gap. Commercial VLMs perform well on short documents but often truncate record lists on long ones. The values that were extracted can all be right even when records are missing, which value accuracy alone does not show.

How structural failures propagate into downstream agent behavior

In a financial report, an incorrect parent assignment attributes subsection content to a sibling or parent section. Retrieval can then select a passage under the wrong scope, and generation reasons from that selection.

Contextual failures take a different route through the workflow. When a table loses its source link, the value remains available but the agent cannot trace it to its data origin. In a table-form document, a checkbox can retain its detected state while losing the field label that defines what the state means. Both outputs can look readable and structurally plausible without preserving the evidence needed to interpret them.

Two mentions of the same party, amount, or date can resolve to different canonical values, causing reconciliation to fail or silently accept an inconsistency. ExtractBench captures long-list incompleteness under its T1 challenge tag, which concerns recovering every record in repeated structures spanning many pages. A truncated schedule can thus look complete even when whole rows are missing.

Hierarchical chunking allows retrieval to move from a passage to its parent context when the passage alone is insufficient. Flat chunking discards that route. Once the relationships have been removed from the representation, downstream reasoning cannot rely on them to correct an ambiguous retrieval.

Why the failure pattern looks different across document-intensive industries

Healthcare workflows combine nested explanation-of-benefits tables with prior authorization forms across multi-page patient charts. A collapsed table hierarchy can misattribute a benefit category, while a dropped widget-field link can send semantically wrong output into an EHR. In finance, a fund-holdings schedule with missing rows can cause reconciliation to fail even when every extracted value is correct, and model risk management rules also require validation and documentation.

Logistics documents vary by carrier and shipper. Dense, multi-column packing lists require item codes to retain their regional and positional attribution, because a correct string assigned to the wrong column is still an incorrect extraction. Real estate workflows introduce variability across addendums, county-specific disclosures, and deeds. Before extracting fields from a purchase-and-sale agreement with attachments, the pipeline must recover which pages belong to each component.

These differences make a uniform document treatment insufficient. The common dependency is relational integrity, but the relevant relationship may be table ancestry, column membership, or document membership. Domain-aware evaluation therefore needs to reflect the structures that the workflow actually consumes, rather than substituting a generic page score for all of them.

What a pipeline architecture must handle to recover relational structure reliably

The architectural response must address structure throughout the pipeline. Layout analysis must find headings and work out their levels and parent assignments. That requires attention to the structural signals of real enterprise document types. A heading detector alone does not establish a tree. Hierarchical chunking must also keep that ancestry attached to the sections and passages.

Contextual linking requires a document-scope mechanism. Independent page processing cannot, by itself, recover one-to-many links between tables and spatially distant captions, notes, or sources across pages. The pipeline must also preserve the local structure of table-forms, where semantic sections and their grids define widget-field relationships. These are distinct responsibilities even when they share the same underlying text.

For a long schedule, the system must establish how many rows should be present and compare that expectation with the output. Relational validation must also check whether invoice totals match line-item sums, dates are in a logical order, and cross-page entity references resolve to the same values. These checks address errors that individual field confidence can miss.

Word- and page-level source metadata allows a reviewer to trace an extracted value or incomplete schedule back to the source region. Without that traceability, locating the cause becomes manual and slow. Grounding is therefore part of the extraction architecture.

How confidence scoring and evaluation design catch relational failures before they reach agents

Confidence is useful only when it is calibrated to the prediction being judged. A high score on a correctly read field does not establish that the field belongs to the right section or that its containing schedule is complete. The prediction target must include the structural task.

Calibration and relational validation serve different purposes. Calibration establishes whether reported confidence reflects the likelihood of correctness at a given threshold, while consistency rules test relationships that may be invalid despite confident local predictions. Neither substitutes for the other. Post-extraction validation can use these rules without changes to the model, so relational checks can improve reliability in practice as well as be addressed during training.

Evaluation must also distinguish correctness from attributable correctness. ExtractBench scores word- and page-level grounding along with value accuracy because the output must help with review as well as return fields that look right. In regulated workflows, confidence and grounding must support an audit trail from a decision back to its source.

Continuous evaluation supplies the feedback that a one-time benchmark cannot. Built-in infrastructure for testing against held-out ground truth allows teams to detect structural regressions before downstream failures expose them. The evaluation must retain separate visibility into hierarchy, contextual links, and completeness. Folding them back into one aggregate number would recreate the blind spot that document-level benchmarks were designed to reveal.

What distinguishes a production-ready extraction platform from one that passes page-level benchmarks

Teams testing platforms should use documents with the structures they work with in production. Clean, single-page, or two-language test sets do not demonstrate performance on nested financial reports, variable-format contracts, or multi-page patient charts. The test set establishes what the result supports. A high score outside those conditions cannot stand in for evidence that the platform handles them.

Within that test set, TOC hierarchy accuracy, contextual relationship recovery, and long-record completeness should be explicit evaluation dimensions. Word- and page-level grounding must also be verifiable. These requirements make it possible to distinguish an accurately read fragment from a complete, navigable, attributable extraction.

Deployment requirements remain an independent condition of readiness. Strong structural extraction does not compensate for an inability to meet applicable HIPAA, SOC 2, or GDPR requirements, or to provide a self-hosted configuration where sensitive data requires one. For workflows processing protected health information, HIPAA requirements include encryption, access control, audit logging, and transmission security.