Researchers have introduced Document Retrieval-Aware Chunking (D-RAC), a framework designed to improve Retrieval-Augmented Generation (RAG) by ingesting heterogeneous enterprise documents such as PDFs, Word files, and scans. To overcome the limitations of rule-based extraction and expensive agentic chunking, D-RAC utilizes a two-stage pipeline. First, it normalizes any input format into a deterministic PDF rendering. Second, it employs a single multimodal LLM pass to convert these rendered pages into retrieval-optimized Markdown, which preserves heading hierarchies and transforms complex tables into self-contained prose. Following conversion, the system performs deterministic parsing into ID-addressable units, followed by lightweight LLM-based chunk planning using identifiers rather than regenerating source text. When tested on a 795-page subset of the RAG-Multi-Corpus benchmark, D-RAC processed the corpus in 72 minutes with zero errors, generating 1,748 retrieval-ready chunks. Compared to traditional agentic chunking using frontier models, D-RAC achieved a 95.7% reduction in output tokens and a 75% reduction in processing time. Furthermore, cost efficiency improved significantly, with reductions of 77.8% based on GPT-4.1 pricing and 85.6% based on Gemini 2.5 Pro pricing. The method demonstrates linear scalability for documents exceeding 500 pages while maintaining high observability and determinism.

Read original