Most RAG pipelines still do the same thing: split a document into flat text chunks, embed them, and stuff the closest matches into a prompt. That works fine for plain text. It falls apart the moment a document has structure that matters — a table referenced three paragraphs later, a figure with a caption, a section that only makes sense in the context of its parent chapter. The pipeline above is a different approach: keep the structure, and store it as a graph instead of throwing it away.
Stage 1: ingestion that preserves hierarchy instead of destroying it
The pipeline starts with multimodal ingestion — PDFs, JPEGs, OCR output — run through layout-aware parsing (tools like LayoutLM) that identifies headers, paragraphs, tables, images, and figures as distinct structural elements, not just a stream of text. Critically, it preserves the document’s hierarchy: chapter, section, page. This matters because a flat chunker throws that hierarchy away at exactly the point where it would be most useful — a text chunk that says “as shown above” is meaningless without a link to the table it’s referring to.
Stage 2: a knowledge graph instead of a flat vector index
This is the core idea. Instead of one embedding per chunk sitting in an undifferentiated vector index, Neo4j stores the actual structure as a graph: a Document node has pages, a page has sections, a section contains text chunks, image chunks, table chunks, and figure chunks, connected by typed relationships — HAS_PAGE, HAS_SECTION, CONTAINS, CAPTIONS, REFERENCES, NEXT_ELEMENT. Every chunk still carries its embedding as a property — this is hybrid storage, entities and vectors together — but the relationships between chunks are first-class, queryable structure, not something you hope the embedding captures implicitly. A table and the paragraph that references it are connected by an explicit REFERENCES edge, regardless of how far apart their embeddings land in vector space.
Stage 3: retrieve, then expand along the graph
Retrieval happens in two steps rather than one. First, ordinary vector search finds nodes whose embeddings are close to the query — this part is identical to a standard RAG pipeline. The difference is what happens next: graph-guided expansion runs a Cypher query outward from those seed nodes to pull in contextually linked content — parent sections, sibling elements, referenced tables, surrounding figures — the things a flat vector search would miss because they’re structurally relevant but semantically distant in embedding space. A query about a chart’s conclusion can pull in the chart’s caption and the paragraph that references it, even if that paragraph’s own embedding wasn’t a close match to the query.
What reaches the LLM at the end is recontextualized content — header, image, caption, referenced paragraph, table, and metadata bundled together — rather than a handful of disconnected chunks the model has to guess at reassembling. That’s the whole point of doing this in a graph: it maintains semantic links and avoids the information loss that a purely flat chunking-and-embedding pipeline accepts as a cost of doing business.
Why this matters more as documents get messier
The gap between flat RAG and graph-based RAG is invisible on clean, single-topic text and enormous on real documents — technical specs with cross-referenced tables, research papers with figures cited pages later, contracts where a clause only makes sense next to its exceptions. Recent work on multimodal GraphRAG splits roughly into two camps: knowledge-based approaches that extract entities and relations into a graph, and index-based approaches — closer to what’s diagrammed here — where the graph indexes the original multimodal content directly rather than trying to fully abstract it into entities first. The index-based approach trades some of the tidiness of a pure knowledge graph for something that degrades more gracefully on messy, real-world documents, which is usually the trade worth making in production.
References
- HVM-GraphRAG: Holistic-View Multimodal Graph Retrieval-Augmented Generation on Complex Documents (arXiv)
- Multimodal RAG for Unstructured Data: Leveraging Modality-Aware Knowledge Graphs with Hybrid Retrieval (arXiv)
- mKG-RAG: Multimodal Knowledge Graph-Enhanced RAG for Visual Question Answering (arXiv)
- Efficient Knowledge Graph Construction and Retrieval from Unstructured Text for Large-Scale RAG Systems (arXiv)
Leave a Reply