July 7, 2026
RAG without a vector database? Fact-checking the vectorless hype
98.7 percent versus 31: a spectacular benchmark number is fuelling the claim that vector databases are obsolete. We checked the primary sources — what is real about vectorless RAG, where the marketing begins, and what companies should actually take away from it.
Hardly any architecture question comes up as often in our advisory conversations right now as this one: does our RAG system still need a vector database at all? The trigger is a number that has been circulating through trade media and tech newsletters for months: 98.7 percent accuracy on FinanceBench, a benchmark for questions against long financial documents — versus 31 percent for GPT-4o with search. The result is attributed to an approach that works without a vector database, without embeddings, and without chunking. The buzzword is "vectorless RAG", the open-source project behind it is called PageIndex. We checked the primary sources. The short version: the technology is real and genuinely interesting for one specific document type. The spectacular comparison number is vendor marketing with an admitted methodological imbalance. And the most important lesson for companies lies elsewhere than the headline suggests.
What is behind the buzzword
PageIndex is an open-source project (MIT licence, over 33,000 GitHub stars) by the company VectifyAI. The approach breaks with the standard retrieval-augmented generation pipeline: instead of splitting a document into chunks, writing them into a vector database as embeddings, and retrieving them via similarity search on every request, PageIndex builds a hierarchical tree from a PDF — essentially a machine-readable table of contents with node descriptions. On a query, a language model navigates this tree by reasoning: it reads the descriptions, decides which section is relevant, and descends until the passage is located. What comes back are page and section references instead of anonymous text snippets.
This addresses a real weakness of classic RAG systems that we know well from our own projects: fixed-size chunking tears apart the structure of long documents, and semantic similarity is not the same as relevance. Especially in financial, legal, and regulatory texts, the same terms appear in dozens of sections — which one carries the answer is decided by document structure, not word choice. That a retrieval approach respecting this structure outperforms naive chunking on long single documents is plausible and matches our project experience.
Fact-checking the numbers
The 98.7 percent does not come from the open-source tool.
The number belongs to Mafin 2.5, VectifyAI's commercial financial RAG product built on top of PageIndex. Installing the GitHub repository does not give you the system that topped the benchmark — for complex PDFs, the repository itself points to the vendor's paid cloud service. It is also a self-reported result: no peer review, no independent replication.
The comparison value was not produced in the same setup.
VectifyAI concedes in its own evaluation repository that the comparison numbers for GPT-4o (31 percent) and Perplexity (45 percent) were taken from external sources — and that these systems only covered around two thirds of the benchmark. That is precisely not a comparison "on the same documents". The scientifically robust finding is older and less spectacular: the FinanceBench paper by Islam et al. (2023) already showed that GPT-4-Turbo with a naive shared vector index answers only 19 percent of the questions correctly. That unoptimised vector RAG fails on long annual reports has been consensus since 2023 — it is not news.
The breakthrough is not one.
In the same evaluation repository, VectifyAI lists the specialised competitor products Fintool (98 percent) and Quantly (94 percent, each with coverage caveats). Specialised financial RAG systems — some with vector components — all sit in the same region. The narrative "tree beats vector by a factor of three" only emerges by picking the weakest available comparison number.
The benchmark is a niche.
In its public version, FinanceBench tests 150 questions against long but individually known single documents — exactly the terrain tree navigation was built for. About the core question of most enterprise applications — finding the right information in a corpus of hundreds or thousands of heterogeneous documents — the benchmark says nothing.
Costs do not disappear, they shift
"No vector database, no embeddings" sounds like a leaner stack. In reality, the approach trades infrastructure costs for inference costs. Building the tree costs roughly one LLM call per node — a technical deep dive calculates around 137 LLM calls just for indexing a 131-page report. Every query then requires several sequential LLM calls instead of a single vector lookup: higher latency, higher variable costs. And the approach scales poorly across document boundaries: tree navigation works within one structured document; for corpus-wide search, vector retrieval remains more robust and cheaper as of today. Anyone updating hundreds of documents weekly is building a cost problem with an LLM-indexed tree, not an architecture upgrade.
The valid core
Despite all this, the hype contains a signal worth taking seriously: structure-preserving indexing beats naive chunking on long, hierarchically organised single documents — annual reports, contracts, standards, technical documentation. Their structure carries information; whoever throws it away at indexing time pays at retrieval time. There is a second, often underrated argument: traceability. A retrieval approach that returns page and section references is a tangible advantage in regulated environments — financial services, legal, compliance, audit — because every answer becomes traceable to its source passage.
The consequence, however, is not "remove the vector database" but hybrid: vector or hybrid search finds the relevant documents in the corpus, structure-aware navigation locates the passage within the document. A substantial part of the advertised advantage can even be achieved without any architecture change — with structure-aware chunking along the document outline, section metadata, and parent-document retrieval in the existing stack.
Five recommendations for companies
1. No architecture decisions based on viral benchmark numbers.
Three test questions usually suffice: who measured — the vendor or independent third parties? Were all systems evaluated in the same setup? Is the benchmark representative of your own use case? The 98.7 percent story fails all three.
2. Build your own evaluation set.
Fifty to 150 "golden" question-answer pairs from real user requests, against which every pipeline change is measured. Without your own evaluation, every stack change is flying blind — with it, hype cycles become answerable questions. The effort is manageable, the insight permanent.
3. Improve chunking first, then think about switching stacks.
Structure-aware chunking, document hierarchy in metadata, and parent-document retrieval bring a large share of the advertised advantage into the existing vector stack — at a fraction of the migration cost.
4. Calculate total cost, do not count components.
"Vectorless" trades database and embedding costs for LLM calls at indexing time and on every single request. At high request volumes or frequent document updates, the calculation quickly tips in favour of vectors. What matters are cost per request, indexing cost, and update frequency — calculated over the expected lifetime, not on demo day.
5. Make traceability an explicit requirement.
The strongest point of the reasoning approach is not the benchmark number but the traceability of every answer to its source passage. Anyone operating in regulated markets should demand this property from any RAG architecture — it is achievable with vector retrieval too, if planned in from the start.
Conclusion
PageIndex is a serious contribution to the RAG toolbox — a real alternative to naive chunking for long, structured single documents. The viral comparison number, on the other hand, is a case study in benchmark marketing: a vendor number for a commercial product, external comparison values, a niche benchmark, and an industry-wide claim derived from it. Vector databases are not being replaced. Companies that now hastily shut down their vector stack are solving a problem they do not have — companies that learn to measure their retrieval quality systematically will win regardless of which architecture prevails.
We run our own RAG systems in production with RAG-Wissen and DocMind, and we support companies from the architecture decision through building their own evaluation sets to a working system. If you want your retrieval architecture — or a tempting benchmark story — examined soberly: an initial conversation via digital-opua.ch/kontakt takes thirty minutes and costs nothing.