A Typical Enquiry — Internal Enquiry Desk, from Idea to Assessment
This is how enquiries reach us: already phrased as a solution. This entry stands for a type of request we see regularly — an organisation holds expertise that has grown over years in internal stores, and wants a chatbot on top of it. Here it was a Swiss research institution with a specialist enquiry desk. What follows is our method: which questions we ask first, why a RAG system did turn out to fit, and where we contradicted the client.
The enquiry as it arrived
About ten people answer specialist enquiries, researching in internal note collections grown over years. Newcomers take a long time to learn where things are. The wish was phrased as «a chatbot that answers natural-language questions, with source attribution, at least 90 percent correct, exclusively on Swiss infrastructure». Like almost every enquiry it therefore already contained a solution — while the actual question was: how do ten specialists get faster access to knowledge the organisation already holds?
What we checked first
Three questions before any talk of architecture. First: which questions actually get asked — and how many of them need a phrased answer rather than just a location? Second: what is the material like? Free text without dependable structure behaves differently from a maintained database. Third: are there questions about relationships and quantities that a text search fundamentally cannot answer, because it cannot count? This clarification is exactly why step one of the quote is a requirements and architecture workshop rather than a prototype.
Why the simpler routes fall short here
Better tagging alone does not help, because the questions arrive paraphrased in domain language that does not match the wording of the notes. Pure full-text search fails at the same point. Hybrid search would get considerably further and therefore stays in the proposal as the foundation — it is the layer everything else stands on. What tipped the decision towards a generated answer was the nature of the material: the answer to a typical specialist question rarely sits in one place but is spread across several notes that must be brought together. A knowledge graph would be the more precise route but presupposes a structuring that does not exist and whose creation would be a project of its own.
Where we pushed back
The client named at least 90 percent correct answers as the success criterion for the pilot. In the quote we stated in writing that this threshold is not reliably reachable after the first proof of function but only after the build-out stage — and that it depends on the jointly chosen model family operable in Switzerland. Plus the more uncomfortable half: what «correct» even means is defined jointly before the project starts, measurement runs against a catalogue of twenty to forty test questions with model answers supplied in advance, in a moderated session, with one rework cycle included. Confirming a number you cannot later evidence is not client-friendliness; it is a deferred argument.
What the pilot deliberately excludes
Replacing the existing note collection with a central knowledge base, automatic write-back of corrections, provision to external users, an extended guard layer against prompt injection, high availability with an SLA, single sign-on, continuous synchronisation of new content, and further languages. Also excluded: fine-tuning the language model — it changes style and format but not the findability of knowledge, and here would be pure cost. All points are accounted for architecturally. A pilot that contains everything is no longer a pilot but a project without an exit point.