Inside Gwydion: the Drupal RAG pipeline behind Visit Wales’ AI Assistant

You might have noticed more websites offering conversational search tools lately. At their simplest, these AI Assistants work by reading through a vast library of information and then summarising the most relevant parts to answer your specific questions in plain English. It is a process that turns a static website into a helpful, interactive guide.

For Visit Wales, we have built Gwydion. The project started life as a Hypothesis-Driven Development test rather than a top-down feature request. A workshop with the client identified conversational search as a priority, and a subsequent A/B test confirmed that this would be a popular feature with users. 

Gwydion sits on top of our existing Drupal 11 multisite platform for Cymru Wales. It is split across three purpose-built backend modules: an AI provider abstraction layer, a vector ingestion pipeline, and the retrieval/orchestration layer that handles a live conversation, plus a standalone React frontend. The chat interface itself is a component injected into the global footer on every relevant page, managed by a “sticky launcher” that expands to open the conversation window. 

Ingestion: turning the drupal CMS into a retrievable knowledge base

Before Gwydion can answer anything, every relevant entity in the CMS (such as articles, accommodation, attractions, and event products) must exist as a searchable, embedded record. That happens continuously, not as a one-off batch job.

Whenever a supported entity is saved, unpublished, or deleted, a hook queues a lightweight sync target (entity type, ID, bundle, langcode) onto one of two Drupal queues: one for indexing and one for removal. A cron-driven queue worker then does the real work:

1. Render the entity as a visitor would see it. We use Drupal’s own view builder against the entity’s full view mode, in the correct language and theme context, rather than reading raw field values. This matters more than it sounds — it means Gwydion’s knowledge base reflects the same computed, templated content (formatted prices, resolved taxonomy terms, rendered paragraphs) that a human sees on the page, not a raw database dump.
2. Convert the rendered HTML to Markdown. LLMs retrieve and reason over Markdown far more reliably than raw HTML; it keeps structure (headings, lists, links) without markup noise burning tokens.
3. Diff before embedding. Every record is content-hashed. If the hash hasn’t changed since the last sync, we skip re-embedding entirely and just refresh metadata. Embeddings aren’t free, and with 15,000+ records across nine sites, needless re-embedding adds up fast.
4. Embed and upsert. Changed content gets sent to an Azure OpenAI embeddings deployment, and the resulting vector is upserted into a vector_records table in Azure Database for PostgreSQL with the pgvector extension enabled, with one row per entity, per language.

Because this rides on Drupal’s native translation system, a Welsh translation of a page produces its own row, its own embedding, and is retrieved independently. There’s no separate “translate the knowledge base” step bolted on afterwards.

A single question’s round trip through the pipeline

When a visitor sends a message, it hits a streaming controller behind Server-Sent Events, and goes through several checks before it ever reaches a model:

• A regex-based PII check runs first, client-side of any LLM call. If a message contains something that looks like an email address, UK phone number, card number, or postcode, we short-circuit immediately. There is no embedding call, no chat call, and no logging of the raw message downstream.
• Origin/Referer validation. The endpoint only accepts requests whose origin matches the site itself, which rules out the conversation endpoint being called cross-origin from somewhere else.
• Rate limiting. This is a sliding-window counter keyed by session (falling back to IP) in Drupal’s cache backend, configurable per environment.
• Message-length validation. This ensures messages are rejected before they cost us a model call.

Once a message clears those gates, the real RAG logic kicks in:

1. Query condensation. Conversational search has a well-known problem: “what about somewhere cheaper?” means nothing without the previous turn. So before we search, a small, cheap LLM call rewrites the latest message into a standalone question using the conversation history, such as “somewhere cheaper than the Northern Snowdonia cottages we just discussed“. It’s that rewritten query which gets embedded and searched, not the visitor’s literal words. History is deliberately short: a sliding window caps it at the last three exchanges, which keeps the condensation prompt cheap and stops long conversations from quietly drifting off-topic. Every history entry is re-checked for PII on the way in, too (not just the live message), so nothing a visitor typed earlier in the conversation lingers in what gets sent to the model later.
2. Vector search. The condensed query is embedded and compared against the vector_records table using pgvector’s cosine-distance operator, filtered by the visitor’s current language and limited to a small top-K. If nothing relevant comes back, we stop here and tell the client there’s no context to answer from; Gwydion isn’t allowed to fall back to the model’s general knowledge.
3. Prompt assembly. The retrieved passages, the conversation history, and the question are assembled into a chat request alongside a system prompt — resolved dynamically per task (there’s a task → prompt mapping, so “Gwydion chat” and other AI-assisted CMS features each get their own prompt, and each prompt is itself translatable through Drupal’s own config translation system, the same mechanism editors use to translate any other piece of site copy).
4. Generation and streaming. The assembled request goes to Azure OpenAI (UK South, for data residency) through a thin abstraction layer built on Drupal’s ai module. Swapping which Azure deployment answers a given feature is a five-line class, not a rewrite. The response streams back token-by-token over SSE and is rendered live in the chat UI as it arrives.
5. Structured extraction without JSON mode. The system prompt requires the model to end every substantive answer with machine-readable [Source: id] tags for every citation and a one-line [Summary: ...] tag. We parse those out of the accumulated stream with a couple of regexes — a deliberately low-tech way to get structured metadata (which sources were actually used, a plain-text summary for analytics) out of a model that’s simultaneously writing free-flowing prose, without needing a separate structured-output round trip.

The guardrails live in the prompt, not just the infrastructure

A lot of what makes Gwydion trustworthy isn’t code at all. It’s a long, carefully engineered system prompt, treated with the same rigour as any other spec and iterated through multiple rounds of QA feedback. A few rules are worth calling out because they map so directly onto real failure modes we saw in testing:

• The geography rule. Ask for a “day trip” without naming a region and Gwydion must ask which part of Wales you mean. Once you’ve answered, it’s explicitly forbidden from mixing North Wales locations into a South Wales itinerary; this is a subtle but very real hallucination risk when retrieval returns strong matches from opposite ends of the country.
• Count what you’re about to say. The model is instructed to count the results it’s about to list before writing its intro line, specifically because “here are five great options” followed by three bullet points was a real risk.
• No citation, no claim. Contact details, prices, and recommendations must be traceable to a specific [page title](URL) from the retrieved context. The model is explicitly told never to borrow a phone number from one venue and present it as belonging to another.
• Instruction resistance. A dedicated section exists purely to stop prompt injection and persona-hijacking attempts, such as “ignore previous instructions”, “respond as if you were X”, and encoded payloads. This includes the rule that a message mixing a legitimate question with an injection attempt gets refused wholesale, rather than trying to salvage the “real” half.

Multilingual by construction, not by translation layer

The “supports almost every language” story isn’t a separate subsystem; it falls out of decisions made elsewhere. Retrieval is filtered by langcode against content that was ingested per-translation in the first place. 

Generation goes through a general-purpose multilingual model rather than a bespoke translation pipeline, with a single instruction in the system prompt telling it to reply in whatever language the visitor wrote in. That instruction is explicitly exempted from the persona/injection-resistance rules, so it can’t be argued away. 

Gwydion is live in production for Visit Wales. The lesson we are taking into the next one is that a RAG pipeline is mostly not an AI problem. It is a content problem, a cost problem, and a prompt-discipline problem, and the model is the easy part.

Key Takeaways

Ultimately, the Gwydion project demonstrates that building a successful RAG pipeline is primarily a content, cost, and prompt-discipline challenge rather than just an “AI problem.” By focusing on continuous indexing, precise prompt rules, and robust guardrails, organisations can create reliable, production-ready AI tools using their existing website architecture.