Users Couldn't Find Answers That Were Already Documented. So the Answers Started Finding Them.
Keyword search replaced with a RAG assistant that answers in plain language, cites its sources, and says 'I don't know' instead of making things up. Support tickets down 40%.
Written by Shubham(opens in new tab), Founder at BestlaTech · Published June 25, 2026
Engagement Type
AI Integration — RAG Semantic Search
Duration
6 Weeks
Client
B2B SaaS
Client
B2B SaaS Platform
Duration
6 weeks
Category
AI Integration / Knowledge Management
Markets
US
Improved Answer Quality
90%
Measured on an evaluation set, not vibes
Fewer Support Tickets
40%
For questions the docs already answered
Answers Cite Sources
100%
Every response links its source documents
Delivered in Production
6 wks
Fixed scope, fixed price
Key Facts
BestlaTech replaced keyword search with a RAG (retrieval-augmented generation) assistant over a B2B SaaS knowledge base — vector embeddings, semantic retrieval, and grounded LLM answers that cite their source documents, validated against an evaluation set before launch. Documented-answer support tickets dropped 40%.
Client:
A B2B SaaS platformWhat was built:
A RAG pipeline: document ingestion and chunking, vector embeddings in pgvector, semantic retrieval, and a chat assistant giving grounded, cited answersTimeline:
6 weeks, including retrieval-quality evaluation before launchEngagement model:
Fixed scope, fixed priceResult:
90% improved answer quality on the evaluation set; 40% fewer support tickets for already-documented questionsThe Challenge
Search That Required Knowing the Answer
Keyword search only matched the docs' exact vocabulary. Users searching in their own words — which is all users ever do — got zero results for fully documented topics.
Tickets for Documented Questions
Failed searches became support tickets. A large share of the queue was agents copy-pasting links to articles that search should have surfaced.
Docs Investment with No Payoff
The knowledge base was genuinely good — and effectively invisible. Every improvement to the docs was gated behind a search box that couldn't find them.
Retrieval Quality First. Grounded Answers Always. Measured Before Launch.
A RAG assistant is only as trustworthy as its retrieval and its restraint. We tuned the ingestion and retrieval against a real evaluation set before shipping, and constrained the assistant to answer only from what it retrieved — with citations, and with 'I don't know' as a first-class answer.
Chunking Tuned to the Docs, Not Defaults
Ingestion and chunking were designed around the knowledge base's actual structure — articles, steps, and version-specific sections — because retrieval quality is decided at ingestion time, not query time.
An Evaluation Set Before a Launch Date
We built a test set of real user questions (many mined from support tickets) with known correct answers, and measured retrieval and answer quality against it until the numbers justified shipping.
Grounded Answers with Citations
The assistant answers only from retrieved documents and cites them. If retrieval comes back weak, it says it doesn't know and offers the support path — a wrong answer is worse than no answer.
A Feedback Loop That Keeps It Honest
Thumbs up/down on every answer feeds the evaluation set. Weak spots show up in the data, get fixed in chunking or prompts, and get re-verified against the same benchmark.
Technologies Used
Key Features
Document ingestion and chunking tuned to the knowledge base's structure
Vector embeddings stored in pgvector — no new database infrastructure to operate
Semantic retrieval matching user vocabulary to docs vocabulary
Grounded LLM answers constrained to retrieved content, with source citations on every response
'I don't know' behavior with a support handoff when retrieval confidence is low
Automatic re-indexing pipeline as documentation is updated
Evaluation harness with a real-question test set, handed over at delivery
User feedback loop (thumbs up/down) feeding the evaluation set
Results & Impact
90% improvement in answer quality, measured against an evaluation set of real user questions
40% reduction in support tickets for questions the documentation already answered
Every answer cites its source documents — users can verify instead of trusting blindly
Docs vocabulary mismatch eliminated — users search in their own words
Evaluation harness handed over, so the team can measure every future change
Delivered in 6 weeks on fixed scope, fixed price
The knowledge base finally pays for itself. Users ask questions in their own words and get real answers with sources — and the support team spends its time on problems that actually need a human, not on copy-pasting links to page four of the docs.
Before vs After
| Before | After | |
|---|---|---|
| Search | Exact-keyword matching only | Semantic — users' own words work |
| Answers | A list of maybe-relevant links | Plain-language answer with citations |
| Unknown questions | Irrelevant results, dead end | Honest 'I don't know' + support handoff |
| Documented-question tickets | A large share of the queue | Down 40% |
| Quality measurement | None | Evaluation set gating every change |
If Your Users Open Tickets for Documented Answers, This Is Solvable.
Any product with a real knowledge base and a keyword search box has the same gap: the answers exist, and users can't find them. RAG closes that gap — when it's built with retrieval evaluation and grounding, not just a demo on top of an API.
B2B SaaS with a Real Knowledge Base
Your docs are good; your search is the bottleneck. A RAG assistant makes the existing investment findable — and ticket deflection is measurable within weeks.
Support Teams Above Capacity
If a meaningful share of tickets are answered with a docs link, that share is automatable — honestly, with citations, and with a clean handoff for everything else.
Internal Knowledge & Ops Teams
The same architecture works over internal wikis, policies, and runbooks — grounded answers with citations, so people trust what they're told.
Your Docs Already Have the Answers. Make Them Findable.
Most discovery calls take 30 minutes. By the end, you'll know whether your knowledge base is a fit for RAG, what retrieval evaluation looks like, and what a fixed-scope build involves.
Book Your Free Discovery Call (opens in new tab)Fixed scope. Fixed price. Zero surprises. Serving US, UAE & Singapore.
Frequently asked questions
How do you stop the assistant from hallucinating answers?
Does our documentation train the model?
What happens when we update our docs?
Which embedding and LLM stack does the RAG search use?
How long does a RAG build take?
Who builds RAG systems and AI-powered documentation search?
RAG or fine-tuning — which do I need for a documentation assistant?
How do you stop an AI assistant from making up answers about our product?
Talk to an expert
Get expert advice from our official advisors
More of Our Case Studies
Explore our diverse portfolio of successful projects and innovative case studies that showcase our expertise in delivering top-notch solutions.
Customer Success SaaS
CAMCESS — How BestlaTech Connected 11 Data Sources into a Real-Time Customer Success Platform
End-to-end enterprise API integration for a B2B customer success SaaS — connecting 11 siloed data sources into a single real-time customer health dashboard, replacing weekly manual reconciliation entirely.
AI Analytics · NLP · Creator Intelligence
YourComments.ai — How BestlaTech Built a YouTube Comment Sentiment Analysis Platform from the Ground Up
Full-stack AI analytics platform built from scratch — fetching YouTube comments at scale, running per-comment sentiment analysis via OpenAI, streaming results through an NDJSON pipeline, and delivering creator and brand intelligence through live Metabase dashboards.
Mobile Web / Performance & Debugging
Diagnosing and Fixing an iOS Safari Navigation Crash on a Top-Tier Indian Bank's High-Traffic Website
Production incident engagement for a top-tier Indian private bank — the website's hamburger menu froze iPhones within seconds of a tap. Root cause diagnosed, patched, and verified across iOS versions in 2 weeks.
InsurTech / Web App Development
A Multi-Project Digital Suite for an Indian Insurance Broker: Premium Calculator, Claims Portal, Partner Dashboard & Lead Microsites
Multi-project engineering retainer for an Indian insurance broker — four tools delivered on one shared platform: an accurate premium calculator, a self-serve claims tracking portal, a partner/agent dashboard, and campaign lead microsites, now serving 10,000+ monthly users.
CMS Development / WordPress / Enterprise
An Enterprise WordPress CMS for a Leading CPaaS Company: 1,000+ Pages, Independent Publishing, Sub-2s Loads
Enterprise CMS build for a leading CPaaS company — a custom component-based WordPress theme, structured content architecture, and multi-language support that lets the marketing team publish independently at 1,000+ page scale with sub-2-second loads.
Legal Tech / CLM / AI Platform
Building an AI-Powered CLM Platform from Zero: Contract Generation, Clause Risk Scoring, Playbooks, E-Signature & Renewals
Full CLM (Contract Lifecycle Management) platform built from scratch for a legal-tech SaaS company (client name withheld) — AI contract generation, metadata extraction and characterization, clause-by-clause risk scoring with suggested alternative language, playbook-driven review, 30/60-day renewal categorization, DocuSign e-signature, and Microsoft Word integration via a Zoho plugin.
Travel & Tourism
Building a Complete Digital Presence for Travel With Bug Travel Company
End-to-end digital transformation for a travel agency - professional responsive travel website development with WordPress CMS, online booking system, tour package management, and SEO optimization for organic lead generation.
On-Demand Services
Full-Stack Mobile App and Web Development for Mechgiri On-Demand Auto Service Platform
Custom mobile app development and web application for on-demand vehicle servicing - featuring real-time tracking, technician management, automated booking system, secure payment gateway integration, and cross-platform admin dashboard.
E-Commerce
Custom E-Commerce Website Development for BugSpeaks Gut Health Brand
Complete digital transformation from offline to online business - custom e-commerce platform development with Django, automated inventory management, secure payment integration, order tracking system, and scalable cloud infrastructure for health and wellness products.
Agritech / Smart Farming
Redesign and Performance Optimization for Letcetra Agritech
Complete frontend modernization for smart agriculture platform - Next.js development with server-side rendering, React component architecture, mobile-first responsive design, advanced SEO optimization, and performance enhancement for agritech industry leader.
FinTech / Financial Operations
How a US FinTech SaaS Unified Stripe, QuickBooks, Salesforce & Slack into One Real-Time Financial Ops Layer
Multi-system API integration for a venture-backed US fintech SaaS — Stripe, QuickBooks, Salesforce, and Slack connected into one real-time financial operations layer, cutting the month-end close from 3 days to 1 hour.
Logistics / Supply Chain
How a US Logistics Operator Unified 5+ Carrier APIs, WMS & ERP into One Real-Time Dispatch Layer
Multi-carrier API integration for a US logistics operator — parcel, LTL, and freight carrier APIs connected with the warehouse management system and ERP into one dispatch layer with automated rate shopping and unified tracking.
AI Automation / Customer Support
How a B2B SaaS Support Team Automated Ticket Triage with an LLM Pipeline — 500+ Tickets a Day, Routed in Seconds
LLM-powered support ticket triage for a B2B SaaS company — automatic classification, prioritization, and specialist routing of 500+ daily tickets, with confidence thresholds and human-in-the-loop review built in.
AI Automation / Accounting
How a US Accounting Firm Automated Invoice & Expense Processing with an OCR + LLM Pipeline — 2,000+ Documents a Month
AI document processing for a US accounting firm — an OCR + LLM pipeline that extracts line items from scanned invoices, auto-categorizes them to the chart of accounts, and flags anomalies, with a human review queue for low-confidence documents.
AI Code Rescue / Security
Rescuing an AI-Generated MVP: 15 Critical Vulnerabilities Patched Before the First Paying Customers Arrived
AI code rescue for a seed-stage SaaS startup — a security audit and hardening of an AI-generated MVP: 15 critical vulnerabilities patched, test coverage built from zero to 80%, and a CI/CD pipeline put in place before launch.
AI Code Rescue / Performance
From Crashing at 50 Users to Stable at 1,000+: Rescuing an AI-Generated App That Failed Under Real Load
Performance rescue for a consumer app startup — an AI-generated app that crashed under real traffic, fixed in 3 weeks: N+1 queries eliminated, memory leaks patched, caching added, and stability proven with load tests.
AI Integration / Knowledge Management
Replacing Keyword Search with a RAG Assistant: Grounded, Cited Answers Over a B2B Knowledge Base
RAG integration for a B2B SaaS platform — LLM-powered semantic search over the product knowledge base with grounded, cited answers, evaluated against a test set before launch. Support tickets down 40%.
AI Integration / HR Tech
AI Resume Screening Inside an ATS: Parsing, Skill Extraction & Explainable Role Matching for an HR Tech Platform
AI integration for an HR tech platform — resume parsing, skill extraction, and explainable candidate-to-role matching built into the existing ATS workflow. Screening 5x faster, with recruiters making every decision.