Customer support bot over company docs (Open WebUI + Llama 3 8B)
Hosting provider HOSTKEY built a tech-support assistant for its front-line team: cleaned Markdown docs, Open WebUI RAG (ChromaDB), a custom model with a strict system prompt.
11-50 peopleCustomer supportDocument search / chat with filesUnder $1kSensitivity: Standard
Hardware
| Item | Approx. price (as of 2026-10-16) |
|---|---|
| On sale −6% MSI GeForce RTX 5060 Ti 16GB Ventus 3X OC (Newegg, related current offer) | $579.99 |
| GPU with 8GB VRAM (enough for llama3-8b per source) | Not stated in source |
Source prices are as stated by the source. Retailer offers: prices checked 2026-10-16, may change. Some links may be affiliate links; we may earn a commission at no extra cost to you. This never changes what we list.
Software stack
- Ollama
- Open WebUI (RAG, ChromaDB)
- sentence-transformers/all-MiniLM-L12-v2
- Bash cleanup script
Models used
- llama3-8b-instruct
Good at
- Answers with links to the docs
- Strict prompt keeps it on-topic
- Surfaced inconsistencies in their documentation
Weak at
- Raw Markdown noise hurt retrieval until cleaned
- Hybrid search added 20-40s latency
- API integration with RAG was not working yet (July 2024)
Rough total cost
Not stated in source.
Getting started
- Convert docs to clean plain text with a source URL appended.
- Open WebUI Admin > Documents: choose embedding model; Top K 10, chunk 1024, overlap 100.
- Upload docs with a collection tag.
- Create a custom model with a strict system prompt; temperature 0.3.
- Attach the collection; re-add it whenever docs change.
Source & credit
Let's build a customer support chatbot using RAG... (08.07.2024) — HOSTKEY blog
Summarized from the source; quotes are the source author's words. Check the original before buying.