Starter document Q&A on one computer (AnythingLLM + Qwen2.5 3B)
A careful workflow for chatting with documents on a single machine, including a hallucination test (ask a question the document can't answer).
SoloDocument search / chat with filesUse existing hardwareSensitivity: High (client/confidential)
Hardware
| Item | Approx. price (as of 2026-10-16) |
|---|---|
| Existing laptop/desktop (Mac, Windows or Linux) | Not stated in source |
Source prices are as stated by the source. Retailer offers: prices checked 2026-10-16, may change. Some links may be affiliate links; we may earn a commission at no extra cost to you. This never changes what we list.
Software stack
- AnythingLLM Desktop
- Ollama
- LanceDB (built in)
- AnythingLLM built-in embedder
Models used
- qwen2.5:3b
Good at
- Retrieval stays on the same computer
- Includes a built-in verification test
Weak at
- Scanned PDFs need OCR first
- Built-in embedder is primarily English
- Not an exhaustive audit tool
Rough total cost
Uses existing hardware; software is free.
Getting started
- Install AnythingLLM Desktop and Ollama; download the model while online.
- ollama run qwen2.5:3b, then /bye.
- In AnythingLLM pick Ollama at http://127.0.0.1:11434, choose the built-in embedder and LanceDB.
- Set the workspace to Query mode.
- Embed a test document, ask questions, and check the cited passage. Disable optional telemetry.
Source & credit
Chat with your documents locally — localai.computer
Summarized from the source; quotes are the source author's words. Check the original before buying.