Employee LLM server on ~6x RTX 3090 with Open WebUI + LDAP
A business owner plans an on-prem LLM server for employees on a machine with about six RTX 3090s: Ollama, Open WebUI with LDAP logins, nginx for HTTPS, and a token-protected API for internal developers. Main open question: per-department knowledge-base access.
11-50 peopleDocument search / chat with filesWriting assistant$3k+Sensitivity: Very high (legal/medical/financial)
Hardware
| Item | Approx. price (as of 2026-10-16) |
|---|---|
| Server with ~6x NVIDIA GeForce RTX 3090 (24GB each) | Not stated in source |
Source prices are as stated by the source. Retailer offers: prices checked 2026-10-16, may change. Some links may be affiliate links; we may earn a commission at no extra cost to you. This never changes what we list.
Software stack
- Ollama
- Open WebUI (LDAP)
- nginx (HTTPS)
- Parallama (protected API)
Models used
- Not stated in source
Good at
- Fully on-prem with no cloud providers, per the author
- LDAP gives every employee a login
Weak at
- Segregating knowledge bases by role (HR, finance, engineering) was the unresolved problem the author asked about
- Ollama is less suited to many concurrent users than batching servers
Rough total cost
Not stated in source.
Getting started
- Install Ollama on the GPU server and pull models.
- Run Open WebUI with LDAP authentication.
- Put nginx in front for HTTPS.
- Issue API tokens to developers via a protected proxy.
- Plan knowledge-base permissions per group before loading sensitive data.
Source & credit
Reddit post 'How would you go about serving LLMs to multiple concurrent users in an organization...' — r/OpenWebUI
Summarized from the source; quotes are the source author's words. Check the original before buying.