A private AI assistant should earn your team’s trust before it earns a place in their daily work. A smooth demo proves that the chat box works. It does not prove that the answer is correct, current or safe for the person asking.
Let’s be honest. A confident paragraph about your leave policy can look wonderful until somebody follows the wrong version. Then your shiny new tool becomes another inbox problem.
At UltiMedia, we recommend testing the whole private AI assistant workflow: documents, search, permissions, answers and recovery. This article sets out seven acceptance tests for South African business teams. We checked the linked technical documentation on 18 October 2026. The proposed tests and examples are our recommendations, not reported client results.
How do you test a private AI assistant?
Test a private AI assistant against approved questions, known answers and user permissions. Check whether it retrieves the right documents, supports its claims with accurate citations, refuses unsupported answers and respects access boundaries. Then test outdated information, misleading documents and service failures before allowing staff to depend on it.
Start with one useful job, such as finding approved product guidance or internal procedures. Give the pilot a named owner. Keep purchasing, payroll changes and customer commitments outside its authority while you establish whether its answers deserve attention.
1. Define what your private AI assistant can access
Write down what “private” means for your business. Which machines process questions? Where do documents, search indexes, logs and backups live? Which providers receive any part of a request? A label on a sales slide cannot answer those questions.
Ollama documents a local-only mode that disables its cloud models and web search through the OLLAMA_NO_CLOUD=1 setting or its server configuration. It requires a restart. That establishes a specific product control; it does not describe every component of your application. See Ollama’s cloud and local-processing FAQ.
For a private AI assistant, our suggested test is simple: trace one harmless sample question through every service. Include the component that turns documents into searchable numerical representations, often called embeddings. Also inspect monitoring and error reporting. Record the actual destinations rather than assuming that a local model keeps everything local.
- Draw the data path.
- List each provider and the information it receives, including diagnostic logs that your team might otherwise overlook.
- Record who can change the configuration.
If you are comparing deployment options, our local LLM deployment service describes that part of UltiMedia’s offering. Choose the architecture around the information and operating requirements you actually have.
2. Check the search before judging the answer
Give the system questions whose supporting documents you already know. Ask it to return the passages it used. Does it find the current procedure, or merely something that sounds similar?
LlamaIndex separates retrieval evaluation from response evaluation. The first checks the retrieved material; the second examines the generated answer against context, the question or reference guidance. That distinction helps locate a failure instead of blaming everything on the model. See LlamaIndex’s evaluation documentation.
Your private AI assistant test set should include normal wording, abbreviations and questions that require two documents. Ask staff to supply the phrases they actually use. For a Cape Town distributor, that might mean a product nickname alongside its formal catalogue name.
Include the languages your staff actually use, South African date formats and local product names in the same tests before rollout.
Use reference passages to judge missing evidence. Ragas describes context recall as a measure of how much relevant information retrieval captures and explains why it needs a reference for comparison. Its different implementations use different inputs. See Ragas context recall.
Our practical rule: fix missing or wrong source material before polishing the answer’s tone. A friendly explanation of the wrong document remains the wrong answer. Keep the question, expected source and retrieved passages together so the builder can reproduce the problem.
3. Make every citation carry its weight
A citation should let a colleague check the exact claim without a scavenger hunt. Open the link. Find the passage. Check the version. Then compare what the answer says with what the document actually permits.
Ragas defines faithfulness around whether the answer’s claims follow from the retrieved context. Its score measures that consistency, not whether the underlying document contains current business truth. See the Ragas faithfulness metric. Our interpretation is straightforward: an answer can faithfully repeat an obsolete rule.
Test your private AI assistant with a document containing an exception. For example, create a fictional delivery policy where a standard lead time excludes special orders. Ask about a special order and check whether the response preserves that exception.
Mark a citation as failed if it points to the wrong file, omits the relevant condition or links to something the user cannot inspect. Treat a neat list of filenames as a starting point for review, not proof. The answer must earn the link.
4. Test permissions with separate accounts
Use synthetic documents and two test accounts with different access. Give one account access to a fictional management file and deny it to the other. Try ordinary questions, summaries and requests that combine information across folders.
Qdrant documents payload-based partitioning: records carry a tenant identifier and queries use a matching filter to restrict results. That is a database mechanism developers can use, not a complete employee-permissions system by itself. See Qdrant’s multitenancy documentation.
For your private AI assistant, require the application to derive permissions from the signed-in identity. Do not let a typed instruction choose somebody else’s access group. Ask the implementer to demonstrate that the same rules cover search results, source previews and downloadable files.
Test the database boundary too. Qdrant warns that self-hosted open-source deployments require explicit security configuration, including authentication, network restrictions and encrypted connections. Its local installation alone does not establish those controls. See Qdrant’s security guidance.
Our proposed release rule is strict: any observed disclosure to an unauthorised test account blocks launch. Zero observed leaks in a small test does not prove perfect security. It establishes a minimum condition, followed by further review and monitoring.
5. Give it missing and conflicting information
Ask a question that your approved documents cannot answer. Then provide two fictional procedures with different effective dates. Watch whether the system identifies the gap or quietly invents a tidy compromise.
A useful private AI assistant should make uncertainty actionable. We recommend an answer that names the missing detail and points to the responsible person. “I cannot confirm this from the approved documents” beats a confident guess dressed in your company’s tone.
Agree the document precedence rules before testing. Who approves a policy? Which version takes effect? What happens when the document owner withdraws it? Record those answers in the application design and your operating instructions.
Then change a source and repeat the question. Check the search index, saved conversations and answer caches where applicable. Set a measurable refresh target that fits your workflow. A stock instruction and an annual handbook may need different update policies; choose deliberately.
6. Put misleading instructions inside a test document
In an isolated test library, add a harmless fictional document that tells the assistant to ignore its task and produce an unrelated answer. The test asks whether source text can redirect the system, rather than simply inform its response.
OWASP describes this as an indirect prompt-injection risk when instructions arrive through outside material such as files or websites. It also says retrieval-augmented generation does not fully remove that risk. See OWASP’s prompt-injection guidance.
Keep your private AI assistant limited to its approved job. OWASP recommends least privilege and human approval for high-risk actions. Our suggested pilot therefore answers questions without tools that send messages, change records or export document collections.
Record any redirection, unexpected disclosure or attempted action. Treat a failed test as a design problem to investigate. Treat a passed test as evidence about that particular case, not a promise that nobody can ever manipulate the system.
7. Measure usefulness when something breaks
In a test environment, interrupt the document service or model connection. Ask an ordinary question. Does the interface explain the failure, or does it leave staff waiting for a reply that will never arrive?
A private AI assistant needs a usable fallback. We recommend a clear route to the original document library or an assigned colleague, plus an alert for the operator. Test recovery after service returns, including whether old permissions or outdated content reappear.
Measure the whole task: reading the answer, opening sources, correcting mistakes and completing the work. Compare that with the existing process. Include hosting, maintenance and staff review when assessing cost. The following calculation illustrates the method; it does not predict savings.
Suppose a fictional team asks 100 questions weekly. Manual lookup takes four minutes per question. Assisted lookup plus checking takes two minutes. The difference is 200 minutes, or three hours and twenty minutes, before maintenance and exception handling. Replace every input with your own measurements.
Turn the tests into a release decision
Give your private AI assistant a small, versioned acceptance sheet. Record the question, user role, expected source, expected behaviour, actual result, reviewer and follow-up. Keep failed examples; they become useful regression checks after changes.
For a starter exercise, we suggest 30 questions: ten routine lookups, five needing multiple documents, five with missing or conflicting information, five permission checks and five misleading-document or outage cases. This is an illustrative starting set, not a statistical assurance standard.
- Separate serious failures from minor wording issues.
- Have the document owner review important answers; give the person maintaining the system the underlying evidence, not just a complaint.
- Repeat failed cases after repairs.
- Retest when documents, models, search settings or permissions change, and record which versions produced each result.
Judge the private AI assistant by those records. An overall average can hide a permission failure or a confidently wrong answer about an important procedure. Decide acceptance conditions per task, including when the assistant must hand over to a person.
Our AI agent supervisor article discusses the broader oversight role. For this pilot, keep ownership concrete: one person approves the knowledge, another maintains the system, and staff know where to report a bad answer.
Private AI assistant FAQs
Does a private AI assistant need internet access?
A private AI assistant can use a local model, but the complete workflow may still depend on external search, storage or monitoring services. Inspect each component and test the intended offline behaviour. Define which features must work without internet access and which should clearly report an unavailable dependency.
Does a source link prove an answer is correct?
No. Open the source and check whether it supports the specific claim, including exceptions, dates and scope. A correct link can still point to an outdated document. Ask the document owner to approve the reference material, then evaluate the answer against that approved version before relying on it.
How should we pilot a private AI assistant?
Choose one internal task, a small approved document set and a named owner. Build questions with known answers, test access restrictions and record failures. Let staff compare assisted work with their existing process, including checking time. Expand only after the evidence meets the acceptance rules you agreed.
Should the assistant make business decisions?
For an initial knowledge pilot, we recommend limiting it to finding information and preparing answers. Keep actions that change records, spend money or commit the business under separate controls. Define who approves those actions and test that boundary before adding tools that can carry them out automatically.
Build a private AI assistant your team can check
UltiMedia is a Cape Town-based AI-first digital agency offering local and private LLM deployment and integration. Our recommendation: prove one useful workflow, make its limits visible and keep the evidence close to the answer.
Ready to test a private AI assistant against your real business needs? Discuss a private LLM pilot with UltiMedia. Bring one recurring question, the approved documents and the people who know the right answer.