Ask your own documents

Before running

Make the setup exact.

Model: Select an instruction model and record its exact revision and quantization before testing.

Stack: Local inference server + retrieval application. Select and pin versions before installation.

Record your hardware SKU, memory allocation, topology, dataset revision and thresholds. The downloaded manifest is a starting specification; it does not install software.

Evaluation plan

1. Choose 20 non-confidential documents and write 20 questions with expected source passages. Include five questions the documents cannot answer.
2. Select a runtime supported by your exact OS and accelerator. Record the model revision, embedding model, chunking settings and all application versions.
3. Install following the runtime documentation; index the test documents. Confirm where files, embeddings and logs are stored.
4. Run the fixed question set at one active request, then at your intended concurrency. Export answers, citations, timings and failures.
5. Disconnect external network access for a repeat test if offline use is required. Document any remaining dependencies.

What counts as useful?

• Every factual answer points to a supporting passage.
• Record supported-answer and correct-abstention rates; set your acceptance thresholds before running.
• Record median and p95 first-token latency, full response time, concurrency and wall power.

Known limits

Retrieval quality can dominate model size.

No performance or compatibility has been measured for this package.