01
Question set (gold set)
Built from your documents, stratified so a pass is not just "the model said no".
Your enterprise buyer will ask how you know your AI answers are correct. In four business days you get the answer in numbers: a test set built from your own content, the failure list, the fixes ranked by risk, and a harness your engineers re-run on every release.
The Go-Live Evidence Pack is a fixed-scope evaluation of one AI feature against one corpus. It measures what your buyers will ask about, writes down every number with its sample size, and hands over the code that produced it.
Product, corpus, the questions your buyers ask, your release date, one named owner. Scope confirmed in a one-page SOW.
Up to 60 questions written from your documents and your real user questions. You confirm five samples.
Retrieval, citation and groundedness measured. One agreed change applied and re-measured. Failure taxonomy finalised.
Scorecard, dossier, harness, a 30-minute readout, and fixes ranked by risk. Owner signs acceptance.
Built from your documents, stratified so a pass is not just "the model said no".
Recall@5, context precision, citation correctness, grounded-answer rate, refusal correctness.
Counted causes with question ids — not a list of adjectives.
One guardrail or configuration change, agreed at kickoff, measured twice.
Open-source stack, no licence fees. Your engineers re-run it themselves on every release.
Scorecard, method note, failure register, fix list, evidence index — one document a buyer can read.
These are the published results of a 50-question run over eight public Nigerian health-regulation documents. The retrieval layer is the weak one, and the pack says so: recall@5 is 0.379. The safety layer held — 25 of 25 unanswerable questions refused, zero visible hallucinations.
Every cell carries its n and its source file. That is the standard the pack is held to, including when the number is unflattering.
Download the sample pack↓This is a one-person practice. That is the point: the person who measures your system is the person who signs the report, and there is nobody to pass you to.
I build and measure retrieval systems, and I publish the measurements even when they are unflattering.
The sample pack on this page is my own work, not a mock-up. It is a 50-question evaluation of a health-regulation assistant over eight public Nigerian documents: 1,218 passages, a hand-built gold set, a counted failure register, and a measured recall@5 of 0.379 — the weak number, left in on purpose.
Every figure in it regenerates from a public repository in about twenty minutes, and a stranger has already done exactly that from a clean clone. You can check it before you ever speak to me.
I ran the product on myself before I offered it to anyone. That is the only credential I can give you that costs you ten minutes and requires no trust in me at all.
What I am not: a certification body, a law firm, or an agency. I do not promise that anything will pass. If your procurement needs ISO 27001 before a paid pilot, I am the wrong supplier, and I will say so on the call rather than after the invoice.
The short version, before you ask: by default your production data never leaves your building. When you do send documents, here is exactly what happens to them.
The pilot runs on documents you choose to send, or on a mirrored corpus of the same shape. No access to your systems, repositories, databases or cloud accounts is requested.
Documents arrive through one access-controlled, encrypted folder — never as an email attachment. Transfer is TLS. At rest: full-disk encryption on one workstation, plus provider-side encryption in the transfer folder.
Corpus, derived data and working files are deleted 90 days after you receive the pack, confirmed in writing on request. Earlier deletion on request, within five business days.
If anything goes wrong you hear it in writing within 24 hours, with a full written account within 72. Notification is never delayed to finish an investigation.
The data processing agreement is standard, and it already binds confidentiality. A standalone mutual NDA is signed on request before anything is shared. Both are yours to redline.
There is no ISO 27001 certificate and no SOC 2 report behind this page, and I will not pretend otherwise. What there is, is a written note you can audit.
No discovery project, no retainer to start, no proposal that takes a week to arrive. If the first three days are not useful, you pay nothing.
What the feature does, what your buyers are asking, and whether a mirrored corpus is possible. If it is not a fit, I say so on the call.
Scope, corpus, question count, dates, price and the exclusions — sent within two hours of the call. Signed, then work starts.
Scorecard, dossier, harness, readout. You keep the harness whether or not you ever buy anything else from me.
Up to 300 documents, 60-question gold set, full metrics, failure taxonomy, a two-page scorecard, and a 30-minute readout. Four business days.
If you use the kill switch, every dollar comes back. The deposit is refunded in full within five business days, in the currency and to the account it arrived from, confirmed in writing. No negotiation and no restocking fee. That is what “you pay nothing” means.
I never ask you to pay a third party, never accept an overpayment and refund the difference, and never change bank details by email. If you receive any of those, it is not from me.