Capabilities

LLM evaluation and quality gates.

Judge design and calibration, non-circular retrieval metrics, accuracy benchmarking, automated gates that re-run on every change. The measurement harness is often the real deliverable.

Quality as a measurement, not a claim. A faithfulness judge is only trustworthy after it has been validated against an independent annotator, and a metric that cannot fail teaches nothing.

Claims backed by links

The evidence.

ProofLLM evaluation and quality gates

Faithfulness judge validated at Cohen’s k=0.881; 12 automated gates; 0.984 faithfulness; a retrieval metric redesigned after proving the standard one structurally could not pass

Read the deep dive
ProofLLM evaluation and quality gates

OCR and LLM extraction with the benchmarking harness as the actual deliverable

Read the deep dive
ProofLLM evaluation and quality gates

Adversarial testing and safety review of a vendor-built public assistant; launch gated on closed issues

Read the deep dive

The first step

Twenty minutes. You describe where AI is stuck.

You leave the call knowing whether I can help, roughly what it would take, and what it would cost to find out for sure. If I am not the right person, I will say so on the call.

The 20 minutes, in order

  1. You talk first. Where AI is stuck, what has been tried, what it costs today.
  2. I answer plainly. Whether I can help, and what I would look at first.
  3. You leave with a next step. An audit scope, a pointer elsewhere, or a clean no.

Before you book

  1. Your stack is not too messy to start. Messy is the normal starting condition.
  2. Training that does not survive the week is the normal outcome. These sessions build on your backlog and ship something real.
  3. You do not need budget approved to take the call. You need it approved to start step two.
Book the 20 minutes Not ready. Send me the newsletter