Why we publish what we couldn't verify.

Every audit we run against a real store hits a limit somewhere. A page that returned an error when we tried to check it. A claim on a policy page we had no independent way to confirm. A dimension we simply didn't test because it was out of scope for that engagement. The easy thing to do with that limit is to leave it out of the report and let a clean-looking summary imply completeness it doesn't have. We do the opposite: we write down what we didn't check, next to what we did.

This is not a legal hedge bolted onto the end of a document. It is the difference between a report you can trust and one you can't, and it shapes how we build every deliverable, not just how we phrase the disclaimer.

A score without its boundaries is a claim you can't check

If we hand you a finding — "your returns policy is missing," "your shipping page contradicts your FAQ" — and don't tell you what we looked at to reach it, you have no way to know whether that finding is solid or whether we just didn't happen to find the counter-example. Naming the boundary of what was checked is what turns a bare assertion into something you can verify yourself: here is the page we looked at, here is what we found, here is what we did not attempt to look at. You can go check any of it.

The honest version of a finding is longer than the flattering version. "We could not confirm your payment-processor claims because that check requires a live transaction we're not authorized to run" is a less impressive sentence than silently leaving payment methods out of the report and letting the overall picture look cleaner than it is. We publish the first kind of sentence anyway.

Reporting an honest negative is not a failure to deliver

When a check comes back negative — a page is missing, a claim doesn't hold up, a dimension scores poorly — that is a real, useful finding, not a shortfall in our work. The alternative, softening or omitting a negative result because it's uncomfortable to deliver, is what turns a report into marketing copy. A negative finding you can act on is worth more than a positive summary you can't trust. If something scored badly, the report says so, in the same voice it uses for what scored well.

The same principle is why we label demos as demos elsewhere on this site, rather than letting an illustrative example read like a client result. It is the same instinct applied at a different layer: say plainly what category a piece of evidence belongs to, rather than letting the reader assume the more impressive one.

Why this is trust infrastructure, not a courtesy

A vendor who tells you everything is fine is either right, lucky, or not looking closely enough to find the problems — and from the outside, those three look identical. A vendor who tells you specifically what they checked, what they found wrong, and what they didn't get to, gives you something you can actually reason about: a bounded claim instead of a blanket assurance. That is what lets you build on the report instead of just trusting it on faith. It's also what lets you catch us if we're wrong about something — which is the whole point of writing down what was checked in the first place.

Think about the alternative from the buyer's side. A report that only lists what passed reads well, but it leaves you with no way to tell whether "passed" means "checked and confirmed" or "not checked, so nothing came up." Those are completely different situations, and only one of them means your store is actually in good shape. A boundary line — here is what we tested, here is what we didn't — is what separates the two. Without it, silence about a dimension looks exactly like a clean result on that dimension, and it isn't.

What this looks like in a real deliverable

In practice it means a report has two kinds of statements sitting next to each other, in the same font, with the same weight. "Your product pages include structured data and it validates" is one kind. "We could not test how your checkout handles an international shipping address because that requires a live order we're not placing on your account" is the other. Both are useful. Only the first one is flattering. We don't let the second kind quietly disappear because it complicates the summary.

This also means we don't extrapolate past what we actually checked. If we tested five product pages out of a catalog of five hundred, the report says five, not "your catalog." If a check requires access we don't have — a backend system, a live payment run, a locked-down admin page — the report says so instead of guessing at what the answer probably is and presenting the guess as a finding. A guess dressed up as a finding is worse than no finding at all, because it looks like evidence when it isn't.

This approach is described more fully, with the reasoning behind each part of it, on our method page. If you'd rather see it applied than described, the fastest way is to run our spot check against your own store and read the report the same way: findings next to their boundaries, nothing smoothed over.