Is It Actually Working?
Some numbers go up and prove nothing. How to tell real effect from noise: the decision test, the one question that separates lift from luck, and the few numbers worth watching.
The managing partner of a small law firm opens the monthly marketing report. It's a handsome document: impressions up 38 percent, engagement up 22, average position improved, click-through rate above industry benchmark. Four pages of green arrows. The agency is pleased, and the report is designed so that she will be too.
Then she opens the practice management system. New matters this quarter: level with last year. Revenue: level with last year. Somewhere between the green arrows and the bank account, the good news evaporates — and the report has nothing to say about where.
Both documents are honest. Neither is lying. But only one of them is measuring the business, and it isn't the pretty one.
Measure What Matters
Constraint check
Is This Your Constraint?
Money goes out to marketing, and you honestly can't say what comes back? This is probably your constraint.
Five minutes · tells you what to fix first
Cluster tool
The Measure What Matters Self-Assessment
How honestly your numbers tell you what's working, scored with fixes ranked.
Shows what you're deciding blind
The decision test
Everything else in this library builds parts of your customer system. This cluster is different: it's the instrument panel — the thing that tells you whether any of it, or anything you're paying for, is actually moving money. And the whole discipline compresses into two ideas an owner can carry around.
The decision test: a number matters only if it would change a decision. If it can go up or down without you doing anything differently, it's decoration.
The one question: for anything you spend on, ask — if we hadn't done this, would the result have happened anyway? The gap between what happened and what would have happened anyway is the only thing you're ever really buying.
Run last month's report through those two filters and watch it shrink. Impressions, engagement, even traffic: decoration, unless they change what you do next. The numbers that survive are humbler and fewer — enquiries, booked work, where each new client actually came from, what returned versus what you spent — and they fit on an index card, which is exactly where they belong.
Why good numbers lie
Not usually through dishonesty — through incentives and wishful thinking. Reports grade the people who write them, so they're built from the numbers that always go up. Tools count what's easy to count, and what's easy to count is activity, not effect. And every owner quietly wants the campaign they've already paid for to be working, which makes green arrows pleasant company.
The antidote isn't cynicism or a data team. It's the operator's habit of tracing every claim one step closer to money: not "did people see it" but "did anyone enquire because of it"; not "did we rank" but "did the phone ring"; not "did it happen after we spent" but "would it have happened anyway." Practices that hold that line make fewer, better bets — and cancel the rest without guilt.
What's in this cluster
Vanity vs real metrics — the sorting test in depth: which numbers change decisions, and which just feel good.
What to actually measure — the index-card set: the five numbers a busy practice can keep honestly, and how.
Attribution is mostly guesswork — why "this channel drove 14 enquiries" is a guess wearing a report's clothes, and how to hold claims like it.
Would it have happened anyway? — the incrementality idea in plain language: the single most useful question in marketing.
Testing without a data team — the holdout method: a real experiment any practice can run with a calendar and honesty.
Why your numbers don't match your sales — the usual causes of dashboard-versus-bank-account drift, in the order to check them.
Did AI send that customer? — the new blind spot: seeing the clients who arrived from an assistant's answer, imperfectly but usefully.
Your next step
Start the cheapest measurement system that exists: from today, every new enquiry gets asked one question — "how did you hear about us?" — and the answer gets written down, verbatim, every time. It costs nothing, it needs no tools, and within a month it will contradict at least one thing you currently believe about where your customers come from.
Questions practices actually ask
I have Google Analytics. Isn't that enough? Analytics counts visits; your business runs on enquiries, bookings, and money. The gap between those two worlds is exactly where the pretty-report problem lives. Analytics is an input to measurement, not the thing itself.
Do I need dashboards or a data person? No. Five honest numbers reviewed monthly beat forty automated ones nobody acts on. The scarce ingredient is the decision test, not the tooling.
My agency sends detailed reports. Why doubt them? Because reports grade the people who write them. A useful report leads with money numbers and tells you at least one thing that isn't working. If yours never does, that's not a report, it's reassurance.
How do I actually know if my ads work? By asking the one question with teeth: pause them somewhere, or for a while, and watch what changes. That's the holdout method, and a small practice can run it better than most big companies — fewer moving parts, cleaner signal.
