What one week of pub listing results can tell you
We changed the menu links and descriptions across a group of pub profiles. A week later the results went in every direction. Here is why a single pub's before-and-after story needs a comparison.
We changed the menu links and descriptions on Google Business Profiles across a group of pubs in one estate. A week later we compared calls, direction requests and website clicks with the previous week, pub by pub.
Some pubs rose sharply. Others fell. Some barely moved. Directions and website clicks scattered the same way, up sharply at some pubs and down sharply at others, with no pattern tying the swing to anything we had actually changed.
We also ran the same week against the pubs we left alone, as a check. Their figures moved too, in their own direction, for reasons that have nothing to do with a menu link.
Small numbers lie confidently
Here is the part that matters more than the headline. Most of the pubs throwing the biggest swings are the smaller ones. A pub taking three calls a week that gets one call the next week has “fallen” 67%. A pub that goes from one call to two has “risen” 100%. Neither number describes anything real happening in that pub. It describes what a small number does when you divide it by another small number.
That is not a reason to stop measuring. It is a reason to stop trusting any single pub’s story after one week, however dramatic the percentage looks on the page.
Believe the test, not the story
We built the pipeline behind this precisely so we would stop eyeballing a dashboard and start proving a count: pull the live write to the listing, pull the intent data either side of it, and hold it against pubs we did not touch in the same window. Most marketing reporting in this trade skips that last part. A single site’s calls go up after a change and the change gets the credit. Nobody asks what happened to the sites nobody touched that same week.
We are keeping the same group of pubs running for a few more weeks against the same untouched group before we say anything at all about whether the menu link and description change did what we hoped. At this stage, we do not know. That is not a comfortable line to publish, but it is the true one, and it is the whole point of building the measurement rather than reading the first number that flatters you.
The value of doing it properly is that we can now afford to be patient. We are not chasing a headline for this week’s client update. We can wait for a whole group to tell us something a group of one never could.
What leaders should do next
- Ask anyone reporting a single-site marketing win what happened to a matched group of sites you didn’t touch that same week. No comparison, no result, just a story.
- Before you credit or blame a low-volume site for a swing, check the base number first. A jump from one call to two is not a trend, it is arithmetic.
- Decide now how many sites and how many weeks you need before you’ll believe a result, and write it down before the first flattering number turns up, not after.
- If your current reporting cannot show you the sites you didn’t touch in the same window, fix that gap before you fix anything else.