I have a screen that tells me how my content is doing. I built it because I was tired of having opinions about my own marketing instead of facts.
Last night I found out that most of what it was showing me was inferred, not measured. Not wrong exactly. Guessed.
The number that would not move
One of the things the screen does is flag outliers: posts that did far better than my usual, so I can go and look at why and do more of that. It is the single most useful thing on there, because it is the only part that tells me something I did not already know.
For weeks it found one outlier. One, across every account I publish to.
I assumed that was the honest answer. My posts were consistent, nothing was breaking out, fine. I even took a small amount of comfort in it, which should have been the tell. A metric that never moves and never makes you uncomfortable is not measuring anything.
What it was actually doing
The rule is simple: flag a post that did five times the median for its own page. To run that rule you need view counts for every post.
The screen was getting those by reading the public version of each page, the same way you or I would if we opened it in a browser. That works on some platforms. On others it does not work at all, because some platforms never show view counts publicly to anyone.
So for those accounts the rule had nothing to divide. No views meant no median, no median meant no comparison, and no comparison meant no outlier. Ever.
The dashboard was not lying to me. It was answering a question it could not actually see, and the answer it gave for "I have no data here" was indistinguishable from the answer for "nothing notable happened here."
That gap is the whole thing. Absence and zero look identical on a screen and mean opposite things.
Asking the platform instead of reading the page
The fix was not clever. Instead of reading the public page, ask the platform directly through its own interface, the one that exists precisely so that the owner of an account can get real numbers about it.
I already had the permissions sitting in another one of my own systems from months ago. I had just never wired them into this one, because scraping the page had appeared to work, and appearing to work is how a thing survives without being checked.
I connected the official sources for the accounts where one exists. Same accounts, same posts, same rule. Nothing about the logic changed.
Outliers went from one to twelve, immediately.
Twelve things had happened that were worth looking at, and for weeks the screen had told me nothing was happening. Every one of those was a post I could have learned from at the time, and did not.
The two things I found while I was in there
The first was that I had been requesting a couple of numbers that no longer exist. The platform changed what it publishes at some point, and my request for those had been quietly failing rather than loudly breaking. I had been looking at a screen with fields on it that were never going to be filled, and I had stopped noticing them, the way you stop noticing a light that has always been off.
The second is more awkward, and I have decided to leave it as it is.
There is one account where I still cannot get follower numbers officially. The permission for that specific figure was never granted to me, and I cannot grant it to myself retroactively. So that number is not available.
The tempting thing is to estimate it. Read it off the page, put it on the screen, and let it sit next to the real numbers looking exactly like them.
I am not doing that. It stays blank and labelled as unavailable. A dashboard with one honest hole in it is worth more than a dashboard where every box is full and I cannot remember which ones I made up.
Why this is the same mistake in a different coat
I keep finding versions of this. A process that runs and reports success without doing the work. A file that reads fine and has silently reverted. A screen that shows a number it inferred rather than one it measured.
They are all the same failure: something that produces output confident enough that nobody checks the input.
And I am the one who built all of them, which means the pattern is not in my tools. It is in me. I build the thing, I watch it produce something plausible, and I promote it from "seems to work" to "works" without anything in between. The check is the step I skip, every time, because the output looked right.
What I have started doing about it is small. When something reports a number, I ask where that number physically came from, once, out loud. Not whether it looks right. Where it came from. Most of the time the answer is fine. The times it is not have all looked exactly like this one: the system was confident, the number was plausible, and nobody had ever asked.
What I would tell someone with a dashboard
Go and find the emptiest thing on it. The metric that always reads zero, or one, or never changes.
Do not assume that is the answer. Ask whether that part of your screen can actually see anything at all, or whether it is reporting the absence of data as if it were a finding.
Then check where each number comes from. Not what it says. Where it comes from. Something you read off a page is a guess dressed as a fact, and it will sit next to your real numbers looking just as certain as they do.
I spent weeks being reassured by a one that should have been a twelve. The reassurance cost more than the missing data did.