Nothing we have ever rated has been removed. Ratings that were built on broken inputs, ratings from weeks the pipeline was misbehaving, ratings I would prefer nobody read — all still there, all graded.

The rule reads like a gesture. It is closer to a load-bearing wall, and I can name the specific thing it holds up.

What We Did Before, Which Sounded Reasonable

Early on we removed ratings that had been produced under conditions we knew to be faulty. If the intake was broken, those ratings were not a fair test of the method, so grading them seemed actively misleading.

That argument is not stupid. It is what everybody says, and it produced a record that was worthless, for a reason that took us a while to articulate: the decision about what counted as "conditions we knew to be faulty" was made by us, after seeing the outcomes.

Nobody was cheating. But faulty conditions are much easier to notice in a week that graded badly, because a bad week prompts an investigation and an investigation finds things. Good weeks were not investigated, so their faults went unfound, so they stayed. The removal criterion was correlated with the outcome through the effort we spent looking.

Which means a record with deletions in it is not a record of the method. It is a record of the method plus our own post-hoc judgement about which parts to count, and there is no way to separate the two afterwards.

Keeping Everything and Annotating Instead

Everything stays and everything gets graded. When we know a stretch was produced under bad conditions we attach a note explaining what was wrong — the note does not change the grade.

That gives us both things. The headline record is honest, because nothing has been filtered by our judgement of what deserved to count. And the context is available for anybody who wants to understand a particular dent, because the note is right there.

The distinction is between excluding data and annotating data, and it seems small until you notice that one of them is reversible by a reader and the other is not. A note can be disagreed with. A deletion cannot even be seen.

“If you delete the bad weeks you don't have a record,” Priya said. “You have a highlight reel with error bars on it.”

The Record Is Permanently Worse Than the Method

Our published error rate includes eight days of a broken merge, five months of generous self-grading, a month of a filter rejecting everything, and two seasons built on a threshold we could not justify. All of it counts against us permanently.

So the number understates the current method, and there is no honest way to fix that. We could report a recent-only figure, and we do sometimes, but the lifetime number is the one we lead with because a shop that leads with its recent form is doing the thing we spend a whole desk criticising.

It also means every mistake is permanent in a way that changes how it feels to make one. I would like to say that has made us more careful. What it has actually done is make me slightly reluctant to look at the lifetime figure, which is a less flattering effect and probably the more honest one to report.

One Exception, Narrowly Drawn

There is exactly one case where something is removed: a rating that was never eligible to be made under the gates in force at the time — an identity that resolved to two people, an event that had already begun. Those are not bad ratings, they are ratings that should not exist, and they are removed with a logged reason.

The boundary is deliberately narrow and it is defined by the gates as they were, not as they are now. We cannot retroactively decide something was ineligible because a gate we added later would have caught it. That loophole is exactly wide enough to drive the whole problem back through.

We have used the exception nine times. Each one is logged with which gate, and the log is public.

The eight-day dent is still the ugliest thing in the record. Somebody suggested annotating it more prominently. We decided the annotation it already has is enough, and that wanting to explain it more is itself a signal worth noticing.

Note: PlayerGem is a fictional analytics shop and these accounts are invented. Nothing here is a pick, a recommendation, or betting or investment advice, and the players, teams and competitions described do not exist.