The feed did not break. That would have been easy. A broken feed throws errors, fills logs with noise, stops ingestion entirely — and you notice within a day. What happened instead was quieter and, in retrospect, more instructive about how a corpus actually fails. The data kept arriving. The pipeline kept running. The ratings kept publishing. The numbers just meant something slightly different than they had the week before, and nothing in our infrastructure was built to notice that distinction.

It started, as best we can reconstruct, when an upstream provider changed how they reported one category of performance data — not the values, but the denominator they were implicitly attached to. The raw figures looked identical. A player who had previously logged a count of twelve under one definition was now logging a count of twelve under a definition that had quietly expanded its scope. The delta was small enough per record that no individual entry triggered a range check. Aggregated across a season's worth of historical backfill, the shift was enough to move a confidence estimate by several points — which is enough to matter when calibration is the whole game.

We did not catch it for eleven weeks. That is the part I keep returning to. Eleven weeks of ratings built on a corpus that was measuring something real, just not quite the thing we thought it was measuring.

The Denominator Nobody Told Us Had Moved

The category in question was a rate-based basketball stat — one of those numbers that looks like a count but is actually a ratio, and whose meaning depends entirely on the definition of the activity being counted. The provider had expanded the qualifying threshold for what counted as an eligible possession. Their documentation was updated; we had simply never built anything to watch the documentation.

The practical effect was a systematic upward drift. Players who had been sitting near the median of our corpus on this measure were now sitting above it, not because they had improved but because the ruler had quietly grown. When we ran those records through the gates, more candidates cleared than should have. When we ran them through shrinkage, the base rate we were shrinking toward was itself contaminated. The error compounded at each stage in a way that a single-stage audit would never catch.

Riya was the one who finally spotted it, and only because she was doing something unglamorous: manually checking a handful of older ratings against their original source records as part of a routine calibration audit. She found a discrepancy on a single player's rolling average — not large, but persistent across every week in the window. That kind of consistency is a signature. Random noise doesn't line up like that.

"It wasn't wrong in a dramatic way," she said. "It was wrong in the way a clock is wrong when someone sets it three minutes fast. Everything still tells time. You just miss the train."

Once we knew what to look for, the scope became clear quickly. The feed had changed shape at a specific date. Everything ingested after that date carried the new definition. Everything before it carried the old one. The corpus now contained two populations pretending to be one.

Rebuilding the Affected Window Without Losing the History

The first instinct was to re-pull everything from the affected date forward and overwrite. We resisted that, partly for principled reasons and partly because our policy on grading rather than deleting extends to source data as well as to ratings themselves. Overwriting would have cleaned the corpus, but it would also have erased the evidence of what we had believed and why — and that evidence is what calibration runs on.

Instead, we versioned. Every affected record was re-ingested under a new definition tag, and the historical records were left intact but flagged. The corpus now explicitly contains two parallel series for this measure: one under the old definition, one under the new. Any rating that drew on the contaminated window was re-graded with a notation in the audit log. The hit-rate figures for that period dropped slightly when we applied the corrected data. We published the revised numbers in our calibration log without annotation beyond the date range and the cause.

The gate logic also needed adjustment. We added a schema-version check to the ingestion layer — a simple lookup that compares the current definition hash against the definition hash on record for that provider and that stat category. If they diverge, ingestion pauses and flags for manual review rather than proceeding silently. This is not a sophisticated solution. It is the kind of thing that should have been there from the beginning, and the reason it wasn't is that we had never been hurt by a silent definitional change before.

The rebuild took about two weeks of actual work spread across six calendar weeks, mostly because the re-ingestion had to be interleaved with the ordinary corpus maintenance that doesn't stop just because something else broke.

Eleven Weeks of Ratings We Can't Fully Trust

Here is the honest accounting: we cannot fully reconstruct what the ratings for that eleven-week window would have said under the correct definition, because the contamination touched not just the stat in question but the cross-sport normalization tables that use it as a calibration anchor. Those tables are rebuilt quarterly. The quarterly rebuild that ran inside the contaminated window used bad inputs, and some of that error propagated into sports that don't even use the affected measure directly.

The magnitude was not large. On the calibration log, the affected period shows a confidence gap of roughly four points — we were stating confidence at a level that our observed accuracy did not support, which is precisely the failure mode that publishing our error rate is supposed to surface. Four points is not catastrophic. It is also not nothing, and the fact that it took eleven weeks to find suggests our monitoring was not as tight as we had believed it to be.

We also got the scope of the problem wrong on first assessment. When Riya brought the initial finding to the group, I estimated the contamination was limited to one sport and one stat category. That estimate was wrong. Marcus ran the full audit two days later and found the normalization table exposure, which I had not considered. I had been too quick to bound the problem because bounding it felt like progress. It wasn't; it was just comfort.

That pattern — confident scoping as a form of premature relief — is one I have noticed in myself before. It is not a corpus problem. It is a judgment problem that the corpus cannot fix, which is worth saying plainly in a piece about what the corpus can and cannot do. A clean run of results has a way of making the infrastructure feel more solid than it is, and we had been in a clean run when the feed changed.

What the Feed Incident Changed About How We Watch the Corpus

The schema-version check stayed. So did a new practice: every stat category that feeds the normalization tables now has a designated owner who is responsible for reviewing provider changelogs on a fixed schedule — not when something breaks, but routinely, as maintenance. This is boring work. It is also the kind of work that the corpus desk exists to do, and we had been skimping on it because nothing had gone wrong yet.

We also kept the versioned parallel series, even though maintaining two definitions for the same measure adds overhead. The reason is that the old series is now genuinely useful — it lets us study what the corpus believed under the old definition, compare it to what the corrected corpus believes, and understand the delta. That delta is itself a form of information about how sensitive our ratings are to definitional choices. The answer, in this case, was: more sensitive than we expected for a measure we had considered relatively stable.

What we did not do is add more automated alerting beyond the schema check. The temptation after an incident like this is to instrument everything, to build tripwires at every possible failure point until the monitoring infrastructure is more complex than the thing it monitors. We have been down that road. Complex monitoring systems fail in complex ways, and a system that pages on every minor anomaly trains the people watching it to ignore pages. The schema check is specific, cheap, and catches the exact class of failure we experienced. That felt like enough, with the acknowledgment that it is probably not enough for the failure mode we haven't thought of yet.

The deeper thing the incident clarified — and this connects to something Marcus said during the post-mortem — is that the corpus is not a database. A database holds records. The corpus holds beliefs about what those records mean, and those beliefs are embedded in the definitions, the normalization choices, and the gates, not in the raw numbers. When any of those change, the corpus has changed, even if the numbers look identical. We had been treating stability of the numbers as a proxy for stability of the corpus. It isn't.

The feed will change shape again. Providers update their methodologies, expand their scope, quietly revise what a number means without revising the number itself. The question we have not fully answered is whether the versioning approach scales — whether, five years from now, we will be maintaining parallel series for a dozen definitions of the same measure, and whether anyone will still remember why each version exists or which one the current gates are built on.

Note: PlayerGem is a fictional analytics shop and these accounts are invented. Nothing here is a pick, a recommendation, or betting or investment advice, and the players, teams and competitions described do not exist.