The Gate We Added Twice and Never Noticed

Sometime in the middle of a busy hockey stretch, we added a gate. The gate was straightforward: disqualify any candidate whose last confirmed appearance was more than ten days prior. Availability signals decay fast in hockey, and we had been burned by stale data often enough that the rule felt obvious. We wrote it into the sequence, tested it on the previous week's corpus, and moved on.

Six weeks later, during a calibration review, Priya flagged that one slice of the hockey corpus was running thin. Not thin in the way a quiet week looks thin — thin in the way a sample that has been filtered twice looks thin. The candidates were being cut harder than the gate logic should have allowed. We pulled the sequence apart and found it immediately: the ten-day recency gate appeared in two places, written slightly differently but doing the same work. One check used calendar days. The other used days since last recorded event. For most candidates they agreed. For a specific subset — players returning from a certain kind of absence — they disagreed in a way that eliminated almost everyone.

We had not added the gate twice on purpose. We had added it once, forgotten we had added it, and added it again. The second version was written by me. I have no memory of writing it.

How a Gate Ends Up in the Sequence Twice

The gates live in a document that has been edited by several people over several years. There is a canonical version, and there are working copies, and at some point the working copies and the canonical version are reconciled — which is a polite word for "someone reads through and tries to notice what changed." That process is imperfect. It has always been imperfect. We knew it was imperfect, which is why we had a rule that new gates required a second pair of eyes before they were committed.

The rule existed. We did not follow it for this one. The hockey stretch was busy, the logic felt self-evident, and I did not ask anyone to check it. That is the whole explanation.

What made the duplicate hard to spot was that the two versions were not identical. One sat in the availability block, where it belonged. The other had been placed in what we call the activity block — a separate check that asks whether a player has been meaningfully involved in recent events, not just whether they were present. The framing was different enough that neither version looked wrong in isolation. You had to read them side by side to see they were asking the same question twice.

This is related to a problem we have written about before: a gate added from a misremembered rationale is almost impossible to audit from the inside, because the person who added it believes it is correct. A gate added twice is the same failure with a different surface. The belief that the gate was new was the problem. The gate itself was fine.

What We Did When We Found the Duplicate

The immediate fix was easy. We removed the second instance, reconciled the calendar-day and event-day logic into a single definition, and re-ran the previous six weeks of hockey ratings against the corrected sequence. Most of the affected candidates came back through cleanly. A small number — the ones who had been eliminated by the disagreement between the two definitions — turned out to be genuinely borderline. Some would have cleared a single gate. Some would not have. We graded the ones we could and flagged the rest as ungradeable for that window.

The harder task was the audit. We went through the entire gate sequence looking for any other case where two checks were doing overlapping work. We found two more, both minor, neither as consequential as the hockey duplicate. One was in the soccer availability block, which had its own history of structural problems that baseball never encountered because the data sources are different. The other was in a general activity check that had been tightened twice in response to two separate incidents and never consolidated.

We also added a step to the reconciliation process: before any gate is committed, the sequence is searched for any existing gate that asks a materially similar question. It takes about four minutes. We should have been doing it for years.

"The problem isn't that we added it twice," Priya said, during the review. "The problem is that we had a process for preventing exactly this and we decided it didn't apply. That's the thing worth writing down."

She was right, and I wrote it down, and then I wrote this piece.

What the Double Gate Actually Filtered Out

Here is the part that is genuinely embarrassing. When we re-ran the six affected weeks and looked at the candidates the duplicate gate had eliminated, a disproportionate number of them were players returning from short absences — exactly the population the original gate was designed to handle carefully, not eliminate wholesale. The ten-day window was meant to catch stale data, not to block every player whose last recorded event was nine days ago and who had since been confirmed available. The second gate, sitting in the activity block, was applying the same cutoff without the same context, and the combination was too aggressive.

We had, in effect, built a filter that was hardest on the candidates we most needed to evaluate correctly. Players with long, stable corpora — the ones where grading every rating rather than deleting it matters most — tend to have predictable activity patterns and would have cleared both gates easily. Players with interrupted histories, which is where the interesting evaluation questions live, were the ones being cut.

The calibration impact was real but not catastrophic. The hockey slice for those six weeks was too small to produce a reliable calibration figure, which is how Priya spotted the problem in the first place. We did not produce a bad calibration number — we produced no number, which is a different failure. A bad number at least tells you something. A missing number tells you only that something went wrong upstream, and you have to go find it.

We also cannot fully reconstruct what we missed. The candidates that were eliminated and never scored do not have outcomes we can attach to them now. They are a gap in the record, and the gap is permanent. That is the actual cost.

What Stayed in the Sequence After We Fixed It

Both the calendar-day and event-day logic are still in the sequence — consolidated into a single gate that uses whichever definition is more conservative for the candidate in question, rather than applying both independently. The gate is stricter in some cases than the original calendar-day version was, and looser in others. We think that is the right trade. We are not certain.

The reconciliation search step stayed. It has caught one potential duplicate since we added it, a minor one in the baseball corpus that would probably not have caused the same problem. We caught it before it was committed, which is how the process is supposed to work.

What we did not do is go back and re-evaluate the ungradeable candidates from those six weeks. The temptation was real — we knew some of them would have cleared the corrected gate, and we had their outcomes available. But reconstructing a rating after you know the outcome is not evaluation, and we have a specific rule against it. The gap stays a gap. This connects to something we examined in an earlier piece about a gate added retroactively and dressed up as foresight — the error there and the temptation here are the same shape, just arrived at from different directions.

The gate sequence now has a version comment on every entry that records when it was added and why. Not a full audit trail — we are a small shop and we do not have the infrastructure for that — but enough that the next person reading it can see whether a gate is recent or inherited, and can ask the question we did not ask six weeks into a hockey stretch: has anyone checked whether this already exists?

I still do not remember writing the second gate. I have looked at the working copy timestamps and the edit is clearly mine, made on a Tuesday evening. Whatever reasoning I had at the time did not survive the week. That might be the more interesting problem — not that we lack a process for catching duplicates, but that a person can do a thing, believe it is new, and have no record of it in their own head three days later. I am not sure what a gate sequence does about that.

Note: PlayerGem is a fictional analytics shop and these accounts are invented. Nothing here is a pick, a recommendation, or betting or investment advice, and the players, teams and competitions described do not exist.