The Gate We Passed Twice Before We Got It

The gate is supposed to stop you. That is its only job. You build it, you set a condition, and candidates that cannot clear it die there — quietly, before you have spent any real time on them. The theory is clean. What we found, over two separate months and two separate sports, is that a gate can be technically functional and still pass the wrong candidates, and that you can watch this happen once without understanding why, log the miss, and then watch it happen again in almost identical circumstances before the mechanism finally becomes visible.

This is a piece about that gate. We called it the availability gate, which is a bland name for something we apparently did not understand as well as we thought. It asks one question: is the player actually expected to feature in the event being evaluated? Not "are they on the roster," not "are they healthy by last report," but something more specific — are they expected to play a meaningful role in this particular contest. We had been running a version of this gate for over a year before the first miss. We had been running the corrected version for six weeks before the second one.

Both failures cost us ratings that should never have been published. Both were graded, logged, and are sitting in our calibration record where they belong. The reason I am writing this up is that the second failure was worse than the first, because we thought we had already fixed it.

What the availability gate was actually checking — and what it wasn't

When we built the availability gate, we were thinking about the obvious disqualifiers: a player listed as out, a confirmed scratch, a late withdrawal. Those are easy. The gate caught all of them. What it did not catch — what we had not thought carefully enough about — was the category of player who is listed as available, is expected to play, and is nonetheless going to contribute in a way that makes the underlying corpus essentially irrelevant to that night.

The first miss involved a basketball player. He had cleared every gate cleanly. Corpus was deep — over two seasons of consistent role data. The availability check showed him active. The line was consistent with his established profile. We published the rating. He played. He played about eleven minutes in a game that was decided early, was pulled in the third period, and finished with numbers that bore no relationship to anything in his history. Not because he was injured. Because the game context made his normal role meaningless.

We logged it, wrote it up briefly, and adjusted the gate to ask not just whether a player was available but whether the game context supported the role the corpus was describing. This felt like a thorough fix. It was not a thorough fix. It was a fix for the symptom we had just seen, rather than for the underlying condition.

The underlying condition, which I will describe in the next section, took a second miss to name properly. Most candidates should die before you score them — we have written that principle into our process documentation more than once. The problem here was that this candidate had died at none of the gates, and we had not built a gate that could see the particular way he was going to fail.

The patch after miss one, and why it passed miss two anyway

After the basketball miss, we added a game-state flag to the availability check. The logic was: if the corpus describes a player whose role is heavily dependent on competitive game conditions — close games, specific lineup configurations, opponent pressure — then availability alone is insufficient. You need to assess whether the expected game context supports the role the corpus is modeling.

Riku, who maintains most of our gate logic, was skeptical from the start. His note at the time was characteristically short:

"We are trying to predict game state before the game. That is a different problem than availability. I do not think we should fold it into this gate."

I disagreed with him. I thought the two questions were close enough to live together. I was wrong, and the second miss is essentially a proof of that.

The second miss came six weeks later, in hockey. The player had a deep corpus, cleared all gates including the revised game-state check, and received a published rating. The game went to overtime — not an unusual outcome — but the player's role in the final period contracted sharply due to a lineup adjustment that had been telegraphed in pre-game notes we had access to but had not built into our ingestion process. He played. He was available. The game was close. The role the corpus described simply did not materialize, for a reason that was technically knowable before we published.

The gate passed him because our game-state flag was checking for the right category of risk and applying the wrong data source to evaluate it. We were reading historical game-state distributions — how often does this team play close games — rather than the specific pre-game signals available for that night. The gate was functional. It was checking something real. It was checking it badly, and we had not noticed because the first miss had made us confident we understood the problem.

That confidence is the actual miss. The gate logic was secondary.

What two published ratings with the same failure mode cost the calibration record

The direct cost was two ratings graded as misses inside the same six-week window, in two different sports, with the same root cause. That is a pattern. When the calibration review flagged it, we had to account for the fact that the second miss happened after a documented correction — which is a different kind of problem than a first-time failure, and a worse one for the record.

We had stated, internally, that the availability gate had been improved. That stated confidence was now measurably wrong. Writing the threshold down before you see the data is a discipline we apply to scoring, and we apply it inconsistently to gate logic — which is a gap we have not fully closed. If we had written down what the revised gate was supposed to catch, and what it explicitly was not designed to handle, the second miss would have been predictable from the design rather than a surprise from the outcome.

There was also a subtler cost. The two misses were in sports with different corpus depths — the basketball corpus was richer — and the calibration review initially treated them as separate events. It took a third read to see that the mechanism was identical. That delay meant we spent time looking for sport-specific explanations when the explanation was structural and sport-agnostic. We lost about two weeks of useful diagnostic time.

I want to be precise about what we are not saying here. The gate did not fail because the players underperformed. Players underperform constantly, and that is not a gate failure — that is what grading is for. The gate failed because it published ratings in conditions where the corpus was not a valid description of what was about to happen. Those are different failures, and conflating them would have made the fix worse.

What the gate looks like now, and what Riku's original objection turned out to mean

We separated the questions. Availability is one gate. Role viability — whether the specific conditions of this event support the role the corpus describes — is a second gate, and it runs on different data with a different reviewer. Riku's original objection was not that game state was irrelevant; it was that combining two distinct questions into one check creates a gate that can pass a candidate for the right reason on one dimension while failing to check the other dimension at all. That is exactly what happened twice.

The role viability gate now requires a pre-game signal source — not historical distributions, but event-specific notes, lineup confirmations, and reported context — before a candidate clears. If that source is absent or ambiguous, the candidate does not pass. It dies at the gate, which is where it should have died both times.

We also changed how we document gate revisions. After the first miss, we wrote a brief note and updated the logic. We did not write down what the revised gate was explicitly not checking — the negative space of the design. That omission is what allowed the second miss to feel like a surprise. It was not a surprise. It was a predictable consequence of a gate that had been patched rather than rebuilt. The gate that leaked for eight days taught us something similar about patch logic, and we apparently needed to learn it again in a different context.

What we kept, unchanged, is the principle that a gate failure and a grading miss are not the same event and should not be logged the same way. A grading miss means the rating was reasonable and the outcome was different. A gate failure means the rating should not have existed. Both go into the calibration record. They are counted separately. That distinction has held up across every failure mode we have encountered, and it is the one part of this process I am least inclined to revisit.

Riku asked, after we finished the postmortem, whether a gate that requires event-specific data is actually a gate or just another form of judgment with a checkbox attached. I did not have a clean answer. I still do not.

Note: PlayerGem is a fictional analytics shop and these accounts are invented. Nothing here is a pick, a recommendation, or betting or investment advice, and the players, teams and competitions described do not exist.