The Gate We Pass First: Cheapest to Check

The gates are supposed to run in order of cost. You do the cheap ones first and let them kill as many candidates as possible before you spend time on the expensive ones. That is the logic, and it is sound logic, and we did not follow it for the first two years we ran a formal gate sequence. We ran the gates in the order we had written them down, which was roughly the order they had occurred to us, which had nothing to do with cost or efficiency and everything to do with which problem had been most recently embarrassing.

The cheapest gate we have is availability. Not fitness, not form, not whether the player is likely to feature heavily — just whether the player has been confirmed as available at all. A player who is ruled out, suspended, or otherwise not in the event cannot accumulate the statistics we are trying to evaluate. This is not subtle. It does not require a model. It requires reading a status update, which is public, and discarding the candidate if the answer is no. The gate costs roughly nothing to run, and it kills a meaningful fraction of candidates on any given day.

We still run it second sometimes, out of habit, and Renata has pointed this out to me more than once. The habit is the interesting part. There is something in the way we built the sequence that makes us want to establish a candidate's quality before we check whether they are going to show up. As if it would be wasteful to discard a strong candidate on a technicality. It is not a technicality. A player who is not playing is not a candidate. But the instinct to preserve promising candidates longer than we should — that instinct has survived every process revision we have made, and I have stopped pretending it is fully gone.

Why the Cheapest Gate Gets Skipped in Practice

The problem is not that we forgot availability matters. The problem is that availability information is noisy and sometimes arrives late, and we developed a quiet habit of treating uncertain availability as probable availability. A player listed as questionable is not confirmed out. A player whose status has not been updated since the previous day is not confirmed in. The gate, as written, says confirmed availability required. The gate, as practiced, had drifted toward something more like no confirmed absence required, which is a different thing entirely.

The drift happened gradually. There was a stretch in the hockey corpus where late status updates were common — teams would not confirm lineups until close to the event, and we were running our sequence earlier than that. Rather than hold the candidate and re-check, we had started passing them provisionally. The provisional pass became the default pass. By the time we noticed, we had been running a weakened version of the gate for most of a season without realizing it.

This is exactly the kind of failure that inherited gate logic tends to produce — not a dramatic collapse, but a slow softening where the gate still exists in the documentation and no longer exists in the behavior. We had not inherited this one from anyone else, which made it more embarrassing: we had softened our own gate, without a decision, without a discussion, and without noticing until the grading came back wrong in ways that traced directly to players who had not been confirmed available.

How We Tried to Enforce the First Gate Properly

The fix we attempted was a hard timestamp requirement. A candidate could not pass the availability gate unless there was a confirmed status update logged within a defined window before the event — we settled on a cutoff that varied by sport, because the information ecology is genuinely different across basketball, baseball, soccer and hockey. Basketball injury reports run on a schedule. Baseball lineup cards arrive closer to game time. Soccer is, as we have written about before when discussing the different failure modes across sports, its own particular problem.

We added a logging requirement: every candidate that passed the availability gate had to have a source and a timestamp attached to the pass. Not elaborate — a one-line record. The point was to make the gate visible in the output, so that if a rating later graded badly we could trace whether the availability check had been current or stale.

"The logging didn't change the gate. It changed whether we could lie to ourselves about whether we'd run it." — Renata

She was right. The gate itself was unchanged. What changed was the audit trail. Before the logging requirement, a candidate that had passed provisionally looked identical in the record to one that had passed on confirmed information. Afterward, the stale passes were visible, and visible problems get fixed in a way that invisible ones do not. Within a few weeks of adding the logging, the provisional passes had nearly disappeared — not because we had made them harder to do, but because we could now see exactly how often we were doing them.

What the Stale Passes Actually Cost the Corpus

When we went back through the grading from the season where the drift had been worst, the pattern was clear enough to be uncomfortable. Candidates who had passed the availability gate on stale or provisional information graded worse, on average, than candidates who had passed on confirmed information — not because they were weaker players, but because a non-trivial number of them had not actually played, or had played reduced roles after being listed as questionable and then managed carefully by their teams.

The specific failure mode was late scratches. A player confirmed available two days before an event who then does not feature is not a gate failure in the traditional sense — they passed the gate when the gate ran. But a player passed provisionally on no information at all, who then turns out to have been rested, is a gate failure, and we had been counting those as normal misses rather than process failures. That accounting was wrong. A miss that traces to a gate we did not run properly is not the same as a miss that traces to genuine unpredictability, and conflating them was inflating our confidence in the model's underlying accuracy.

I want to be precise about this: the effect was not enormous. It did not invalidate the corpus or collapse the calibration numbers. What it did was introduce a quiet bias in the direction of overconfidence — the kind that is easy to miss until you are specifically looking for it. We had been presenting our calibration results as if every candidate had cleared every gate properly, and that was not true for the period in question. We corrected the historical record. It was not a pleasant afternoon.

What the First Gate Looks Like Now, and What We Left Alone

The gate now runs first, without exception. Confirmed availability, within the sport-specific window, with a logged source. A candidate without a current confirmed status does not proceed — it gets held and re-checked if time allows, or dropped if it does not. The provisional pass is no longer a category that exists in the system. There is confirmed, there is not confirmed, and there is not enough information, and the last one fails the gate the same as a confirmed absence does.

What we did not change is the definition of availability itself. We had a discussion about whether to expand the gate to include expected role — not just whether the player is available, but whether they are expected to feature at the level the corpus assumes. We decided against it. Expected role is a judgment call. Availability is a fact. Mixing them into the same gate would make the gate harder to audit, and the audit trail is now the thing we care about most. Role assessment happens at a later gate, with different logic and different tolerance for uncertainty.

The ordering principle we came back to is the one we should have applied from the start: run the gates that produce binary answers before the gates that require estimation. Availability is binary. Role is not. Fitness is not. Whether there is enough corpus to say anything meaningful is not. The binary gates are cheap and they are fast and they should always run first, because every candidate they kill is a candidate that does not consume time at the expensive gates downstream. This is not a sophisticated insight. We simply had not followed it, and following it turned out to matter.

There is a version of this problem that recurs in a different form — not softened gates, but duplicated gates that create false confidence in coverage that was never as complete as the documentation suggested. The underlying failure is similar: a gate that exists on paper and behaves differently in practice, undetected until the grading makes it visible.

The cheapest gate is the one most likely to be skipped, because skipping it feels costless in the moment and the cost only shows up later, diffused across a season's worth of grading. I am not sure we have fully solved that. The logging requirement helped. Whether it helped enough is a question the next calibration review will answer, and I have learned not to predict those in advance.

Note: PlayerGem is a fictional analytics shop and these accounts are invented. Nothing here is a pick, a recommendation, or betting or investment advice, and the players, teams and competitions described do not exist.