The Gate Soccer Passed That Hockey Retired

The participation gate is supposed to be cheap. You check whether the player is expected to feature in the event — not just on the roster, not just healthy, but genuinely expected to play meaningful minutes — and if the answer is unclear, you stop there and move on. It is the first gate we pass because it costs almost nothing to check, and because everything downstream is wasted if the player sits. We have written about that principle before and we still believe it.

What we did not notice, for roughly two seasons, was that "expected to feature" meant something structurally different in soccer than it did in hockey. In soccer, lineup news travels through a recognizable press-conference cycle, and a manager who names a player in the starting eleven three days before a match is making a public commitment that is rarely reversed. In hockey, a coach can scratch a player twenty minutes before puck drop with no prior signal at all. The gate was the same gate. The conditions it was checking were not the same conditions.

We ran the same logic across both sports because the gate had worked in soccer and we assumed the mechanism transferred. It did not, and the miss cost us calibration we are still accounting for.

How hockey quietly made the participation gate irrelevant

In soccer, a starting lineup is public information by roughly seventy-five minutes before kickoff in most competitions we track. That window is narrow, but it is reliable. A player named in that lineup has cleared the participation gate with high confidence. The edge cases — late injuries, substitution-before-kickoff events — exist, but they are rare enough that the gate holds as a meaningful filter.

Hockey does not have that structure. A player can be listed on the projected lineup the morning of a game, appear in warmups, and still be scratched before the first period. The decision is the coach's, it is made close to game time, and it is sometimes not communicated publicly until after the fact. We knew this in a general sense. What we failed to do was treat it as a structural property of the sport rather than an edge case.

The gate we were running asked: is this player expected to feature? In soccer, that question had a checkable answer at T-minus-75. In hockey, the same question at the same interval had a much softer answer — something closer to probably, unless something happens. We were treating both answers as equivalent, which meant we were grading hockey ratings against a participation assumption that was less reliable than the one supporting soccer ratings.

"You built a gate that checks a fact in one sport and a rumor in the other. Those aren't the same gate." — Priya

Priya said that during a calibration review in what I think was the second season we ran both sports simultaneously. She was right, and I did not immediately agree with her, which is its own kind of data point.

What we tried when we finally acknowledged the gap

The first adjustment was a timing change. We moved the hockey participation check as close to game time as our process would allow — essentially shifting it from a pre-processing step to a last-minute confirmation. This helped with the scratch problem. It did not help with the cases where a player was listed, confirmed, and then given reduced minutes for reasons nobody announced. A player who skates twelve minutes instead of their corpus average of nineteen has technically participated. They have not participated in any way the corpus was built to evaluate.

The second adjustment was a minutes floor. We set a threshold — a number of minutes below which we would treat a hockey rating as ungradeable rather than wrong — and we applied it retroactively to the two seasons of data where we had not been using it. This created a different problem. Several ratings we had been quietly proud of turned out to have passed only because the player had cleared the minutes floor by a narrow margin. Strip those out and the hit rate in hockey looked considerably worse than we had been reporting.

We published the corrected numbers. This is the part of the process I find genuinely uncomfortable and also genuinely necessary. The corpus looking healthy from the outside is not the same as the corpus being reliable, and a calibration exercise that only runs on the clean data is not a calibration exercise.

The third adjustment was the one that actually held: we split the gate into two versions. The soccer participation gate remained structured around confirmed lineup news within a defined pre-match window. The hockey participation gate added a post-game minutes check before a rating was considered gradeable at all. Same name, different logic, documented separately.

Two seasons of hockey calibration we had been reading wrong

The cost was not dramatic. There was no single catastrophic week where everything broke. The cost was quieter than that: we had been reporting hockey calibration figures that looked roughly comparable to our soccer figures, and when we corrected for the participation assumption gap, hockey came out noticeably worse. Not catastrophically worse. Noticeably.

The gap was about eight percentage points on stated-versus-observed confidence at the high end of our scale. When we said we were highly confident in a hockey rating, we were right at a rate that did not match that confidence. In soccer, the same confidence level tracked much closer to what actually happened. We had assumed the gap was noise. It was not noise. It was a systematic error in the gate.

The part we got wrong — and I want to be specific about this because vague self-criticism is not the same as honest self-criticism — was that we saw the hockey calibration numbers drift early and attributed them to sample size. Hockey has fewer games per season than baseball, and fewer players in scope than soccer, so the corpus is thinner. When something looks off in a thin corpus, "wait for more data" is a reasonable first response. We waited for more data for two seasons. The data kept saying the same thing and we kept deciding it was thin.

This is the failure mode that concerns me most in the shop's work. It is not the dramatic wrong call. It is the slow accumulation of evidence that something is off, combined with a plausible methodological reason to keep deferring. Passing the same error twice because the second pass looked different from the outside is a specific kind of mistake, and we made the hockey version of it for longer than I am comfortable admitting.

What the split gate changed and what it left unresolved

The split gate — soccer version and hockey version, same name, different logic, documented separately — has held up for the seasons since we implemented it. Hockey calibration improved. Not to soccer levels, because hockey still has a thinner corpus and a less predictable minutes distribution, but the stated-versus-observed gap closed to something we can defend.

What the split gate did not resolve was the underlying question of how many other gates in the shop's process are doing different work in different sports without our having noticed. We added a gate review step to the start of each new season, specifically asking: is this gate checking a fact or a rumor, and does the answer differ by sport? We have found two smaller discrepancies since then, neither as significant as the hockey participation problem. Both were in the grading window logic rather than the gates themselves, and both are documented in the calibration record.

The soccer participation gate also taught us something about the value of the pre-match commitment signal that we had been treating as incidental. When a manager names a lineup publicly and that commitment is rarely reversed, the gate is doing real work. It is not just filtering — it is capturing a piece of information about how seriously the team treats the fixture. Soccer's relationship with lineup certainty is different from most other sports we track, and the gate benefits from that in ways we had not made explicit.

We kept the minutes floor in hockey. We kept the retroactive correction to the historical data. We kept the published acknowledgment that two seasons of hockey calibration had been reported against a flawed assumption. The shop's entire premise is that it publishes its own error rate. A miss that stays in the archive is more useful than one that gets quietly retired.

The question I have not answered to my own satisfaction is whether the right response to a gate that works differently across sports is to split it, or to accept that the gate is measuring something more fundamental — something about how much certainty a sport allows before it begins — and let that uncertainty propagate into the confidence numbers rather than engineering it away. Priya thinks the split is cleaner. I am not sure cleaner and more honest are the same thing here.

Note: PlayerGem is a fictional analytics shop and these accounts are invented. Nothing here is a pick, a recommendation, or betting or investment advice, and the players, teams and competitions described do not exist.