TBF: a decision support system for referee and evaluator assignment

Proposal

Decision

The system proposes, the board commits

9

Hard rules

Never violated, in eligibility or in the model

Rolling

Horizon

A long window is solved, the first week is committed

1,400+

Tests

Back end + front end; independent constraint checker

Problem

Every week the Central Referee Board has to assign three-person crews to hundreds of matches from among hundreds of referees. Done by hand, the work runs into four problems:

• Fairness cannot be measured. Who got how many matches and who travelled how far cannot be tracked across a season; complaints stay subjective.
• Rule breaches slip through. Excuses, bans, city restrictions, the ban on refereeing the same team twice in a row, two matches on one day: nobody can keep all of these in mind at once.
• Match difficulty and crew strength do not match. A weak crew can go to a big match, an unnecessarily strong one to a small match.
• It takes far too long. A week’s assignment takes hours; when the fixture changes it starts over.

The board makes the final decision

The system only produces proposals; the board decides. The flow has three steps. The system solves and produces a proposal. The board reviews it and, if needed, changes assignments one by one; every change is checked instantly against the hard constraints, and any breach is flagged with its reason. Finally the board commits. A committed match is never changed automatically again and enters later solves as history. The reasoning behind every proposal, meaning which rule and which value it rests on, is recorded.

This principle was a precondition for the board accepting the system. A referee board cannot defend an assignment with “the algorithm said so”; it has to be able to point to the rule and the value behind it.

Model

The decision variable is binary: is referee r on duty at match m. The model is built per classification. A, B and C are solved separately and pools do not mix.

Most hard constraints are applied at the eligibility stage. Excuses, bans, league licences, city restrictions, a day already filled by another committed assignment, a committed duty within the rotation window: in these cases the (referee, match) pair is eliminated before a variable is even created. What remains in the model: exactly three referees per match, the crew rating floor, the day gap, in-window rotation, the monthly cap.

The objective function minimises a weighted sum. We deliberately kept the weights far apart, and the priority order is: match-count equality ≫ cross-league balance ≫ different league ≫ rating fit ≫ travel. Match counts and travel are computed cumulatively, so committed history since the start of the season is counted together with the window. We treat fairness at the scale of a season rather than a week.

A match’s required floor rating is the largest of three values: the two teams’ rating sum, a match-specific importance floor (derby, cup, goal average) and a user-defined minimum. The crew’s rating above that floor is penalised as “surplus”, because sending a strong referee to a smaller match than needed wastes them.

Calendar week, solvers and checks

Calendar week. The federation’s fixture round numbers are not aligned across leagues; some leagues have byes and some start on different dates. The system therefore derives a Friday-to-Thursday calendar week and runs every time constraint (window, rotation, rolling horizon) on it. The league week is kept for display only.

Two solvers. The same model is built for both open-source CBC (PuLP) and OR-Tools CP-SAT. In the default mode both run and the better result is picked by a solver-independent canonical objective. A solution cut off at the time limit is reported as “Feasible (time limit)” rather than “Optimal”, so that the user does not put too much trust in the word “Optimal”.

Three separate checks. The first is a feasibility diagnosis before solving: if something is blocked, the solver never runs and the user is told in plain language which match is stuck and why. The second is data quality warnings. Zero-rated referees, missing distances or empty league licences can silently disable rules, and the warnings make that visible. The third is a re-check of the produced solution against the hard constraints by a checker independent of the model. The same checker also runs in the test suite, reading the database directly.

Rolling horizon

A long window is solved, only the first week(s) are committed, and then the window rolls forward. Each run is stored with its window, proposals and record (parameters, solver, canonical score), and several competing runs for the same weeks can be kept side by side. When the fixture changes, only the affected weeks are re-solved.

The second problem: evaluator assignment

We built the assignment of one evaluator (observer) to each match, in addition to the referee crew, as a separate optimization problem. Referee assignment is committed first, and the evaluator solution sits on top of it. The rules differ: rating–league matching, at most one duty a week, locality priority with a distance cost, balanced distribution by season end. An uncovered match does not produce infeasibility; it is reported with a heavy penalty. The fairness rule was not embedded in the model and lives in a separate, replaceable module. We collected the open rules to ask the federation into a question set. Because every decision was built as a parameter or a replaceable module, we could start implementation before the answers came.

System and status

Data lives in PostgreSQL. The business logic is a Python package (cozucu) kept separate from the presentation layer; the interface never talks to the database directly and everything goes through the service layer. The first version ran in the cloud with a Streamlit interface and entered pilot; corrections from an independent audit report were then applied.

The version running today is an authenticated web application (FastAPI + React) under Docker Compose on the federation’s own server. Alongside the asynchronous solve flow, manual editing, run management, commit, report screens, user management and audit trail, it has screens for editing fixture, seasonal rating and person data. The system went live in September 2026; restoring from the nightly backup was rehearsed on the federation’s server. In the 2026–27 season the match programmes the federation publishes are loaded into the system, and the board makes its referee and evaluator assignments there. Next is integration with the federation’s Basketball Management System (BYS).

Live use also brought up needs the test environment never showed. Two examples: how a referee whose rating changes after a solve should be reflected in the existing proposal, and swapping referees between two matches in a single step. Both were noticed in the board’s day-to-day use and went onto the roadmap.

The system is covered by more than 1,400 tests across back end and front end; the independent hard-constraint checker re-checks every solution by reading it back from the database.

How the model was built

Objective	Direction	Tension
Match-count inequality (most – least)	↓	The heaviest term. Fairness is cumulative across the season; new weeks count together with history
Cross-league distribution deviation and clustering in one league	↓	A referee going evenly to the leagues within a classification can conflict with short-term match equality
Rating surplus (the crew's rating above the floor)	↓	Sending a strong referee to a small match is waste; but fairness sometimes requires it
Travel inequality (km)	↓	The lightest term; dropped from the model entirely when distance data is missing

Constraints

• Exactly three referees per match; the crew's total rating cannot fall below the match's required floor
• No assignment on dates with an excuse, ban, Euroleague duty or administrative restriction; such pairs are filtered out before they are created
• The referee must hold the league licence; in classifications B and C, cannot be assigned to a team from their own city
• A minimum day gap between two matches; the same team cannot be refereed twice within the rotation window
• A cap on matches per calendar month (committed history counts toward it)
• Committed matches never enter the model and are treated as fixed; only proposals are re-solved
• Evaluators are assigned only to matches with committed referees, at most once a week, by rating–league match
• An uncovered match does not produce infeasibility; it is reported with a heavy penalty

Related

• Derimod: seven projects in three layers, from data warehouse to Optail
• Gürmen MATS: moving inter-store transfers from Excel to a system that decides at SKU level
• Optail Initial Allocation: first distribution of new product with a master plan, store grouping and capacity

All our work