# Probability Calibration for Bettors: Are Your 60% Chances Really 60%?
The direct answer
A forecaster is well calibrated when events given a particular probability occur at roughly that rate over time. If you label many selections as 60 per cent chances, about 60 per cent should happen. Calibration checks the honesty of the probability scale, not whether one prediction won and not whether the bets made a profit.
This matters because betting decisions depend on comparing probability with price. A person who routinely calls 50 per cent events 65 per cent chances will appear to find value everywhere. The problem is not the odds calculator. It is overconfidence in the input.
The process is simple in principle: record forecasts before events, group similar probabilities, compare forecasts with outcomes and score every prediction. The difficult part is collecting enough clean, genuinely pre-event observations.
Calibration is not strike rate
A 70 per cent forecaster should win more often than a 40 per cent forecaster, but raw strike rate alone says little. The claimed probabilities matter.
Forecaster A records ten selections at 90 per cent and eight win. Forecaster B records ten at 60 per cent and six win. A has the higher strike rate, but B is perfectly aligned with the stated forecast in that tiny sample. A may be overconfident, although ten observations are far too few for a firm conclusion.
Calibration also differs from discrimination. A model can be calibrated overall while doing a poor job of separating strong and weak chances. Assigning every event the long-run base rate may look calibrated but offer little selection value.
Build a proper forecast record
Record the following before the relevant price moves or the event starts:
| Field | Why it matters |
|---|---|
| Event and market | Prevents different questions being mixed. |
| Selection | Identifies the forecasted outcome. |
| Probability | Preserves the actual pre-event belief. |
| Available odds | Allows later value and closing-price analysis. |
| Time and source | Shows what information was available. |
| Model version | Separates forecasts made by different methods. |
| Final outcome | Supplies the zero or one used for scoring. |
Do not rewrite a 55 per cent estimate as 62 per cent after team news or after seeing the result. Create a new time-stamped forecast if the information set changes. BetOwl's guide to keeping an accurate betting record explains the wider discipline.
Use probability buckets carefully
One readable calibration table groups forecasts into ranges such as 50 to 54 per cent, 55 to 59 per cent and 60 to 64 per cent. For each bucket, calculate the average predicted probability and actual outcome rate.
Suppose 100 forecasts average 60 per cent and 57 outcomes occur. The observed rate is 57 per cent, giving a calibration gap of minus three percentage points. That is useful evidence, but it is not proof that every future 60 per cent forecast should become 57 per cent. Sampling variation remains.
| Forecast bucket | Forecasts | Average prediction | Outcomes | Actual rate | Gap |
|---|---|---|---|---|---|
| 50% to 54% | 180 | 52% | 91 | 50.6% | -1.4 points |
| 55% to 59% | 130 | 57% | 69 | 53.1% | -3.9 points |
| 60% to 64% | 100 | 61% | 57 | 57.0% | -4.0 points |
| 65% to 69% | 55 | 67% | 33 | 60.0% | -7.0 points |
This fictional illustration suggests growing overconfidence in the higher buckets. It does not represent BetOwl results or a live model.
Wide buckets hide detail; narrow buckets become noisy. Publish both the count and the rate so readers can see how much evidence sits behind each point.
The Brier score
The Brier score measures the squared difference between a probability forecast and the binary outcome:
`Brier score = average of (forecast probability - outcome) squared`
An outcome is 1 when it happens and 0 when it does not. Lower scores are better. A perfect forecast receives zero.
For one 60 per cent forecast that wins, the squared error is `(0.60 - 1)² = 0.16`. If it loses, the error is `(0.60 - 0)² = 0.36`.
If 100 identical 60 per cent forecasts produce 57 wins and 43 losses, the score is:
`(57 x 0.16 + 43 x 0.36) / 100 = 0.246`
The number becomes meaningful through comparison: against an earlier model, a simple base-rate forecast or another method evaluated on the same events. It should not be used as a universal pass mark across unrelated sports and market types.
Reliability diagrams reveal overconfidence
A reliability diagram plots average predicted probability against actual outcome rate. Perfect calibration sits on the 45-degree line. Points below it indicate that events occurred less often than forecast; points above it indicate underconfidence.
Always show how many forecasts are in each point. Ten forecasts at 80 per cent can land far from the line by chance. A marker sized by sample count or accompanied by a table prevents a visually dramatic but weak conclusion.
The uncertainty lesson is developed in sample size in betting tests. Calibration should be presented with ranges, not as a verdict after a convenient short run.
Calibration does not guarantee profit
A well-calibrated probability estimate can still be unprofitable if the available price is too short. If an outcome is a genuine 60 per cent chance, fair decimal odds are 1.67. Repeatedly backing it at 1.55 would be correctly forecast but badly priced.
Conversely, a noisy-looking bucket can contain profitable bets if prices were sufficiently generous, although confidence in that conclusion may be low. Calibration, price quality, selection, staking and market limits are different parts of the analysis.
Use the expected value calculator only after assessing the probability. The calculator faithfully processes the number entered; it cannot detect confidence unsupported by evidence.
Common ways calibration analysis goes wrong
First, forecasts are recorded only when a bet is placed. That hides the full decision process and can bias the sample. Second, probabilities are rounded to favourite numbers such as 60 or 70 without a model. Third, different sports, odds ranges and market definitions are pooled even when their behaviour differs. Fourth, the model is altered repeatedly while old and new forecasts are reported as one system.
Fifth, people judge calibration from profit. A profitable run can be poorly calibrated and lucky; a calibrated method can lose during an ordinary spell of variance. Keep the scoring questions separate.
A monthly calibration routine
1. Freeze the forecast log for the completed period. 2. Validate outcomes and remove only documented data errors. 3. Calculate the overall Brier score. 4. Build buckets with counts, average probabilities and outcome rates. 5. Split only by categories chosen before inspecting the results. 6. Compare with a simple baseline and the prior model version. 7. Record any proposed change and test it prospectively. 8. Retain the original log and calculation code.
Do not stake more because one bucket looks good. Calibration is a measurement tool, not a reason to increase financial risk.
Frequently asked questions
How many forecasts do I need?
There is no universal number. It depends on the precision required, probability range, event dependence and number of subgroups. Show uncertainty and read the sample-size guide before setting a threshold.
Is 60 per cent calibration good if 57 of 100 win?
It is reasonably close in that sample, but the difference may be ordinary variation. More observations and an uncertainty interval are needed before recalibrating.
Can a model be calibrated and still useless?
Yes. A model that assigns everyone the base rate can be calibrated overall but fail to rank opportunities or beat available prices.
Should I recalibrate using the same data used to build the model?
That risks overfitting. Evaluate on genuinely later or otherwise held-out forecasts, then test any recalibration prospectively.
Does a lower Brier score mean higher betting profit?
Not necessarily. It means better probability accuracy under that scoring rule. Profit also depends on the prices accepted and how selections are chosen.