An intent score can be read as a probability only when its scale explicitly predicts a defined outcome by a defined date and the predictions are calibrated for the population you use. A score of 80 can mean relative priority, a percentile, or an 80 percent predicted probability. Ask which one it is before using it to forecast revenue or size an advertising audience.
Start with the buyer decision
Start with a dated score export and a mature outcome table. If a scoring tool reports a probability of an accepted discovery within 30 days, place those predictions into bins and calculate their average prediction and observed acceptance rate. Compare them on the same unit. An account with two contacts produces one account outcome when the score applies to the account. A rep's individual email reply belongs to a different evaluation.
Imagine a field-service software team that labels an opportunity accepted only after the prospect confirms a dispatch problem, a deployment owner, and a next step. A priority score sends the most useful accounts to a small queue. The same ranking can remain valuable even if a number in its top band is nowhere near the observed chance of qualification. Recalibrating probability and changing the ordering are separate operations.
What the current guidance changes
IAB's October 20, 2026 measurement consultation discussed using evidence for specific decisions. A score suitable for a queue is different evidence from a calibrated probability suitable for forecasting. See the IAB cross-channel measurement consultation.
A fictional buyer walkthrough
The fictional high band predicts 45 percent on average and yields 19 qualifying accounts out of 50, or 38 percent. Its observed rate is seven percentage points below prediction. The lower band predicts 20 percent and yields eight of 50, or 16 percent. Its observed rate is four points below prediction.
Those two bins are consistent with a useful ordering and optimistic predictions in this small worksheet. They do not establish a calibration curve. A later validation should include other score bands, uncertainty and the share of eligible accounts with predictions.
Prediction specification
Write the event, time horizon, population, and score scale in the worksheet header. Record the model version and the date it issued each prediction.
For the prediction specification check, plan a sensitivity check that changes one defensible assumption at a time. When reviewing the prediction specification check, record which cases it moves into or out of the primary denominator. Recalculate calibration worksheet using the approved definition, then assess whether unresolved cases could change the interpretation. If they could, report the size of the gap rather than applying a favorable assumption. For this case, the expected next step is clear. Ask the provider whether an 80 is a percentile, an index, or a probability before averaging it as a rate.
A useful checkpoint includes both a success measure and a reason to pause.
Make a small calibration plot interpretable
In a score worksheet, use prediction probability on one axis and observed outcome frequency on the other. Points near the diagonal indicate that forecasts agree with observed rates for those bins under the sampled conditions. Add counts or error bars rather than letting a band of six accounts look as persuasive as a band of several hundred. A calibration slope or average error can be useful with enough data. A tiny pilot often benefits more from transparent tables and an explicit warning that estimates will move with more observations.
Frozen data
Evaluate predictions saved before any results or sales updates arrived. A refreshed score may incorporate information that was unavailable to the rep at the original decision. To close the review, write down what the decision owner is expected to do. Do not overwrite the original values when a revised model rescores the same accounts.
Apply the frozen data check rule before seeing the outcome totals. This keeps a retrospective analysis from silently favoring one group. If the definition genuinely needs correction, describe why, keep a dated revision, and show the impact when the prior definition was feasible. A common outcome definition is necessary, and reviewers also need a reasonably comparable opportunity to observe that outcome. An approver can check that the rule behaved as specified.
Agree on the point at which the frozen data check becomes a decision someone can act on. Earlier evidence can stay visible in the approved working record. The final disposition should say which requirement was satisfied. A later change can then update the decision without turning earlier uncertainty into a historical fact.
Band interpretation
Write down how the band interpretation check affects the eventual claim. In the fictional worksheet, suppose the two bands have average predicted probabilities of 45 percent and 20 percent. Compare these averages with the observed rates and inspect the sizes of both bands.
Apply the rule while the source evidence is still available. Preserve both the average forecast and the observed outcome count so the calculation is reproducible. For the band interpretation check, a study reviewer should be able to explain the decision time, comparison unit, and evidence that would justify any exclusion. Where those conditions differ between groups, report the difference as part of the method instead of presenting the final rate as self-explanatory. Do that before treating the reviewed sample as representative of the full queue.
In the worksheet, attach the expected result of the band interpretation check to a dated record. Ask the reviewer to point to the evidence that supports it. For a case that cannot be resolved, record what is missing and leave the substantive decision open until the right owner can respond.
Coverage and review
Some accounts have no usable score, and some remain unresolved at the extraction date. Treat scoring coverage and label completeness as separate diagnostics.
Specify the contribution of the coverage and review check to the study design before calculating outcomes. When reviewing the coverage and review check, record which cases it moves into or out of the primary denominator. Recalculate calibration worksheet using the approved definition, then assess whether unresolved cases could change the interpretation. If they could, report the size of the gap rather than applying a favorable assumption. Put the next step in the operating notes. Investigate whether missing predictions cluster by company size, territory, or source before generalizing.
Read this checkpoint from the buyer side as well as the operator side.
Before treating the coverage and review check as ready for routine use, agree how to handle evidence that arrives after the initial decision. Identify which current state can change and which original snapshot remains frozen. This preserves the difference between the latest information and the evidence available when the earlier action was approved.
Check the incentive created by each score interpretation
If an intent score is used to allocate scarce seller time, the team can ask how many qualified opportunities appear in the first forty reviewed accounts. If finance uses predictions for expected pipeline, the same output should be tested as a probability on the right population and horizon. These are different uses even when they share a data export. Also distinguish lead conversion under a specific outreach policy from the probability of some abstract state called buying intent. The model's target should identify an observable outcome and the operating conditions under which it was learned.
Management decision
Use ranking evidence to prioritize rep effort. Use adequately validated probabilities with appropriate assumptions if the task is forecasting expected conversion counts. The review record for the management decision check should retain this disposition. If a band lacks outcomes, retain it for investigation and avoid giving it a false precision in a dashboard.
Apply the management decision check rule before seeing the outcome totals. This keeps a retrospective analysis from silently favoring one group. If the definition genuinely needs correction, describe why, keep a dated revision, and show the impact when the prior definition was feasible. Keep decisions that require sensitive detail with authorized reviewers and export only the flags needed for analysis. If they need a spreadsheet maintained by someone else, either include the approved reference or narrow the action.
When comparing higher probability band with lower probability band, verify how the management decision check was handled in both views. Different rules can explain an apparent difference even if the underlying cases behave similarly. Keep the rules visible with the results and show any material exception when the two groups cannot be judged on equivalent terms.
Build the review worksheet
An analyst can build a worksheet with account key, score version, prediction timestamp, stated horizon, predicted probability, band and mature outcome. Calculate each band's mean predicted probability using its member scores. Calculate observed rates from all eligible members with mature outcomes rather than selecting accounts that answered outreach.
Calibration can change when a vendor moves from enterprise accounts to regional businesses with smaller projects. Test under the actual buying cycle and offer. If the score is a rank, keep the band outcome rate as evidence about relative prioritization, without pretending it validates a promised probability.
A worksheet for calibration worksheet
In the fictional worksheet below, the denominator consists of mature eligible accounts in the two score bands. The numerator counts cases that met the frozen acceptance rule by day thirty. Both figures belong in the report. A reader who has only a percentage cannot tell whether a handful of outcomes drove the apparent difference.
Divide 19 by 50 and multiply by 100 for the first rate. For the second rate, use 8 divided by 50. Subtract the unrounded percentages to find that the first is 22.00 percentage points above the second. A relative comparison divides that difference by the second rate, so the first is about 137.5 percent higher on that basis.
These are bands of predictions from one fictional model, so do not interpret their difference as model improvement. The correct comparison for calibration is each band's observed rate versus its mean forecast.
Record the snapshot date beside the calibration worksheet result. For this hypothetical comparison, use the same written review standard for both groups. If a reviewer changes the comparison rule, keep the original calibration worksheet calculation and clearly identify the revised version.
In a live analysis of calibration worksheet, identify the point where a difference would actually change a budget, capacity, or score policy. Show how uncertainty and incomplete labels affect that choice. A sample that appears directionally promising can be worth extending without supporting a permanent operating change.
Decide how to handle the middle bands
Do not build a calibration claim from only the strongest and weakest scores. The middle of the distribution is often where sales or finance makes a threshold decision. If an account is near the cutoff, a modest probability error can affect its expected value. Combine sparse bands using a rule set before viewing the outcome, or keep them separate and report the uncertainty. Compare the model to a simple reference forecast using the same eligible population.
On the fictional calibration worksheet worksheet, one additional qualifying case in the first group would move its rate by 2.00 percentage points if the denominator stayed at 50. The second group changes by 2.00 points for one case on its own fixed denominator. This is a sensitivity illustration, not an uncertainty interval. A real review should show how many cases were unresolved and whether a reasonable interpretation of those cases would change the decision.
If the forecast bands are set from the test scores themselves, write down their boundaries before observing the labels. A daily dashboard can change group membership as new records arrive. Save a stable cut and examine the counts in the actual forecast band your sales or finance team will use. Probability interpretation is particularly fragile when a score developed for one offer is reused across very different deal cycles.
For more on the signals behind this decision, read the Cortex guide to buyer intent data.
Set the decision before opening the result
Decide whether the score supports queue ranking, probability forecasting, both, or neither in this use case. Take one score band and the mature outcomes it was supposed to predict. Ask which decision would change if the observed rate were several points lower. That question makes a probability validation worthwhile and helps separate its purpose from the team's need for a ranked queue.
See how Cortex Marketing AI uses customer feedback to refine intent scores.
Bring the current score definition, available outcome counts, and the operating decision you need to make. To put the decision into practice, call Cortex Marketing AI to arrange a consultation.
