How to Backtest an Intent Score Against Your Sales Team’s Weekly Capacity

November 11, 2026•11 min read

An intent score is effective for a sales campaign when it moves more of the right accounts into the limited number of slots your team can actually work. To test that, reconstruct the queue as it looked before each selling week, take the top accounts your team could have handled, and count how many later met a fixed qualification standard. Compare that result with a simpler queue built from the same eligible accounts. A score that looks promising across a whole database can still fall short on the first page of the list where reps make their decisions.

This is a worksheet for a retrospective evaluation. Its results can tell you whether a scoring rule ranked recorded outcomes usefully in a particular setting. A separate prospective test can tell you whether changing the order of work improves what happens next. Keeping those two jobs separate makes it easier to decide when a model is ready for a live pilot and what the pilot should be designed to answer.

Build a worksheet around a decision you can replay

Start with one selling motion and one operating rhythm. A rep assigning Tuesday morning prospecting time to commercial refrigeration accounts faces a different decision from an account manager monitoring renewals across a quarter. For a prospecting worksheet, give each row one account and one weekly decision date. Record account eligibility at that date, score and score version, baseline rank, actual contact attempts, and a downstream qualification outcome with its date. Include the business segment or territory only at the level necessary to check comparability.

Set the cutoff in the same time zone as the queue, then carry the cutoff forward consistently. If the team sees scores Monday morning, the inputs need to have been available before that view was generated. A research event may have occurred Sunday night but arrived Tuesday. For the Monday replay it belongs in a later snapshot. Keep both the occurrence time and the arrival time if you have them. Replacing arrival time with occurrence time can let delayed information slip into a week where no rep could have used it.

Specify which accounts could genuinely have entered the queue. Territory rules, an existing opportunity, a partner assignment, and contact restrictions can change eligibility without changing interest. Keep the business reason for an exclusion in a separate field. If an account was already in an active opportunity, excluding it from the prospecting evaluation prevents the score from getting easy credit for identifying work already underway. If reps could only take action on a defined set of accounts, let both ranking rules compete for those same slots.

A single account can enter multiple weekly snapshots. For an initial evaluation, consider excluding it from reentry for the same interval your live team uses to avoid duplicate work. If you keep repeated rows, report whether your denominator is queue slots or unique accounts. One large account that appears in six snapshots can otherwise make six seemingly successful predictions out of a single opportunity. The correct choice depends on the decision. A queue review tracks slots, while a customer acquisition estimate usually needs unique accounts.

Define success before opening the historical outcomes

Ask the sales manager to show you the evidence required for an accepted opportunity. A fictional refrigeration distributor might require a site in its territory, a type of equipment it can service, a buyer who can describe a project or operating problem, and a mutually agreed next step. A polite email response would be useful activity, but it would stay in a different column. This distinction allows the team to examine whether the score produces conversations that can turn into work.

Choose a time window that matches the outcome. A short horizon may suit a first diagnostic conversation. An installation proposal could need substantially longer. A weekly snapshot from the end of the export might still be waiting for outcomes. Mark that row as immature and include it in a later review. Treating immature rows as failures can make a newer model appear weaker for the simple reason that it was evaluated too soon.

Keep four states available: qualified, reviewed and unqualified, pending, and outcome unobserved. An account never reached by sales belongs in the last group unless you have another reliable way to observe the outcome. If the company wants to measure inbound opportunity creation across the entire CRM, that is possible as a separate outcome. It answers a narrower question about recorded opportunities, and you should say which routes into the CRM count. Good labels make it possible to see whether a low result reflects weak targeting, poor follow-through, or an incomplete trail.

A useful ownership rule is to let sales approve the commercial qualification criteria and let operations approve which evidence satisfies the fields. For example, a rep can enter the next step and the buyer's stated need, while operations checks that those fields were recorded within the evaluation window. This avoids a common retrospective problem where a successful deal gradually becomes better documented than an equally relevant account that remains open.

Use a baseline the business can recognize

A good baseline is the queue your team can realistically use without the new score. It could prioritize known fit, then sort by the recency of a relevant inquiry. It could also mirror a documented process that reps already follow. Give it a fixed tie breaker so two analysts can reproduce the same list. A random ranking can provide a reference for context, but beating random selection rarely settles an operational decision.

Imagine a fictional team with room for twelve new accounts each week. For one mature week, the custom intent queue contains four accounts that later meet the qualification rule, and the baseline contains three. Both lists have twelve slots, so the observed qualified share is four divided by twelve versus three divided by twelve. That is about 33.3 percent versus 25 percent, a difference of about 8.3 percentage points. Relative to the baseline share it is about 33.3 percent higher. These are arithmetic examples with invented inputs, and a single week offers little basis for a durable estimate.

The two queues might share nine accounts. In that case the practical argument concerns three decisions on each side. Review the accounts moved into the queue and the accounts displaced by them. Ask whether the change selected a different industry, a different geography, or a stronger buying situation within the same niche. If the only additional qualified account came from a territory the team stopped serving, the original comparison has an eligibility defect. If the score repeatedly trades out weak recent inquiries for viable commercial projects, you have a specific hypothesis to test live.

Report the selected count, the number with known outcomes, the qualified count, and the overlap between lists. If unknown outcomes are frequent and uneven, calculate a simple range for each queue. The lower bound treats each unknown as a failure and the upper bound treats each unknown as a success. The bounds are deliberately blunt. If they overlap widely, the decision may depend more on better observation than on a fancier ranking formula. Preserve the unknown count even when the main number uses a consistent complete-case definition.

Inspect the rank that sales actually uses

Review the result at several realistic queue sizes, such as the normal weekly capacity, holiday coverage, and the capacity available when a rep is out. Choose those values before seeing outcomes. A model that excels in the first eight slots and flattens out in the next twelve might be useful for a small specialist team. A model with steady performance through a larger queue could suit a centralized group. Avoid using whichever cutoff happens to make the chart most flattering.

Consider the workload created by the replacement accounts. Two accounts can have identical qualification status while taking very different amounts of prospecting time. A spreadsheet can include attempted contacts and time spent researching or validating each account. Compare qualified opportunities per eligible slot and, where tracking is consistent, per selling hour. The first tells you about ranking under a volume limit. The second adds a practical check on whether reps can afford the additional work.

A ranking can be helpful without being calibrated as a probability. If an account has a score of 80, ask whether the system claims that 80 means an 80 percent chance of a specific outcome over a specific period. If the score is simply an ordering tool, use rank-based measures. If the model actually publishes probabilities, compare predicted and observed rates by preselected score bands and track uncertainty within each band. Writing this distinction into the worksheet makes it harder for someone to turn a convenient scale into a sales promise.

Make the 60-day comparison fair

With Cortex two-way intent data, clients can choose relevant websites, topics, sources, and buying signals, then use their own lead, customer, website and deal feedback to refine prioritization. Scoring can begin about 24 hours after a lead-list subscription. A day-one score is a starting rule for allocating attention. It can be recorded and evaluated while the client gathers enough feedback to justify changes. Customers supplied by the business can also inform the learning even when they came from outside the original list.

To test improvement after 60 days, freeze the first rule and the updated rule. Use a later evaluation set and apply both versions to the same eligible snapshots where possible. Use only the facts available at each snapshot, and hold the outcome definition, maturity window, and queue size constant. Keep accounts or events used to tune the score separate from the evaluation evidence. If you only compare the first month with a later month, changes in staffing, demand, or CRM completeness may explain the difference. Document these changes and present the comparison as observational.

If model versions cannot be reconstructed on historical inputs, begin capturing them now. Save the version and rank for the queue the team sees and, where practical, record the alternative rule in parallel. This small administrative choice prevents a future debate about whether the new model performed better or whether the old predictions simply disappeared. Customer-specific learning can become proprietary to the client's niche, offer, geography and sales cycle. The evaluation should establish whether that learning improved the client's decision, while source licensing and any exclusivity of underlying records are separate matters.

Decide what to test in a live campaign

A retrospective backtest has a built-in limitation when reps historically worked the old list. Accounts the new rule would have promoted may have received little attention and therefore have fewer observable outcomes. A live test can assign comparable accounts or teams to different queue rules under the same outreach policy. If several people at the same company can be contacted, assign at the account level to reduce spillover. Define the handling for inbound requests and existing commitments before assignment. Record whether the assigned workflow was followed and keep assigned counts visible even if some accounts never receive a first touch.

Agree on an outcome that would change the budget or workflow. For example, a manager may need the new queue to generate enough additional accepted opportunities to justify the time required to maintain CRM feedback. Decide how many mature weeks you can wait and what uncertainty would warrant extending the evaluation. If the result is mixed, inspect displacement cases and workload before rewriting the score. A narrow positive result in one market can justify keeping the rule local while you gather evidence elsewhere.

For related implementation detail, read Cortex’s Cortex overview of website engagement and matching. The worksheet makes those labels accountable to a decision date, a fair comparator, and an observable business outcome. Bring a sample of past queue snapshots and one qualification rule when you contact Cortex Marketing AI to request an intent review. Start with the question your sales lead has to answer every week: which accounts should get the next available slots?

For examples of the buying signals that feed these decisions, see Cortex’s buyer intent data guide.

Before the review, ask one analyst and one sales manager to reconstruct the same past Monday independently. Have them write down the first account they disagree about and trace that account through eligibility, score availability, and outcome status. If the disagreement is caused by a changed territory or an ambiguous CRM stage, update the worksheet instructions while preserving the original data. A reproducible weekly comparison is worth more than a detailed dashboard whose results depend on which team exports the spreadsheet. Once the replay is consistent, establish who will sign off on changes to the qualification rule and where those changes will be recorded.

Back to Blog