A buyer intent benchmark starts with a data request that a sales team and analyst can fulfill consistently. Ask for dated events, frozen score snapshots, membership in each eligible cohort, and clear outcome rules. Include denominator counts and known exclusions. Request a narrow sample extract before asking for a full analysis period. This article proposes a research packet and makes no claim that Cortex ran a benchmark.
Where the decision gets complicated
Begin with a short project memo. State whether you want to study repeat visits, pricing engagement, combined external and website intent, or changes in a scoring model after feedback. Each question requires a different index point. A request for all activity and all deals can produce a large file while leaving the analyst unable to tell which activity happened before the decision. Ask for stable anonymized keys in approved environments and avoid fields that reveal individual customers unnecessarily.
Suppose a fictional agency receives a spreadsheet with qualified leads and score totals, but no account snapshots. The team can still audit outcome definitions. It cannot reconstruct the candidates that were eligible on the date the score was used. A short test extract containing a mix of accepted, pending and excluded records can expose this problem before a client spends days preparing files.
What to record at handoff
Request an intake file that shows a few cohorts with pseudonymous units and observation dates. A score file can hold versions and dates at separate grain. A CRM file can hold confirmed outcome timestamps. The analyst should verify join uniqueness and calculate the eligible denominator without asking for names or exact client records in a shared report.
Each of the four benchmark questions has a different evidence gate. Repeat visits need a stable observation rule. Pricing needs an active comparison. Combined intent needs aligned clocks and observation coverage. Feedback needs model versions and a test cohort insulated from learning. A packet is ready when the specific question can be reproduced under approved controls.
Event file
For the event file check, plan a sensitivity check that changes one defensible assumption at a time. Ask for a pseudonymous unit, event family, observed timestamp, ingestion timestamp, source type, and approved match level. Request only the categories needed for the question.
For this case, the expected next step is clear. Require explicit codes for unavailable, withheld, and ineligible records so analysts can report the difference. For the event file check, a study reviewer should be able to explain the decision time, comparison unit, and evidence that would justify any exclusion. Where those conditions differ between groups, report the difference as part of the method instead of presenting the final rate as self-explanatory. The rule should be precise enough to explain the difference to the owner.
Write the request with the eventual reviewer in mind
An anonymized packet should be sufficient for an authorized reviewer to reproduce a count without receiving a complete client database. Request data dictionaries with permitted value codes, timestamps with time zones, and a record of which clients or markets are in scope at an approved aggregate level. Ask how CRM changes are represented and whether one account can appear multiple times. Obtain outcome maturity dates and document when a case is considered unresolved. This makes a later benchmark table intelligible to people who did not work on the original pilot.
Score snapshots
For feedback research, request a frozen baseline and updated model with training cutoffs and a shared out-of-time evaluation cohort. Preserve the prediction target and version.
Ask whether the score snapshots check is measured equally well in both groups. When reviewing the score snapshots check, record which cases it moves into or out of the primary denominator. Recalculate packet readiness using the approved definition, then assess whether unresolved cases could change the interpretation. If they could, report the size of the gap rather than applying a favorable assumption. To close the review, write down what the decision owner is expected to do. Check whether a refreshed score reflects events that occurred after the intended evaluation moment.
Set an owner for the exceptions before turning on broader routing.
Connect the score snapshots check to the report on packet readiness. Establish whether an exception changes eligibility, leaves the result unknown, or just produces a note. Removing a record and counting it as a failure have different effects on the reported percentage. Describe the handling clearly enough for a second reviewer to reconstruct it.
Follow two fictional records
For repeat engagement, a sample file should include a visitor or account who has a second observed session before the index and a person whose second session falls after it. For pricing, include a known open-deal record that needs to be excluded from a new-demand cohort.
For feedback evaluation, try a frozen score with an old version ID and another with a revised version. If the export only keeps the most recent score, request a snapshot archive or narrow the research question to a prospective study.
CRM outcomes
Request qualification events with a versioned definition, timestamps, disqualification reasons at an approved level, and final outcomes when mature. Apply the rule while the source evidence is still available. Count open and unresolved cases separately instead of silently dropping them or calling them losses.
Apply the crm outcomes check rule before seeing the outcome totals. This keeps a retrospective analysis from silently favoring one group. If the definition genuinely needs correction, describe why, keep a dated revision, and show the impact when the prior definition was feasible. Keep those details adjacent to the finding so they survive when it is copied into a decision memo. Capture the business reason, the first affected record or reporting date, and the person who approved the change.
Put a failure example beside the standard example for the crm outcomes check. For the first example, require the reviewer to explain why the rule should stop. For the second, ask why it should advance. This exposes definitions that sound precise in a meeting but do not lead to consistent decisions under ordinary conditions.
Cohort file
Specify the contribution of the cohort file check to the study design before calculating outcomes. Ask for observation dates, assignment rules, denominator counts, campaign group and exposure only when appropriate, and approved comparison strata.
Put the next step in the operating notes. Preserve assigned groups even when delivery falls short if the analysis estimates assignment effects. For the cohort file check, a study reviewer should be able to explain the decision time, comparison unit, and evidence that would justify any exclusion. Where those conditions differ between groups, report the difference as part of the method instead of presenting the final rate as self-explanatory. Decide that severity before looking at the case totals so difficult failures receive consistent attention.
Start where the data can answer a specific question
If the scoring archive is missing, a retrospective comparison of two model versions may be impossible. A prospective study can begin by saving the current version and agreeing on a later holdout. If there are strong events but weak CRM qualification records, the first deliverable could be a measurement-quality review rather than a public benchmark. Keep a gap register with the missing field, the effect on the proposed finding and the owner who can repair it. The register should identify which hypotheses remain testable after narrowing the population.
Publication approval
Agree on aggregate release limits, ownership review, reproducibility requirements and the rules for small cells before collecting data.
Turn the publication approval check into an auditable row-level rule. When reviewing the publication approval check, record which cases it moves into or out of the primary denominator. Recalculate packet readiness using the approved definition, then assess whether unresolved cases could change the interpretation. If they could, report the size of the gap rather than applying a favorable assumption. The review record for the publication approval check should retain this disposition. Keep the raw packet in approved handling and publish only authorized totals and enough method detail to evaluate the claim.
Record the decision at the grain the business uses to act.
Test the boundary of the publication approval check using a case that almost meets the documented condition. Record the exact reason it falls outside. If two reviewers give different answers, inspect the underlying definition before you widen the rule. The disagreement may show an ambiguous requirement or a field that should be confirmed at intake.
A source to check before implementation
IAB Tech Lab's finalized ECAPI guidance specifies aspects of exchanging event data into clean rooms. A common schema is a useful starting point, while dates, cohort eligibility and CRM outcome definitions remain research-specific. See the IAB Tech Lab ECAPI clean room guidance.
A fictional packet readiness check
The following example is hypothetical and uses requested field-and-rule items as the denominator. It counts cases that were available with definitions and approved lineage. Save the underlying counts together with the label used to report packet readiness. Totals and rates will answer different questions when group sizes differ.
| Illustrative group | Count meeting rule | Eligible count | Rate |
|---|---|---|---|
| Narrow research request with defined fields | 16 | 22 | 72.73% |
| Earlier general export request | 12 | 22 | 54.55% |
The calculation gives 72.73 percent for narrow research request with defined fields and 54.55 percent for earlier general export request. The first is 18.18 percentage points above the second, using the underlying rates before rounding. The rate gap and the volume gap are separate. In this example the two numerators are 16 and 12. A report that drops the underlying counts can hide that relationship.
Completeness is a property of the packet and research question. It cannot establish a finding about buyers. Missing score versions can block one research question even when other tables are complete.
Record the snapshot date beside the packet readiness result. For this hypothetical comparison, use the same written review standard for both groups. If a reviewer changes the comparison rule, keep the original packet readiness calculation and clearly identify the revised version.
A publishable comparison of packet readiness needs dated cohort membership, an approved aggregation policy and a reproducible calculation. Use a method that fits the assignment and overlap. The study owner should be able to give a reviewer a permitted replication path before a public number is written.
Specify who can link the approved data
The team that owns the score file may not be authorized to inspect the full CRM export. Agree on where linking occurs and which aggregates can be released. Ask whether the chosen pseudonymous keys are stable over the research period and whether one key can represent multiple locations. A controlled workspace can support an approved join, but the study still needs valid time order and a consistent denominator. Retain the restricted method evidence and publish only appropriate results.
Before reusing the fictional packet readiness comparison, give each group an observation date and a reason it belongs in the report. The 22 and 22 denominators are the eligible groups in this example. A report restricted to the 16 and 12 successes would lose the reference needed to interpret the result. Retain a separate ledger for exclusions and date the moment each group completed its observation window or test sequence.
A research owner reviewing packet readiness needs a way to reproduce the comparison and a documented decision about what is publishable. Confirm that every released aggregate has the right unit, denominator and approved scope. Keep the hypothesis and method available even if the actual outcome data are not sufficient to support a result.
Collect what the scoring question requires
If you are studying two-way intent data, approved examples outside the original list may be informative. Preserve the original acquisition route and which examples were used in learning. That way an analyst can ask what the examples added to the scoring model without attributing those customer sales to an earlier intent list.
Keep client permissions and the observation period on the first page of the research request.
For more on the signals behind this decision, read the Cortex intent data insights.
Give a reviewer enough evidence to challenge the design
Start with the best-supported research question and retain unsupported questions as proposals. Choose one benchmark question to start the conversation. A sample showing its denominator, date boundaries and qualification record will be more useful than a bulk list of contacts. Agree on the approved handling and output before discussing what would be appropriate to publish.
See how Cortex Marketing AI uses customer feedback to refine the signals a team can test.
Bring your anonymized data inventory, a proposed comparison, and the approvals available for aggregate research. For help designing a test around that decision, call Cortex Marketing AI to book an intent consultation.
