Core OOH Knowledge Series · Buyer's guide

How to Choose a Billboard Pre-Testing Tool: 6 Questions to Ask

Choose a pre-testing tool that tests under outdoor viewing conditions, explains how it scores, gives the same result for the same file, and tells you what to fix. Use the six questions below to check any tool, including this one.

Three approaches Buyer's checklist Method transparency Reading vendor claims
TL;DR
  • Pre-testing tools fall into three groups: perception panels, predictive attention models, and deterministic rule-based review. They answer different questions.
  • Ask six questions: Is the method published? Is the result repeatable? Does it test OOH conditions? How fast is it? What does access require? Does it tell you what to fix?
  • Treat a performance claim with no baseline, method, or sample as marketing, not evidence.
  • Best practice: confirm the ad can be read first, then run perception research if you need it.

The three approaches to billboard pre-testing

Most tools sold for billboard and out-of-home (OOH) pre-testing fit one of three approaches. None is "best" for every question. What matters is knowing which question each one answers.

Swipe to see the full table →

Question Perception panels Predictive attention models Deterministic rule-based review
What it measures What people say, feel, or remember after seeing the ad Where viewers are likely to look, estimated by a trained model Measurable properties of the artwork, such as contrast, copy length, hierarchy, and letter size, under stated viewing conditions
Where the result comes from Answers from a recruited sample of people Patterns the model learned from its training data, often eye-tracking recordings Fixed, published rules applied to the file and settings
Same input, same result? VariesA new sample can give different answers Usually, per model versionCan change when the model is retrained Yes, per software versionSame file, copy, and settings give the same result
Can you trace why? Through the questions and responses Harder: the logic sits in learned model weights Yes: each score maps to a stated rule
OOH conditions Only if the study simulates speed and distance Depends on the training data. Widely used benchmarks record people looking at images on a screen, not at road speed. Only if the rules include format, distance, and speed
What it can't tell you Whether the ad is physically readable at speed, unless tested that way Whether the copy is long enough to read in the time available How people feel about the ad, or whether it will drive sales

Notes: "Predictive attention models" refers to computational saliency models, which researchers evaluate against human eye-tracking data. The MIT Saliency Benchmark, for example, scores models against fixations recorded while people viewed images on a screen. Research has also found that these models can generalize unevenly to new types of images. Sources are listed at the end of this article.

The key point

Panels and attention models tell you about people's responses. Rule-based review tells you whether the message can physically be read at the speed and distance you choose. For OOH, the second question comes first. For more on panels, see algorithmic vs panel testing and why traditional billboard testing methods fail.

The 6 questions to ask any pre-testing tool

Use these questions whether you are comparing vendors, checking a free tool, or reviewing a report someone sent you.

  • Is the method published?

    You should be able to read what is measured, how much each check counts, and what a grade means. If a score cannot be explained, it cannot be checked or defended to a client.

  • Does the same creative get the same result?

    Run the same file twice with the same settings. If the grade changes, you cannot tell whether a design change helped or the tool simply varied.

  • Does it test outdoor viewing conditions?

    Look for board format, viewing distance, and speed as inputs. A test of how an ad looks on a desk screen does not answer how it reads from a moving car. See the Time-in-Environment Fallacy.

  • How long until you get a result?

    Pre-flight checks happen during design, often several times. A tool that returns results in minutes can be used on every revision. One that takes days usually gets used once, if at all.

  • What does access require?

    Check the cost, whether you need a sales call, demo, or account, and where your artwork goes. Unreleased creative is often confidential.

  • Does it tell you what to fix?

    A score alone does not improve an ad. Look for specific, prioritized fixes tied to the measurement, such as "increase headline contrast" or "cut copy to fit the reading time."

How Ad Corrector answers the 6 questions

Ad Corrector is a deterministic, rule-based review tool. Here are its answers, with links so you can check each one yourself.

Swipe to see the full table →

Question What a good answer looks like Ad Corrector
1. Published method Checks, weights, and grade bands are public Six scored checks, their weights, and the A–F grade bands are published on the methodology page. Contrast uses the WCAG relative luminance definition.
2. Repeatable Same input gives the same result Yes, for the same file, confirmed copy, settings, and software version. Re-saving or recompressing an image changes its pixels, so test the same file when comparing.
3. OOH conditions Format, distance, and speed are inputs You set format or board size, viewing distance, speed, and campaign goal. Results include Speed View, a letter height reference, and reading time compared with time in view.
4. Speed Fast enough to use on every revision Runs in your browser. Results typically appear within seconds.
5. Access Clear cost, minimal barriers Free. No account, demo, or sales call. Optional saved tests stay on your own device.
6. Fixes Specific, prioritized changes A Fix First plan, issue markers on the artwork, and a downloadable report.
What Ad Corrector does not measure

It does not measure emotion, recall, purchase intent, or sales, and a high grade does not predict campaign performance. For digital billboards, it reviews a static frame, not animation or display schedules. Use it to confirm the ad can be read, then use audience research for the rest.

How to read a performance claim

Pre-testing marketing often includes figures like "X% more effective." Before you rely on one, ask for:

  • The baseline: more effective than what, measured how?
  • The outcome: attention, recall, clicks, store visits, or sales? These are different results.
  • The method and sample: how many ads, how many people, and under what viewing conditions?
  • Who measured it: the vendor, a client, or an independent party?
  • Whether it is published: can you read the study, or only the headline number?
Rule of thumb

A percentage without a baseline is not a result. If a vendor can't share the method, treat the figure as a claim, not evidence.

The right order: readability first, perception second

If drivers cannot read the message in the time they have, perception results mostly measure that problem rather than the idea. Confirm readability first.

Step 1: Structural pre-flight

Check contrast, copy length, hierarchy, and letter size at your real viewing distance and speed. Fix and re-test until the message reads.

Step 2: Perception research (optional)

Once the ad reads, test how people respond to the idea, if the budget and timeline allow.

For the full standard, see the Billboard Pre-Testing Standard.

FAQ

What is a billboard pre-testing tool?
A billboard pre-testing tool checks outdoor ad creative before it is printed or goes live. Depending on the approach, it measures audience response, predicts where people will look, or checks whether the artwork can be read at a set viewing distance and speed.
What is the difference between predictive attention AI and rule-based testing?
Predictive attention models estimate where viewers are likely to look, based on patterns learned from training data such as eye-tracking recordings. Rule-based testing applies fixed, published rules to the artwork, such as contrast and copy length, so each score can be traced to a rule and repeated.
Why does repeatability matter in billboard testing?
If the same file can get different results, you cannot tell whether a design change improved the ad. A repeatable test lets you compare versions and know the change caused the difference.
Is a free billboard testing tool reliable?
Price does not decide reliability. Method transparency and repeatability do. A free tool that publishes its method and gives the same result for the same file can be checked. A paid tool that does neither cannot.
Should I use panels or a rule-based tool?
They answer different questions. Use a rule-based tool first to confirm the ad can be read at real speed and distance. Then use panels if you need to know how people respond to the idea.

Sources

  • Judd, T., Durand, F., and Torralba, A. (2012). A Benchmark of Computational Models of Saliency to Predict Human Fixations. MIT Technical Report MIT-CSAIL-TR-2012-001. dspace.mit.edu/handle/1721.1/68590
  • MIT Saliency Benchmark (datasets and evaluation). people.csail.mit.edu/tjudd/SaliencyBenchmark
  • Linardos, A., Kümmerer, M., Press, O., and Bethge, M. (2021). DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling. arxiv.org/abs/2105.12441
  • W3C. Web Content Accessibility Guidelines (WCAG) 2.1, definition of relative luminance. w3.org/TR/WCAG21
  • United States Sign Council. Guideline Standards for On-Premise Signs (2018), legibility index. usscfoundation.org
  • National Academies of Sciences, Engineering, and Medicine. NCHRP Report 600: Human Factors Guidelines for Road Systems. nap.nationalacademies.org

OOH Pre-Flight Creative Checklist

Opens in a new tab. Print-ready.
Dan Resnikoff Ad Corrector
Author: Dan Resnikoff
Principal Billboard Strategist
Connect: Ad Corrector | LinkedIn