GUIDE — EVALUATION

Awareness evaluation, find where people drop off, and fix that

Awareness evaluation measures how far a target group knows about, understands, trusts and acts on a policy, programme or service, and through which channels it heard. The result shows the stage where most people drop off, so the next round of communication can change its message, channels and audience.

In this guide

01Awareness is more than "I've heard of it"
02The four stages: aware, understand, trust, act
03What to measure
04Designing the evaluation: before, during and after
05From results to communication strategy
06Commissioning an awareness evaluation: what the terms of reference should say

4stages of awareness

3moments to measure: before, during, after

6things a questionnaire should measure

In brief

Awareness has four stages: aware, understand, trust and act. Measuring only "heard of it" cannot explain why outcomes fall short
Measure a baseline before launch and again with the same questions, to show the communication worked
Find where each group drops off most, then change the message, channel, messenger or process at that point

Awareness is more than "I've heard of it"

"Heard of it" figures often look good, but they do not explain why people are not taking part.

Measuring only "heard of it"

Shows that publicity reached people
The figure usually looks good
Cannot say why people do not take part

Measuring every stage

Shows where people drop off
Separates reach, message, credibility and process problems
Points to what to change next round

Many awareness evaluations ask only whether people have heard of a programme and report the share who say yes. That number often looks good, yet the number of people who claim a benefit, register or change behaviour is much lower. The gap exists because awareness has stages. Hearing the name is only the first. People must understand what it is and how it concerns them, believe it works and is worth the effort, and then act. An evaluation that measures only the first stage can say that the publicity reached people, but not why the outcome did not happen. Measuring every stage shows where most people drop off, and each drop-off is a different problem with a different fix. If people have not heard of it, the problem is reach. If they have heard but do not understand, it is the message. If they understand but do not believe, it is credibility. If they believe but do not act, the barrier is usually in the process itself.

The four stages: aware, understand, trust, act

The share of the target group reaching each stage shows where communication gets stuck.

Tap a stage to see the detail

Figures are illustrative, as a percentage of the whole target group.

2 · Understand

How it is measured: Choosing or stating key facts, such as who is eligible and what to do
Why people drop off before this stage: The message is long, complex, or does not say why it concerns them
What to fix: Cut the message to a few facts and say clearly what people get

A framework that works for public policy, programmes and business campaigns alike splits awareness into four stages. Aware means having seen or heard of it, measured both unprompted and when shown the name or an image. Understand means being able to state the essentials correctly, such as who is eligible, what to do and what they get. Trust means believing it is useful, credible and right for them. Act means claiming the benefit, signing up or changing behaviour as intended. Each stage is measured as a share of the whole target group, so the largest loss between two stages is visible at once. The figures in the diagram are illustrative. In real work the shape of the funnel differs widely between groups, areas and channels, which is why results must be analysed by group.

What to measure

Measure both awareness and what explains it.

1Unprompted awarenessWhether it is really on people's minds
2Prompted awarenessRecognised when shown the name or an image
3UnderstandingKey facts stated correctly
4ChannelsWhere people heard, and which channels led to correct understanding
5Attitude and trustToward the subject and the messenger
6Intention and behaviourChecked against actual service data

Also ask what people still want to know or find confusing, to shape the next round of messages.

A good questionnaire measures both awareness and what explains it. The first part is unprompted awareness, asked without naming the subject, to see whether it is really on people's minds, and prompted awareness when the name or an image is shown. The second is understanding of the substance, asking respondents to choose or state key facts, which is more accurate than asking whether they understand. The third is channels: where people heard of it, and which channels led to a correct understanding. The fourth is attitude and trust, toward the subject and toward the organization communicating it. The fifth is intention and actual behaviour, checked against service or registration data where available. The last is what people still want to know or find confusing, which is direct input for the next round of messages.

Designing the evaluation: before, during and after

Plan the evaluation together with the communication plan, not after the campaign ends.

1Baseline before launchThe starting level, to prove the effect of communication
2Track duringMedia, social and service data
3Measure afterSame questions; compare exposed and unexposed
4Analyse by groupEnough sample for every group compared

Awareness evaluation is most useful when it is planned together with the communication plan, not after the campaign ends. First, measure a baseline before launch, to know the starting level. Without a baseline, the post-campaign figure cannot show whether awareness rose because of the communication or was already there. Second, track during the campaign with media, social and service data, to adjust before it is too late. Third, measure after the campaign with the same questions and, where possible, compare people who saw the communication with those who did not, or areas with and without activity. The sample must cover every target group to be compared. Social media data shows conversation and sentiment in real time but cannot replace a survey, because the people who post are not representative of the whole target group.

From results to communication strategy

Different stages, different problems, different fixes.

Tap a card to see the fix

Awareness results are valuable when they become decisions about what to say, to whom and through which channels in the next round. Start by finding the stage with the largest drop-off and fix that problem specifically. If people have not heard of it, increase reach in the channels the target group actually uses, not the ones the organization is used to. If they have heard but do not understand, cut the message down to the few facts people need. If they understand but do not believe, change the messenger or add evidence, such as stories from people who have benefited. If they believe but do not act, check the claiming or sign-up process, because the problem usually sits in the service rather than the communication. Then set approaches by group, since groups often drop off at different stages, set a target for each stage and measure again with the same questions. The same logic applies to business campaigns, from brand awareness to purchase and repeat purchase.

Commissioning an awareness evaluation: what the terms of reference should say

Check whether your terms of reference will produce results you can build a communication strategy on.

Tap each item your terms of reference already include

0 of 4 included

Good terms of reference make an awareness evaluation answer strategic questions rather than report the share who have heard of something. They should define the target group and areas clearly, including the subgroups to be compared and the minimum sample for each. They should require all four stages to be measured, aware, understand, trust and act, together with the channels of awareness. They should ask for a baseline before the communication where possible, and the same core questions when measuring again. The analysis should show where each group drops off and link results to service or registration data. Recommendations should state clearly which message, channel or process to change, and for which group. Deliverables should include the report, an executive summary, the dataset with identifying details removed, and a presentation to the communication team and the leadership.

How is awareness evaluation different from satisfaction evaluation? Awareness evaluation measures how far a target group knows, understands, trusts and acts on a policy or programme, including people who have not used it. Satisfaction evaluation measures how people who used a service feel about it.

Should awareness be measured before and after a campaign? Yes, where possible. A baseline shows whether the rise in awareness came from the communication. Without one, the post-campaign figure only shows the level at that moment.

What is the difference between unprompted and prompted awareness? Unprompted awareness is asked without naming the subject and shows whether it is really on people's minds. Prompted awareness shows the name or an image and asks whether people have seen it, so it is always higher.

Can social media data replace a survey? It is a good complement, because it shows conversation and sentiment in real time, but it cannot replace a survey: people who post are not representative of the target group, and social data cannot measure understanding or action.

How large should the sample be? For a large target group, a 95 percent confidence level and a 5 percent margin of error need about 400 respondents. If groups or areas are compared, each needs enough respondents of its own.

How are awareness results used? Find the stage where each group drops off most and fix that point: more reach, a shorter and clearer message, a different messenger, or a simpler claiming process. Then set targets and measure again with the same questions.

Insights