Satisfaction evaluation, from an average score to what to fix first

Many organizations run a satisfaction survey every year because it is a required indicator. The result is usually an average of around 4 out of 5, read as high or very high, and the report is filed while the service barely changes. The problem is not the score but the question the evaluation was set up to answer. An evaluation for reporting asks only how satisfied users are. An evaluation for improvement must answer four more questions: what users are dissatisfied with, which users, why, and what to fix first when budget and staff are limited. Answering them has to be designed in from the questionnaire onward: ask about both satisfaction and importance for each aspect, collect data so user groups can be separated, and include open questions that capture reasons. The extra cost is small, and the result turns from a number to report into information the leadership can act on.

Start by defining the population clearly: the people who actually used the service in the period being evaluated. Then plan the sample to cover different channels, service points and times. For a large population, a 95 percent confidence level and a margin of error of plus or minus 5 percent need about 400 respondents, following Yamane's formula or the Krejcie and Morgan table. If results must be compared between groups, such as branches or types of user, each group needs a large enough sample of its own. Questionnaires commonly use a five-point scale, with importance ratings and open questions. Fieldwork should be done by people independent of the service: when the staff who just served someone hand out the form, respondents tend to score higher than they feel. Mixing channels, such as on site, by phone and online, also captures people who never come to the office.

The tool that turns results into decisions most directly is importance–performance analysis. Each aspect of the service is placed on a two-axis grid: importance to users on one axis and the score achieved on the other. Aspects that matter a lot but score low are the ones to fix first. Aspects that matter and score well are strengths to keep. Aspects that score well but matter little may be using resources that could move elsewhere. The aspects to fix are then traced along the user's journey to find where the problem really sits, such as duplicated documents or a step that needs several visits, and the service is redesigned at that point. Where groups want different things, service approaches are set by group, which in business is the work of customer strategy and retention. Finally, set targets for the aspects being fixed and measure again with the same questions in the next round, to show that the change worked.