Posted by: Ali | October 16, 2009

Improving the response rate in surveys through incentives

One of the great things about working at a university library is you can get to along to lunchtime lectures on Wednesdays to hear academics talking about their research. The Dept of Communication, Journalism and Marketing here at the Turitea campus of Massey University has been running an excellent series of talks this year. This week Dr Mike Brennan spoke about “Doing research on the cheap”.

Mike spoke about using surveys to conduct experimental studies on how to improve return rates (this was for mail surveys). One of the key ways to improve return rates is to use an incentive, but what works best? As part of this experiment surveys had 20 cents, 50 cents, or $1 attached to them, or a chance to go into the draw for $200 or a $200 voucher, and there was a control with no incentive.

Return rates are improved by providing the incentive with the survey, rather than the promise of a prize draw or voucher. Turns out the 50 cent incentive got the best return from the first mail out in this experiment:

  Mail 1 Mail 2 Mail 3
Control 24.7 46.6 57.5
20c mailout 1 27.1 43.5 54.1
50c mailout 1 46.0 66.7 74.7
$1 mailout 1 42.3 59.2 69.0
20c mailout 2 28.9 51.8 63.9
50c mailout 2 15.7 39.8 54.2
$1 mailout 2 23.5 51.9 69.1
$200 prize draw 25.6 43.6 57.7
$200 voucher 18.3 46.3 61.0

(NB We didn’t get a date that this research was done. Many thanks to my colleague Jane Brooker for noting the figures for this table.)

Obviously a small cash incentive is not true compensation for someone’s time, so wording such as “please accept this as a token of our appreciation” in the covering letter works well.

These days NZ Post doesn’t allow cash to be sent through the mail. Intrepid researchers have tried alternatives to cash in postal surveys. These include:

  • Pens
  • Tea bags, coffee bags, or both
  • Scratch and win cards
  • Stamps
  • Golf balls (!)
  • Turkeys (presumably vouchers for them!)

                                                                                                                                                                                                                                                  Gold foil wrapped chocolate coins have also been tried, but a better option is the chocolate squares from Whittakers. Judging by the murmurs of approval from the audience this is likely to be a great incentive!

 The other important thing is not to use just one mail out, but to send a reminder or another copy of the questionnaire in subsequent mailouts. As the table above shows this will increase response rates. Various combinations have been trialled and the 3 stage combo of questionnaire with chocolate/replacement questionnaire/follow up letter was mentioned as being successful.

Other external treatments have also been researched – these include using stamped v franked envelopes, brown v white envelopes, tone of the cover letter, status of the researcher (professor v student), colour of the questionnaire. Mike’s profile page details the research he and colleagues have published in this area.

PS –  I see there was a session at the recent LIANZA conference on designing effective surveys by Rachel Esson from Victoria University of Wellington, so that’s one conference paper I’ll be looking out for.

 Originally posted http://alisonwallbutton.wordpress.com

Reflections

  • backs up what I have learnt in my postgraduate marketing studies
  • question of incentives is interesting – the mini chocolate fish as incentive has worked well in our surveys we run, so an incentive doesn’t need to be large to get a response. Interesting to reflect that the I was initially skeptical about the $5 printing credit for students evaluating the online learning objects, but in fact students are very happy with this. An important learning i think!

fish poster

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: