By: Mudassir Malik
Submitted: 2009-07-07 01:25:34 | Word Count: 977
Are web surveys as good as mail surveys? Some research suggests that, in fact, web surveys are better than mail surveys. Here is the source for those interested: Kiernan, N.E., Kiernan, M., Oyler, M.A., & Gilles, C. (2005). Is a Web Survey as Effective as a Mail Survey? A Field Experiment Among Computer Users. American Journal of Evaluation, 26(2), 245-252.
Now lets take a look at the abstract. With the exponential increase in Web access, program evaluators need to understand the methodological benefits and barriers of using the Web to collect survey data from program participants. This means that the people who want to find the answer about web surveys try to understand the benefits and barriers in web survey use.
[ advertisement ]
In this experimental study, the authors examined whether a Web survey can be as effective as the more established mail survey on three measures of survey effectiveness: response rate, question completion rate, and the lack of evaluative bias. They study measured the response rate (how many people actually responded to the survey), and second how many questions they actually completed. Evaluative bias is the trickier term it means that people tend to evaulate their own characteristics more favorably. For example, if you are a man, you would look more favorably on masculine characteristics, and more negatively on female characteristics.
So what did these researchers do? They had 274 educators from university settings and the community who were attending a 2-day program to randomly receive a Web or physical mail survey evaluating the program (that they were attending). The abstract goes on to say, Among those participants successfully solicited by e-mail, Web survey participants were more likely to respond (95%) than mail survey participants (79%).
Now 95% is an extremely high participation rate but the important matter is that it is 16% higher than the mail survey. What about the other two factors question completeness and bias? Web survey participants completed similarly high numbers of quantitative questions as mail survey participants, provided longer and more substantive responses to qualitative questions... This means that on questions requiring true or false, or rating type of answers, both surveys were equally effective. For survey questions which required some writing, the computer survey did better than the mail-in survey.
The final point is that the research indicated that the online survey users did not demonstrate evidence of evaluative bias. The abstract does not go into detail on this critical point, but it is possible that the full research paper does.
Online surveys can have several other advantages over mail surveys. For one, they contain more data than mail in surveys - the time to completion can be measured in other words, how long did it take to answer all questions. One can also measure how long it took the person to click the survey link. Results are also instantly available to the person web site polls have this feature. Web site polls entice a person to vote, and after their vote, the current results are shown the the user. Online surveys can be taken anywhere and it is certain that computer penetration has increased since the original research in 2005. The advantage of mail surveys is that they be able to reach users previously inaccessible- for example people give out their mailing address to hospitals, oil change places, and electronic stores and sometimes the only way these companies can get ahold of that person is by physical mail. Clearly though, online surveys will continue to experience strong advantages over mail surveys.
