METHODOLOGICAL CHALLENGES IN THE USE OF THE INTERNET FOR SCIENTIFIC RESEARCH: TEN SOLUTIONS AND RECOMMENDATIONS

ULF-DIETRICH REIPS; TOM BUCHANAN; JOHN KRANTZ; KEN MCGRAW

doi:10.21697/sp.2015.14.2.09

Published: 2015-09-20

Vol. 15 No. 2 (2015)

METHODOLOGICAL CHALLENGES IN THE USE OF THE INTERNET FOR SCIENTIFIC RESEARCH: TEN SOLUTIONS AND RECOMMENDATIONS

ULF-DIETRICH REIPS , TOM BUCHANAN , JOHN KRANTZ , KEN MCGRAW

Studia Psychologica: Theoria et praxis

Section: Commentaries

https://doi.org/10.21697/sp.2015.14.2.09

Abstract

Researchers, editors, and other players in the scientific community have shown and continue to show high interest in and great enthusiasm for Internet-based research methods. However, methodological research has indicated that there are a number challenges that online researchers must consider. In this article, we provide information about solutions to ten common obstacles in Internet-based research. These are: Repeat or multiple submissions, comparisons between modes, reporting demographics of Internet participants, achieving compatibility for samples when Web and lab samples methods are to be compared, persistency of instrument, international nature of the Internet,
possible confounds arising from recruitment techniques, reliance on a single Website or recruitment method to attract participants, dropout and other types of nonresponse, and the preconception that Internet samples invariably lead to different results.

Keywords:

methodology, Internet-based research, guideline, recommendations, equivalence

Download files

PDF

Citation rules

REIPS, U.-D., BUCHANAN, T., KRANTZ, J., & MCGRAW, K. (2015). METHODOLOGICAL CHALLENGES IN THE USE OF THE INTERNET FOR SCIENTIFIC RESEARCH: TEN SOLUTIONS AND RECOMMENDATIONS. Studia Psychologica: Theoria Et Praxis, 15(2), 139–148. https://doi.org/10.21697/sp.2015.14.2.09

Cited by / Share

References

Aust, F., Diedenhofen, B., Ullrich, S., Musch, J. (2013). Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 45(2), 527–35. doi: 10.3758/s13428-012-0265-2
Google Scholar

Birnbaum, M. H. (2004). Human research and data collection via the Internet. Annual Review of Psychology, 55, 803-832
Google Scholar

Buchanan, T. (2000). Potential of the Internet for personality research. In M. Birnbaum (Ed.). Psychological Experiments on the Internet (pp. 121–140). San Diego: Academic Press.
Google Scholar

Buchanan, T., Johnson, J. A., & Goldberg, L. (2005). Implementing a Five-Factor personality inventory for use on the Internet. European Journal of Psychological Assessment, 21, 115–127.
Google Scholar

Buchanan, T. & Joinson, A. N. (2004, March). Are online-offline differences in personality test scores due to increased self-disclosure? Paper presented at German Online Research ’04, Duisburg, Germany, 30th March 2004.
Google Scholar

Buchanan, T. & Reips, U.-D. (2001). Platform-dependent biases in Online Research: Do Mac users really think different? In K. J. Jonas, P. Breuer, B. Schauenburg, & M. Boos (Eds.), Perspectives on Internet research: Concepts and methods. Retrieved February 2, 2013 from http://www.psych.uni-goettingen.de/congress/ gor-2001/contrib/buchanan-tom
Google Scholar

Buchanan, T. & Smith, J. L. (1999). Using the Internet for psychological research: Personality testing on the World-Wide Web. British Journal of Psychology, 90, 125–144.
Google Scholar

Buchanan, T. & Whitty, M. (2014). The online dating romance scam: Causes and consequences of victimhood. Psychology, Crime & Law, 20(3), 261–283.
Google Scholar

Buffardi, L. E. & Campbell, W. K. (2008). Narcissism and social networking Web sites. Personality and Social Psychology Bulletin, 34, 1303–1314.
Google Scholar

Coomber, R. (1997, June 30). Using the Internet for survey research. Sociological Research Online, 2. Retrieved June 16th, 2002, from http://www.socresonline.org.uk/2/2/2.html
Google Scholar

Frick, A., Bächtiger, M. T., & Reips, U.-D. (2001). Financial incentives, personal information, and dropout in online studies. In U.-D. Reips & M. Bosnjak (Eds.), Dimensions of Internet Science (pp. 209-219). Lengerich, Germany: Pabst.
Google Scholar

Freelon, D. G. (2010). ReCal: Intercoder reliability calculation as a Web service. International Journal of Internet Science, 5(1), 20–33.
Google Scholar

Freelon, D. G. (2013). ReCal OIR: Ordinal, interval, and ratio intercoder reliability as a Web service. International Journal of Internet Science, 8(1), 10–16.
Google Scholar

Furnham, A. & Buchanan, T. (2005). Personality, gender and self-perceived intelligence. Personality and Individual Differences, 39(3), 543–555.
Google Scholar

Hewson, M., Laurent, D., & Vogel, C. M. (1996). Proper methodologies for psychological and sociological studies conducted via the Internet. Behavior Research Methods, Instruments, and Computers, 28, 186–191.
Google Scholar

Joinson, A. N., Woodley, A., & Reips, U.-D. (2007). Personalization, authentication and self-disclosure in self-administered Internet surveys. Computers in Human Behavior, 23(1), 275–285. doi:10.1016/j.chb.2004.10.012
Google Scholar

Krantz, J. H. (2001). Stimulus delivery on the Web: What can be presented when calibration isn’t possible? In U.-D. Reips & M. Bosnjak (Eds.), Dimensions of Internet Science (pp. 113–130). Lengerich, Germany: Pabst.
Google Scholar

Krantz, J. H., Ballard, J., & Scher, J. (1997). Comparing the results of laboratory and World-Wide Web samples on the determinants of female attractiveness. Behavior Research Methods, Instruments, & Computers, 29, 264–269.
Google Scholar

Krantz, J. H. & Dalal, R. (2000). Validity of Web-based psychological research. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 35–60). San Diego, CA: Academic Press.
Google Scholar

McGraw, K. O., Tew, M. D., & Williams, J. E. (2000). The integrity of Web-delivered experiments: Can you trust the data? Psychological Science, 11(6), 502–506.
Google Scholar

Reips, U.-D. (1996). Experimenting in the World Wide Web. Paper presented at the 1996 Society for Computers in Psychology Conference. Chicago, IL.
Google Scholar

Reips, U.-D. (1997). Das psychologische Experimentieren im Internet [Psychological experimenting on the Internet]. In B. Batinic (Ed.), Internet für Psychologen (pp. 245–265). Göttingen, Germany: Hogrefe.
Google Scholar

Reips, U.-D. (2000). The Web experiment method: Advantages, disadvantages, and solutions. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 89–117). New York: Academic Press.
Google Scholar

Reips, U.-D. (2002a). Internet-based psychological experimenting: Five dos and five don’ts. Social Science Computer Review, 20(3), 241–249.
Google Scholar

Reips, U.-D. (2002b). Standards for Internet-based experimenting. Experimental Psychology, 49(4), 243–256.
Google Scholar

Reips, U.-D. (2002c). Theory and techniques of conducting Web experiments. In B. Batinic, U.-D. Reips, & M. Bosnjak (Eds.), Online Social Sciences
Google Scholar

(pp. 229–250). Seattle: Hogrefe & Huber.
Google Scholar

Reips, U.-D. (2007). The methodology of Internet-based experiments. In A. Joinson, K. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford Handbook of Internet Psychology (pp. 373–390). Oxford: Oxford University Press.
Google Scholar

Reips, U.-D. (2010). Design and formatting in Internet-based research. In S. Gosling & J. Johnson, Advanced methods for conducting online behavioral research (pp. 29–43). Washington, DC: American Psychological Association.
Google Scholar

Reips, U.-D. & Birnbaum, M. H. (2011). Behavioral research and data collection via the Internet. In K.-P. L. Vu and R. W. Proctor (Eds.), The handbook of human factors in Web design (2nd ed., pp. 563–585). Mahwah, New Jersey: Erlbaum.
Google Scholar

Reips, U.-D. & Garaizar, P. (2011). Mining Twitter: Microblogging as a source for psychological wisdom of the crowds. Behavior Research Methods, 43, 635–642. doi:10.3758/s13428-011-0116-6
Google Scholar

Reips, U.-D. & Stieger, S. (2004). Scientific LogAnalyzer: A Web-based tool for analyses of server log files in psychological research. Behavior Research Methods, Instruments, & Computers, 36, 304–311.
Google Scholar

Schmidt, W. C. (1997). World-Wide Web survey research: Benefits, potential problems, and solutions. Behavior Research Methods, Instruments, and Computers, 29, 274–279.
Google Scholar

Schmidt, W. C. (2007). Technical considerations when implementing online research. In Joinson, A., McKenna, K., Postmes, T. and Reips, U.-D. (Eds.), The Oxford handbook of Internet psychology (pp. 461–472). Oxford, Oxford University Press.
Google Scholar

Stieger, S. & Reips, U.-D. (2010). What are participants doing while filling in an online questionnaire: A paradata collection tool and an empirical study. Computers in Human Behavior, 26, 1488–1495.
Google Scholar

Voracek, M., Stieger, S., & Gindl, A. (2001). Online replication of Evolutionary Psychology evidence: Sex differences in sexual jealousy in imagined scenarios of mate’s sexual versus emotional infidelity. In U.-D. Reips & M. Bosnjak (Eds.), Dimensions of Internet Science (pp. 91–112). Lengerich, Germany: Pabst.
Google Scholar

Whitty, M. T., Buchanan, T., Joinson, A. N., & Meredith, A. (2012). Not all lies are spontaneous: An examination of deception across different modes of communication. Journal of the American Society for Information Science and Technology, 63(1), 208–216. doi:10.1002/asi.21648
Google Scholar

Wright, K. B. (2005). Researching Internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of Computer-Mediated Communication, 10(3).
Google Scholar