FREE ELECTRONIC LIBRARY - Books, dissertations, abstract

Pages:   || 2 | 3 | 4 |

«Web Surveys Optimized for Smartphones: Are there Differences Between Computer and Smartphone Users? Ioannis Andreadis Aristotle University of ...»

-- [ Page 1 ] --

methods, data, analyses | Vol. 9(2), 2015, pp. 213-228 DOI: 10.12758/mda.2015.012

Web Surveys Optimized for Smartphones:

Are there Differences Between Computer

and Smartphone Users?

Ioannis Andreadis

Aristotle University of Thessaloniki


This paper shows that computer users and smartphone users taking part in a web survey

optimized for smartphones give responses of almost the same quality. Combining a design

of one question in each page and innovative page navigation methods, we can get high quality data by both computer and smartphone users. The two groups of users are also compared with regard to their precisely measured item response times. The analysis shows that using a smartphone instead of a computer increases about 20% the geometric mean of item response times. The data analyzed in this paper were collected by a smartphonefriendly web survey. All question texts are short and the response buttons are large and easy to use. As a result, there are no significant interactions between smartphone use and either the length of the question or the age of the respondent. Thus, the longer response times among smartphone users should be explored in other causes, such as the likelihood of smartphone users being distracted by their environment.

Keywords: web surveys, mobile surveys, AJAX navigation, data quality, item response times, smartphones © The Author(s) 2015. This is an Open Access article distributed under the terms of the Creative Commons Attribution 3.0 License. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

214 methods, data, analyses | Vol. 9(2), 2015, pp. 213-228 1 Introduction The aim of this paper is to study the differences between computer and smartphone users when they complete web surveys optimized for smartphones. The comparison is done on two dimensions. The first dimension refers to the quality of the responses, e.g. the frequencies of no answers or neutral responses. The second dimension refers to the time the respondents spend to answer the questions, i.e. the item response times.

The most recent studies on the effects of mobile use on data quality report limited differences between mobile and computer respondents. Mavletova (2013), analyzing an experiment in Russia, reports that computer respondents type longer responses on open-ended questions. One the other hand, she finds that mobile and computer users have similar levels of socially undesirable and non-substantive responses. In addition, the two groups do not differ significantly in terms of primacy effects. De Bruijne and Wijnant (2013) after running an experiment with participants randomly assigned to three modes (mobile, computer and a hybrid) have not found any significant differences. Toepoel and Lugtig (2014) have not found differences between mobile and desktop users with regard to item nonresponse, the length of answers to a short open-ended question and the number of responses in a check-all-that-apply question. Finally, Wells, Bailey, and Link (2014) have randomly assigned roughly 1,500 online U.S. panelists and smartphone users to either a mobile application or a computer. They have not found any significant responseorder effects across modes. However, they report that computer respondents provide significantly longer responses than mobile respondents.

Many web survey researchers have reported that the number of people who use mobile devices to take part in web surveys is increasing rapidly. In addition, the time spent on a web survey is crucial for the quality of the collected data. Longer web surveys suffer from larger break-off rates and greater probability of lower quality responses. Therefore, many recent publications deal with the time spent on responding to web surveys while using mobile devices. Both Mavletova (2013) and De Bruijne and Wijnant (2013) report that mobile device users need more time to complete the questionnaire than computer users. Conversely, Toepoel and Lugtig (2014) find that total response times are almost the same across devices.

Direct correspondence to Ioannis Andreadis, Laboratory of Applied Political Research, Department of Political Sciences, Aristotle University Thessaloniki, 46 Egnatia St., Thessaloniki, 54625 Hellas (Greece) E-mail: john@polsci.auth.gr Andreadis: Web Surveys Optimized for Smartphones 215 2 Designing for Smartphone Users Previous studies on measurement effects have found minimal differences between mobile and computer respondents. The most challenging difference concerns the length of open-ended responses. Nowadays, people become more and more experienced in typing texts using the small keys of their smartphones. Nevertheless, it is still much easier to type using a regular full-size keyboard than using a keypad on a mobile device. As a result, we should continue to expect longer responses on open-ended questions by computer users, especially when the response needs more than 3-4 words.

A good web survey design can remove most of the remaining differences. Survey design always plays an important role. According to Stapleton (2013), horizontal orientation of response choices may increase satisficing by smartphone users, i.e. they are more likely to select one of the first response choices. Vertical scrolling seems to be better than horizontal scrolling. In fact, Mavletova and Couper (2014) argue in favor of using a vertical scrolling design and they report that it leads to significantly faster completion times, and fewer technical problems. As they argue, the smaller number of interactions with the server reduces the risk of dropped connections. On the contrary, Wells, Bailey, and Link (2014) argue in favor of minimal vertical scrolling and support the idea of using one question per page, short questions and short sets of response lists.

A solution that gets the best from both worlds is the use of Asynchronous JavaScript and XML (AJAX) technology. By using AJAX, survey designers can display one question per page while minimizing the risk of failed connections with the server. This is achieved by downloading all pages to the users’ browser during the first connection with the server. Then, AJAX takes care of the navigation from page to page. In that way, there is only a second and final connection with the server when the user submits the completed questionnaire. Furthermore, with AJAX technology we avoid any lags between pages1. This means that we can have accurate measurement of the time spent between clicks.

3 Data The findings presented in this paper are based on the analysis of the paradata collected in May 2014 by the Greek Voting Advice Application (VAA) HelpMeVote VoteMatch Greece (Andreadis, 2013). Voting advice applications are special types of opt-in web surveys that help users find their proximities with the political parties.

1 Couper and Peterson (2015) refer to two kinds of times: between-page (transmission) time and within-page (response) time. Using the AJAX navigation system the former time is zero.

216 methods, data, analyses | Vol. 9(2), 2015, pp. 213-228 These applications can attract thousands or even millions of users during the preelectoral period. HelpMeVote is the Greek partner in the multi-national European project VoteMatch (votematch.eu). The target of this project is to run VAAs for the European Parliament elections.

HelpMeVote follows the best practices used in both web and mobile survey design. It runs both on computers and on smartphones. It automatically scales to any screen size and it supports both touch and mouse events. It displays one question per page and supports AJAX navigation. It uses large font size, short texts and the response options are displayed vertically with large buttons.

The questionnaire includes 31 Likert type questions. Each question is displayed on a separate page. Respondents have six answer choices: there are five buttons to express their level of agreement with a statement and a “No answer” button. When a respondent clicks on a button, the timestamp is recorded in a hidden input field and the user is forwarded to the next page. Besides the 31 main questions, HelpMeVote users are asked to fill-in a form. This form includes questions about their gender, age group, education level, and voting behavior. Finally, HelpMeVote captures the user-agent header field, which enables the detection of the users’ browser and device type (i.e. smartphone, computer, etc). When the respondent submits the survey, everything is stored to a database. Thus, each database record includes the user responses, the timestamps and the device type.

The HelpMeVote/VoteMatch Greece dataset includes about 80,000 completed questionnaires. The largest part of the dataset consists of computer users (80.7%) and smartphone users (13.5%). The rest of the respondents have used other mobile devices (mostly tablets). The focus of this paper is on the comparison between smartphone and computer users when both groups use a smartphone-friendly web survey. Therefore, users of other mobile devices were not included in the analysis.

4 Methods and Variables

4.1 Quality of Responses HelpMeVote does not include any open-ended items. Thus, the hypothesis that computer respondents provide longer responses cannot be tested. One the other hand, computer and smartphone users of HelpMeVote can be compared for other data quality patterns.2 For instance, if smartphone users selected more non-substantive responses (i.e. “Neither agree nor disagree” or “No answer”) than computer users, this would suggest that smartphone users provide data of lower quality. Similarly 2 For a list of mode effects related to data quality see Bethlehem and Biffignandi, 2011, p.245 Andreadis: Web Surveys Optimized for Smartphones 217 smartphone users can be tested for primacy effects (i.e. selecting the first response choice more often) or any other response-order effects.

When a chi-square test is applied on a large sample, it will almost always give a small p-value. Even when there is no practical difference between expected and observed frequencies, the test will reject the hypothesis of independence. In addition, running a separate test for each of the 31 items included in HelpMeVote would result in multiple comparisons and incorrect rejection of the null hypothesis. Thus, it would be more likely to classify nonsignificant differences as significant.

The aforementioned problems are avoided by creating six new variables. The value of each new variable reflects the number of times the respondent has chosen the corresponding response option (“Frequency of Strongly Disagree” to “Frequency of Strongly Agree” and “Frequency of No Answer”). The range of values of these new variables is from 0 to 31. Each of these variables takes the minimum value (0) when the respondent does not select the corresponding answer in any of the 31 questions. Similarly, it takes the maximum value (31) when the respondent selects the same answer for all questions. With these variables it is easy to analyze mode effects between mobile and computer users. For instance, a comparison of the average values of the variable “Frequency of Strongly Disagree” between mobile and computer users will show if there is a different primacy effect between modes.

Similarly, a comparison of the average values of the variables: “Frequency of Neither agree nor disagree” and “Frequency of No Answer” between the two groups will reveal if smartphone users select non-substantive responses more often than computer users.

5 Item Response Times The analysis of item responses times is much more complicated for two reasons.

First, item response times depend on characteristics of both the respondents and the items. As a result, there is a need for a multi-level analysis of the item response times. Second, there is need for data cleaning in order to deal with extremely short or extremely long item response times.

Pages:   || 2 | 3 | 4 |

Similar works:

«LÄUFER&GUTENBERG GmbH | Läuferweg 1 · D-31303 Burgdorf · Fon: (0 51 36) 80 01-0 · Fax: -140 e-Mail: info@laeufer-gutenberg.de · www.laeufer-gutenberg.de INHALT | CONTENT | SOMMAIRE 03 08 | Radierer · Erasers · Gommes 09 19 | Schreibunterlagen · Desk mats · Sous-main 20 26 | Läufer Büroartikel · Office products · Matériel de bureau 27 34 | Gutenberg Büroartikel · Office products · Matériel de bureau 35 41 | Haushalt/Schule · Household/School · La maison/École 42 | Gutenberg...»

«Rochex, Jean-Yves Échec scolaire et démocratisation: enjeux, réalités, concepts, problématiques et résultats de recherche Schweizerische Zeitschrift für Bildungswissenschaften 23 (2001) 2, S. 339-356 urn:nbn:de:0111-opus-37696 in Kooperation mit / in cooperation with: http://www.rsse.ch/index.html Nutzungsbedingungen / conditions of use Gewährt wird ein nicht exklusives, nicht übertragbares, persönliches und beschränktes Recht auf Nutzung dieses Dokuments. Dieses Dokument ist...»

«Aufnahme von Fettsäuren in Spermatozoenlipide von Sus scrofa domestica und physiologische Auswirkungen Dissertation zur Erlangung des akademischen Grades doctor rerum naturalium (Dr. rer. nat) im Fach Biophysik eingereicht an der Mathematisch-Naturwissenschaftlichen Fakultät I der Humboldt-Universität zu Berlin von Dipl.-Biol. Valentin Svetlichnyy Präsident der Humboldt-Universität zu Berlin Prof. Dr. J.-H. Olbertz Dekan der Mathematisch-Naturwissenschaftlichen Fakultät I Prof. Dr. A....»

«Gisbert Keseling Schreibstörungen Gisbert Keseling Marburg Following Harris (1985) and Rose (1984) I assume that a writer's block can be described in terms of the underlying cognitive processes. The subject of this paper are two kinds of difficulties, namely in elaborating the macrostructure of a planned text and in producing formulations fluently. Students often confound these two processes. They overlook the fact that macrostructures of written texts consist of ideas and formulations...»

«L’apprentissage autonome dans l’enseignement du français langue étrangère Théorie et pratique Thèse présentée à la Faculté des Lettres de l’Université de Zurich pour l’obtention du grade de docteur par Eva Lang de Kreuzlingen / Thurgovie Thèse acceptée par la Faculté au semestre du printemps 2008 sur la proposition de M. Prof. Dr. Jakob Theodor Wüest et de M. Prof. Dr. Urs Ruf Zurich, 2010 Remerciements Je remercie M. Dr. Prof. Jakob Theodor Wüest pour la possibilité...»

«NRC User Guide R330733 K5903036/00 05/06/2015 Product revision Software version: 1.0 Barco NV President Kennedypark 35, 8500 Kortrijk, Belgium Phone: +32 Fax: +32 Support: www.barco.com/en/support Visit us at the web: www.barco.com Printed in Belgium Table of contents TABLE OF CONTENTS 1. Welcome! 1.1 About the product............................................... 3 2. Getting started 2.1 Installing the NRC Client...»

«accessories2013 Sommario Summary Summaire Inhaltsangabe Sumario 目次 Superbike.............................. 4 Diavel.............................. 60 Multistrada........................... 92 Streetfighter........................... 130 Monster.............................. 156 Hypermotard......................... 188...»

«Fachhochschule Hof Fachbereich Informatik / Technik Studiengang Werkstoffund Oberflächentechnik Diplomarbeit zur Erlangung des Akademischen Grades Diplom-Ingenieur (FH) Thema: Innendruckmessung an der MegalixTM Drehanodenröntgenröhre Vorgelegt bei: Vorgelegt von: Prof. Dr. Jörg Krumeich Stefan Lang Alfons-Goppel-Platz 1 Kleukheimer Hauptstraße 35 95028 Hof 96250 Ebensfeld Kleukheim, 16.08.2007 Um zwei Uhr morgens kam das Land in Sicht. Christopher Columbus meiner Familie II...»

«PASSAGE THE LITERARY MAGAZINE OF THE ACADEMIC ENRICHMENT & FIRST-YEAR EXPERIENCE DIVISION NORWALK COMMUNITY COLLEGE SPRING 2014, vol. 8 PASSAGE SPRING 2014 Editor Dr. Forrest C. Helvie, Assistant Professor Academic Enrichment and First-Year Experience Division Advisor Elaine Delvecchio, Director Academic Enrichment and First-Year Experience Division Front cover based upon original photograph of the Alaskan Inside Passage, Copyright Graphic Maps. Amateurs sit and wait for inspiration, the rest...»

«International Assistance System Concept of Operations October 1, 2010 This document has been prepared by an interagency working group under the direction of the Department of Homeland Security (DHS), Department of State (DOS) and the United States Agency for International Development (USAID). As a result of Hurricane Katrina, the United States Government (USG) established policies and procedures to manage the flow of international resources into the United States under the National Response...»

<<  HOME   |    CONTACTS
2016 www.book.dislib.info - Free e-library - Books, dissertations, abstract

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.