Computers in Human Behavior 39 (2014) 128–135

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier .com/locate /comphumbeh

A comparison of difficulties in instructional design processes:Mobile vs. desktop

http://dx.doi.org/10.1016/j.chb.2014.07.0080747-5632/� 2014 Published by Elsevier Ltd.

⇑ Corresponding author. Tel.: +90 444 50 65/1787.E-mail addresses: cetin@yyu.edu.tr (Ç. Güler), eylem@yyu.edu.tr (E. Kılıç),

hcavus@yyu.edu.tr (H. Çavus�).

Çetin Güler ⇑, Eylem Kılıç, Hayati Çavus�Education Faculty, Yuzuncu Yil University, 65080 Van, Turkey

a r t i c l e i n f o a b s t r a c t

Article history:

Keywords:Instructional design difficultiesMobile designMultimedia design

The aim of this study is to compare the difficulties that novice instructional designers experience duringinstructional design processes for mobile devices and desktop computers. The participants/instructionaldesigners of this study include 68 sophomore students from a Computer Education and InstructionalTechnologies (CEIT) department. The participants developed learning content for mobile devices anddesktop computers through the ADDIE model. A descriptive research method was used for the presentstudy. An already developed scale in line with the ADDIE model was used to collect the data. Descriptivestatistics, Mann Whitney U and Kruskal Wallis tests were conducted to analyze the data. The results ofthis study indicated that the difficulties experienced in both processes tended to be similar in developinglearning content. Difficulties in internal design and production and front-end analysis were significantlydifferent in terms of the Internet connection in personal mobile devices. External design and develop-ment difficulties, rolling-out difficulties and total scores were significantly different with regard to levelsof Internet experience.

� 2014 Published by Elsevier Ltd.

1. Introduction

The ownership and use of mobile devices have increased rapidlyin the past several years (Gedik, Hanci-Karademirci, Kursun, &Cagiltay, 2012; Li & Leina, 2012; Mohammad, Mamat, & Isa,2012). This rapid increase can be explained by some of the featuresof mobile devices. Mobile devices facilitate interaction, gaming andaccess to social networks via wireless technologies, and their costsare relatively low (Hashemi, Azizinezhad, Najafi, & Nesari, 2011;Looi et al., 2009). Mobile technologies can provide opportunitiesfor the education field in terms of continuity, expansion and deep-ening the learning process (Bidaki, Sanati, & Ghannad, 2013; Idrus& Ismail, 2010; Sharples, 2000) by supporting interaction andcooperation and facilitating learning activities (Alvarez, Alarcon,& Nussbaum, 2011; Bidaki et al., 2013; Cavus & Uzunboylu,2009; Churchill & Hedberg, 2008; Idrus & Ismail, 2010; Martinet al., 2011; Nordin, Embi, & Yunus, 2010; Sharples, 2000).

The requirements for learning and the capabilities of mobiletechnologies have changed. Currently, cooperative, individual andsituational learning are demanded at all times. Changes in mobiletechnologies can address these demands (Sharples, 2000). Mobile

devices that contain learning content can support effective learn-ing by providing any type of information when and where it isneeded (Clay, 2011; Martin & Ertzberger, 2013). Naturally, learningcontent that is suitable for mobile devices can provide such learn-ing. On the other hand, instructional design principles that areaccepted and applied in multimedia content development for desk-top computers may not be suitable for the content development ofmobile devices. Studies on these subjects are suggested to be car-ried out by researchers to fill in the gap in the related literature(Sung & Mayer, 2013).

In this study, the difficulties that can be faced during the devel-opment processes of learning contents for mobile devices anddesktop computers were compared. The learning content thatwas developed in different processes was multimedia content. Inthe study, the content that was developed for mobile deviceswas called mobile learning content, and the content that wasdeveloped for desktop computers was called multimedia learningcontent. A detailed explanation is given in Section 4.

2. Mobile and multimedia learning

Mobile learning is considered as a part of e-learning in somestudies, despite the fact that it has its own applications, featuresand terminology (Korucu & Alkan, 2011). Mobile learning can bedefined as learning and teaching activities that are facilitated,

Ç. Güler et al. / Computers in Human Behavior 39 (2014) 128–135 129

supported and diffused with the help of portable devices, wirelessand cell phone networks (Hashemi et al., 2011; Mohammad et al.,2012). In addition, mobile learning can be defined as learning inwhich the learner and teacher are able to access the education sys-tem by portable devices and wireless networks (Kwon & Lee, 2010;Sandberg, Maris, & de Geus, 2011).

Mobile learning can be considered as a relatively new learningapproach. The place that mobile learning would have in the currentlearning systems is debatable. Mobile learning can be consideredas the most important item of distance education, whereas in theevaluation of e-learning and mobile learning together, learningand interaction opportunities are given without the limitation oftime and place (Korucu & Alkan, 2011). In addition, mobile learningand web-based learning can be considered to be the most impor-tant items of e-learning (Triantafillou, Georgiadou, & Economides,2008), and it is stated that studies on mobile learning haveincreased in both quantity and quality (Bidaki et al., 2013; Cavus& Uzunboylu, 2009; Nordin et al., 2010).

Bidaki et al. (2013) carried out a study which they preparede-books for mobile phones and measured learners’ attitudes. Inthe study a book maker was developed and content for fourcourses was prepared. The contents were converted to e-booksfor mobile phones and the learners (n = 158) were asked to down-load the e-books for their mobile phones. The learners were free todownload or not. Most of the learners (n = 106) downloaded atleast one e-book and used, 52 learners did not. The researchersapplied pre-test and posttest (20 questions) for attitudes of thelearners. The results of the study showed that e-books for mobilephones were highly welcomed by the learners and the learnerswho downloaded and used these e-books in their dead times orin motion are more motivated than others. Another point that isworth to mention from the research is that the researchers had dif-ficulty in developing the e-book maker that can work with vastmajority (not all) of mobile phones because of their diversity.

Clay (2011) carried out a study that examines usability ofmobile devices for developing clinical skills in the clinical arena(post graduate). The participant of the study was a small groupwho enrolled in a newborn infant physical examination module.The participants were given mobile devices which related contentsinstalled on and they used these devices for 12 weeks. At the end ofthe process they were invited to complete an evaluation tool aboutthe use of mobile devices and join a focus group interview. Theevaluation was consisted of technical use, educational applicationand compatibility of personal learning styles. The results showedthat the participants found mobile devices easy to use and totransport and addressed the need for ‘‘just in time’’ knowledge.

Learning is an individual process which can happen anywhereand anytime regardless of technologies/devices existed or not. Onthe other hand, learners can benefit from using some technologies/devices to use learning materials. Desktop computers can giveopportunity to use multimedia learning materials containing ani-mations, visuals, and sounds, etc. and facilitate learning. Mobiledevices can give this opportunity as well. Through multimedialearning, learners deepen their learning by animating content withtexts and visuals in their minds (Mayer, 2003), whereas mobilelearning provides these opportunities as well. On the other hand, amultimedia content that is developed for desktop computers maynot be suitable for mobile devices because mobile devices have dif-ferent hardware and software capabilities than desktop computers.In this study mobile learning refers to a learning process whichincludes using smart phones for accessing and using learning mate-rials whereas multimedia learning refers to a learning process whichincludes using desktop computers for accessing and using learningmaterials. Difference between mobile and multimedia learningcomes from their hardware and software capabilities/differences.

3. Mobile and multimedia design

Designing and developing learning materials that are suitablefor mobile devices may present some difficulties to researchersand educators (Alvarez et al., 2011). As the amount of mobile learn-ing content increases, developing such content is becoming animportant research field (Bidaki et al., 2013).

The requirements of developing mobile and multimedia contentmay show some differences (Kim & Albers, 2001). Not all mobiledevices have the same hardware and software features (Kim &Albers, 2001); therefore, instructional design for mobile contentrequires specific adjustments (Nordin et al., 2010; Ozdamli,2012). The software development environments that are used todevelop mobile learning content are a part of these adjustments.Bidaki et al. (2013) expressed that although they used the Javaenvironment to develop multimedia content, they needed somesort of software to make the content more suitable for differenttypes of mobile devices.

Kim and Albers (2001) carried out a study with 28 nativeEnglish speakers and they examined access and use of informationfrom a mobile device and a desktop computer. The context of thestudy contained accessing web sites of different lengths from dif-ferent devices and using rich sets of information. The results ofthe study showed that there was not a significant difference inamount of errors while searching for information but the searchingtook more time with a mobile device than a desktop computer.Finding information placed in the middle of the text showed signif-icant difference in point of spent time for text of 225–350 words infavor of desktop computers.

Churchill and Hedberg (2008) tried to use learning objects thatwere developed for desktop computers, in mobile devices and theysuggested several features that mobile content should have.According to these authors, mobile content should (a) work on afull screen, (b) use a screen horizontally, (c) require little to noscrolling, (d) require minimal and task-based touch, (e) provideinteraction immediately, (f) have zooming properties, (h) providethe opportunity to use a stylus.

Mobile devices have limited hardware and software featurescompared to desktop computers, which require some specificadjustments in the process of mobile content development. Themost specific limitation of mobile devices stated in the literatureis their small screens (Huang, Kuo, Lin, & Cheng, 2008; Luchini,Quintana, & Soloway, 2004). Using content that is developed forlarge screens does not usually cause any problems with smallscreens, where in the opposite situation, problems or difficultiesare more likely to arise. Assuming that mobile devices are comput-ers that have small screens and designing and developing mobilecontent according to this assumption is not efficient (Kim &Albers, 2001). Studies that are related to mobile content mostlyconsider technical issues such as compression formats, scrolling,resizing images for the screen size of mobile devices where infor-mation and learning issues are left behind (Churchill & Hedberg,2008; Kim & Albers, 2001). Mobile devices are not user friendlyin terms of typing or entering inputs. Unlike desktop computers,software for mobile devices cannot process most formats. Thedefault web browsers of most mobile devices are not functionalenough (Huang et al., 2008). In addition, most suggestions fordesigning and developing content for small screens are about soft-ware development, not content itself (Kim & Albers, 2001). Briefly,mobile devices have many limitations that should be taken intoaccount while developing contents. In the present study, the limi-tations related to content and presentation into considerationwhile there is not much can be done about technical limitations.These limitations (1, 2, 3. . .) and how the current study tackledwith them (a, b, c. . .) were presented as follows:

130 Ç. Güler et al. / Computers in Human Behavior 39 (2014) 128–135

1. Small screen (Huang et al., 2008; Luchini et al., 2004)a. Dividing content into small meaningful chunksb. Designing interface of the contents for most commonscreen size and resolutionc. Using bigger buttons, fonts, etc.

2. Low memory capacity (Hashemi et al., 2011)3. Battery life (Huang et al., 2008)4. Common operating system (Hashemi et al., 2011)

a. Developing Flash or Java based contentsb. Not using any special file type that is usable only in someoperating systems

5. Common hardware features (Hashemi et al., 2011)a. Taking into consideration the most common hardwarespecifications (CPU, memory, graphic) for mobile devicesand developing the contents that require minimumhardware specifications

6. Less durability (Hashemi et al., 2011)7. Difficulty of using graphics (Hashemi et al., 2011)

a. Taking into consideration the most common hardwarespecifications (CPU, memory, graphic) for mobile devicesand developing the contents that require minimumhardware specifications

8. Low interoperability with other devices (Hashemi et al.,2011)

9. Lack of updates (Hashemi et al., 2011)10. Quality and cost issues of a wireless Internet connection

(Hashemi et al., 2011)11. Convenient for distraction (Kwon & Lee, 2010)

Some models and phases were used in some studies for themobile learning content development process. Furió, González-Gancedo, Juan, Seguí, and Rando (2013) carried out a study onlearning and games in which they developed a game for mobiledevices and used a system development life cycle that consists ofsix phases: 1 – project planning and selection, 2 – system analysis,3 – system design, 4 – system implementation, 5 – testing and6 – evaluation. Nordin et al. (2010) used a framework for mobilelearning design including four elements: 1 – requirement and con-straint analysis, 2 – mobile learning scenarios, 3 – technology envi-ronment design and 4 – learner support services. Kwon and Lee(2010) conducted an analysis of a cycle of: 1 – designing, 2 –developing and 3 – revising to develop mobile learning prototypes.Huang (2005) suggests five phases for interactive multimediadesign: 1 – designing for users’ needs and expectations, 2 –planning, 3 – processing, 4 – testing and 5 – improving. Por,Mustafa, Osman, Phoon, and Fong (2012) used a three-phaseAllessi and Trollip Instructional System Design model – Planning,Design, Development – for multimedia content development. Themodels and the phases that are used in these studies have somesimilarities with the ADDIE model, which implies that the ADDIEmodel can be applied to develop mobile and multimedia content.

There are some other suggestions for developing mobile con-tent in addition to the aforementioned models and phases.Hamat, Embi, and Hassan (2012) stated that mobile learning ande-learning come from the same foundations; therefore, e-learningdesign experiences can be reflected to mobile learning design.Kwon and Lee (2010) suggested that the information processingmodel of multimedia (Mayer, 2003) can be applied to mobile con-tent development. Sung and Mayer (2013) suggested that in theprocess of developing mobile learning content, the content shouldbe divided into small pieces, and emphasis should be applied to thepresentation of the content.

Sung and Mayer (2013) expressed that there is no study thatinvestigates whether the principles of learning content develop-ment for desktop computers can be applied to developing learning

content for mobile devices or not. There are some studies(Churchill & Hedberg, 2008; Hwang & Chang, 2011; Kim &Albers, 2001; Nordin et al., 2010; Ozdamli, 2012; Sun & Cheng,2007) that imply that multimedia content development is differentfrom mobile content development, and there should be studies onmobile content development. In addition, most studies on mobilelearning are about the usefulness of mobile technologies in educa-tion. The number of studies on developing mobile learning contentis very limited (Churchill & Hedberg, 2008; Kim & Albers, 2001;Martin et al., 2011).

Özdamli (2011) draws attention to the importance of includingstudents in instructional design processes as instructional design-ers. She tries to find out the answer for the question ‘‘How oftendid teacher candidates benefit from Preparation stage, Organiza-tion stage, Media-use stage, Navigation Tools-use stage, andAppearance Design and Resource-use stages of Project BasedLearning when they had developed instructional multimedia mate-rials?’’, in her study. She carries out the study with 200 candidateteachers as multimedia instructional material developers workingin groups. The results of the study show that the candidateteachers usually benefit from the stages and importance of collab-oration, interaction in team works are emphasized as well.

In the present study, sophomore students from a CEIT depart-ment who took courses such as ‘‘Instructional Design’’ and ‘‘Soft-ware Development’’ were included as instructional designers. Acomparison of multimedia and mobile learning content develop-ment processes including students as instructional designers inthese processes were realized in the study. In considering the liter-ature on this subject, these two dimensions contribute to theimportance of this study. The main purpose of this study is not onlyto compare the difficulties in the processes but also to find out theeffects of participants’ profile of technology use on the difficultiesexperienced in these processes. Learner’s use of technology isrelated to intellectual cognitive processing and performance inschools (Turner & Croucher, 2013). To put in another way, it canbe said that the use of technology engage learners in deep cognitiveprocess by performing multitasks and it affects the intellectual cog-nitive processing and performance. Therefore, it can be assumedthat the participants’ use of technology may affect the difficultiesexperienced in both development processes.

The main research question was ‘‘To what extent difficulties inthe multimedia learning content development process and themobile learning content development process for instructionaldesigners differ from each other?’’ In addition, the followinghypothesis are guided the current study. There is a significant rela-tionship between difficulties experienced in the processes and

H1. Instructional designers’ computer experience?

H2. Instructional designers’ Internet experience?

H3. Instructional designers’ ownership of mobile devices?

H4. Instructional designers’ having an educational software/appli-cation on their mobile devices?

H5. Instructional designers’ use of their mobile devices for educa-tional purposes?

H6. Instructional designers’ use of the Internet from their mobiledevices?

4. Method

4.1. Participants

A descriptive research method was conducted in this study.Sixty-eight sophomore students from a CEIT Department in a

Ç. Güler et al. / Computers in Human Behavior 39 (2014) 128–135 131

university from Turkey were the participants of the study. The studygroup consisted of 37 (54%) male and 31 (46%) female students. Fourstudents (6%) did not have their own computers, three students (4%)had desktop computers, 55 students (81%) had Notebooks and sixstudents (9%) had Netbooks. The participants were registered inMultimedia Design and Development (MDD) and Mobile Learning(ML) courses, which they attended during the study.

4.2. Context and procedure

The processes compared in the study were carried out as themain parts of two courses for 14 weeks. These courses were‘‘MDD’’ and ‘‘ML.’’ The MDD course has two hours of theoreticallecture and two hours of practice per week and lasted 14 weeksduring semester. The students who took the course were requiredto form project groups of three to five people to develop instruc-tional software for desktop computers. The subject of the instruc-tional software was chosen from primary, secondary or high schoolcurriculums. For four weeks of the course, the generative theory ofmultimedia and the principles of multimedia design wereinstructed. During this time, Macromedia Flash software develop-ment environment training was carried out in laboratories as well.The students were asked to use the ADDIE model, with which theywere already familiar from previous courses such as InstructionalDesign, for their projects. During the process of developing multi-media learning content, the students delivered reports for allphases of the ADDIE model. After each delivery, the students weregiven feedback by the instructor of the course. The reports and pro-jects were revised by the project groups according to this feedback.There were 15 projects and 15 multimedia learning contentsdeveloped for desktop computers at the end of the course.

Similar to the MDD course, ML has two hours of theoreticallecture and two hours of practice per a week and lasted 14 weeksduring the semester. ‘‘Mobile learning,’’ ‘‘distance education,e-learning and mobile learning,’’ ‘‘mobile technologies,’’ ‘‘mobiledevices,’’ ‘‘the advantages and disadvantages of mobile devices,’’‘‘the advantages and disadvantages of mobile learning,’’ ‘‘the accessto and distribution of mobile learning,’’ ‘‘teachers and students inmobile learning,’’ ‘‘designing mobile learning,’’ ‘‘the design anddevelopment of mobile learning content,’’ ‘‘the evaluation of mobilelearning,’’ and ‘‘development tools for mobile content development’’were included as topics in the syllabus of the course. The course wascarried out in parallel to the MDD course. The same students formedthe same project groups and developed 15 mobile learning contents.In the development process, the students had to use the ADDIEmodel phases according to the principles of mobile learning contentdevelopment on which they were instructed in the course. The stu-dents prepared and delivered reports, obtained feedback and revisedtheir reports and projects as they did in the MDD course.

In the MDD course, the participants were trained to use theMacromedia Flash environment to develop multimedia contents.As a result, all of the project groups used the Macromedia Flashenvironment to develop multimedia contents for desktop comput-ers. However, in the ML course, the participants were instructed onhow to use the ADDIE model and different types of software devel-opment environments with regard to mobile content developmentissues. The students were given the opportunity to develop mobilecontent with the software development environment of theirchoice. Most of the project groups (n = 11) used Macromedia Flash,and some of them (n = 4) used the Eclipse environment to developmobile content.

4.3. Data collection

A scale developed by Boot, van Merriënboer, and Theunissen(2008) was adapted for the present study. The instrument was

translated from English to Turkish by two experts with PhDs inEducational Sciences. Then, an English teacher translated theinstrument from Turkish to English. Minor changes were made,and the scale was rearranged for the current study. The scale devel-oped by Boot et al. (2008) notes the difficulties of the ADDIE modelwhile developing learning content for four factors with 35 items. Inthe study, the ADDIE model phases were used in the multimediaand mobile learning content development processes. As mentionedin Section 3, the multimedia and mobile learning content develop-ment processes were different from each other; hence, the items ofthe scale were rewritten to make them comparable between thetwo processes. For example, an item that indicated ‘‘the difficultiesin the analysis phase. . .’’ was converted to, ‘‘the difficulties in theanalysis phase of developing multimedia learning content are sim-ilar to the difficulties in the analysis phase of developing mobilelearning content. . .’’ The project groups developed learning contentfor desktop computers in the MDD course. The same groups devel-oped learning contents for smartphones in the ML course. To makethe items of the scale clear and distinct for members of the groups,the terms ‘‘multimedia’’ and ‘‘mobile learning’’ were used insteadof the terms ‘‘desktop computers’’ and ‘‘smartphones.’’

The data collection instrument was composed of two parts. Inthe first part, there were 10 questions to collect demographic dataabout the study group. The second part of the instrument was theadapted scale that consists of four factors with 35 items to com-pare the difficulties in development processes of multimedia andmobile learning content. The scale was built on four factors: 1.Internal design and development difficulties (12 items), 2. Roll-ing-out difficulties (11 items), 3. External design and developmentdifficulties (eight items) and 4. Front-end analysis difficulties (fouritems). For each item, the similarity of a particular difficulty wasindicated on a scale from 0 (zero) to 10 (‘‘0’’ being that difficultieswere ‘‘completely different’’ and ‘‘10’’ being that difficulties were‘‘completely similar’’).

4.4. Data analysis

Basic descriptive statistics (percentage, frequency, mean andstandard deviation) were used to analyze the data. To determinethe internal reliability of the scale, Cronbach’s alpha coefficientswere calculated. The gathered data did not show a normal distribu-tion, so the evaluations of the differences in responses were testedwith a Mann Whitney U and Kruskal Wallis tests.

5. Results and discussion

5.1. Descriptive results and discussion

5.1.1. The participants’ computer and Internet experienceThe participants responded to the question about their com-

puter experience as follows: 36 participants (53%) indicated anintermediate level of experience, 26 participants (38%) indicatedan advanced level and six (9%) indicated an expert level. Thirty-two participants (47%) stated that they had intermediate Internetexperience, 31 participants (46%) stated that they had advancedexperience, and five participants (7%) stated they had expertexperience. Approximately three of four participants (n = 48, 71%)had an Internet connection in their place of residence, and approx-imately one of four (n = 20, 29%) did not have an Internetconnection.

5.1.2. The participants’ use of mobile devicesMost of the participants (n = 51, 75%) stated that they had used

smartphones, and the rest of them (n = 17, 25%) had not. The par-ticipants who had used smartphones indicated their smartphones’

Table 1Descriptive statistics of the adapted scale for differences between MDD and ML.

Factors Items Mean Std.deviation

Cronbach’sa

Internal design anddevelopment difficulties(IDDD)

IDDD1 5.55 1.86 0.91IDDD2 5.58 2.02IDDD3 5.67 1.88IDDD4 5.48 1.79IDDD5 6.01 2.09IDDD6 5.88 2.10IDDD7 5.85 1.78IDDD8 5.79 1.90IDDD9 6.02 2.17IDDD10 5.86 1.96IDDD11 6.22 1.61IDDD12 5.67 1.86Total 5.80 1.39

Rolling-out difficulties (ROD) ROD1 5.86 1.81 0.93ROD2 5.88 2.11ROD3 5.70 1.90ROD4 5.98 2.20ROD5 5.58 1.94ROD6 5.66 2.10ROD7 5.50 1.9ROD8 5.39 1.82ROD9 5.57 1.89ROD10 5.85 1.74ROD11 5.88 1.86Total 5.71 1.57

External design anddevelopment difficulties(EDDD)

EDDD1 5.82 1.95 0.94EDDD2 5.69 1.78EDDD3 5.77 2.07EDDD4 6.01 2.06

132 Ç. Güler et al. / Computers in Human Behavior 39 (2014) 128–135

brands as follows: 32 ‘‘Samsung,’’ eight ‘‘Nokia,’’ four ‘‘SonyEricsson,’’ three ‘‘LG,’’ two ‘‘HTC,’’ one ‘‘Apple’’ and one ‘‘GeneralMobile.’’ The operating systems of their smartphones were: 22Android, two ‘‘Symbian,’’ one ‘‘iOS,’’ one ‘‘Wave,’’ one ‘‘Meego’’and five were not known. When the participants were askedwhether their smartphones could play Flash or not, 25 participants(49%) stated that their smartphones could play Flash, and 26 par-ticipants (51%) stated otherwise. In addition, 35 participants(69%) stated that their smartphones could run Java applications,whereas 16 participants (31%) stated otherwise. Almost all of theparticipants (n = 48, 94%) who had smartphones had Internetaccess on their smartphones, and a few (n = 3, 6%) did not haveInternet access on their smartphones. More than half of the partic-ipants (n = 31, 60%) who had smartphones had at least one educa-tional software/application installed on their phones, and 20 ofthem (40%) did not have any. Six participants (19.4%) who had edu-cational software/applications on their phones stated that theyused the software/applications at least once a day, eight partici-pants (26.8%) stated that they used them a few times a day, and17 participants (54.8%) stated that they used them a few times aweek. With regard to their satisfaction with the Internet speed oftheir smartphones, 29 participants (60%) responded that they weresatisfied, and 19 participants (40%) responded that they were notsatisfied. The frequency of using Internet access on their smart-phones for an educational purpose was as follows: four (11%)responded that they never used the Internet on their smartphonesfor this purpose, eight (22.2%) responded that they used it at leastonce a day, 13 (36.1%) responded a few times in a day, and 11(30.6%) responded a few times a week.

EDDD5 5.91 1.98EDDD6 5.77 1.91EDDD7 6.00 1.80EDDD8 5.83 1.85Total 5.84 1.64

Front-end analysis difficulties(FEAD)

FEAD1 5.83 2.02 0.89FEAD2 5.97 2.15FEAD3 5.88 1.92FEAD4 6.00 1.96Total 5.92 1.76

5.1.3. Difficulties in the multimedia learning and the mobile learningcontent development processes

In response to the main question, mean and standard deviationof all items and factors were calculated. The descriptive statistics ofthem and Cronbach’s alpha coefficients of the adapted scale weregiven in Table 1.

The reliability coefficients for the sub-factors were between .89and .94. These values are larger than .70, which is the acceptablelimit (Nunnally, 1978) for Cronbach’s alpha coefficients. Cron-bach’s alpha coefficients for the scale were found to be .97. Theaverage points of each item vary between 5.39 and 6.22. The firstfactor was the internal design and development difficulties (IDDD)which were compared between the multimedia learning and themobile learning content development process. IDDD in both devel-opment processes tended to be similar (M = 5.80, SD = 1.39). In par-allel way, rolling-out difficulties (ROD) experienced in bothdevelopment processes (M = 5.71, SD = 1.57) were similar. Theinstructional designers found external design and developmentdifficulties (EDDD) and front-end analysis difficulties (FEAD) simi-lar in both development processes as well (M = 5.84, SD = 1.64;M = 5.92, SD = 1.76, respectively). All difficulties experiencedduring development of the multimedia learning and the mobilelearning content also tended to be similar (M = 5.80, SD = 1.39).

The descriptive statistics of the scale showed that the partici-pants did not exactly decide whether the difficulties that theyexperienced in both processes are similar or not. The mean valuesfor total scores and four factors of the scale vary between 5.71 and5.84 which can be interpreted as the difficulties tended to be sim-ilar. Mobile devices can be considered as mobile multimediamachines (Attewell, 2004) which can process multimedia contents(Huang et al., 2008). Also principles of instructional design fore-learning (which includes multimedia learning) can be appliedfor mobile learning instructional design (Hamat et al., 2012). How-ever, there is diversity in mobile devices’ hardware and softwareand they have relatively limited features compared to desktop

computers. This can be the reason why the participants foundthe processes relatively similar but not the same at all.

5.2. Effects of different variables on difficulties in both processes

The effects of different variables (computer experiences, Inter-net experiences, etc.) on difficulties in both processes are pre-sented in line with the research questions.

5.2.1. Computer experiences and difficulties in the processesIn response to H1, a Kruskal Wallis test was conducted to ana-

lyze the differences between the difficulties that were experiencedduring the processes with regard to the participants’ computerexperiences. The test results showed that there were no significantdifferences in terms of internal design and development difficulties(v2(2) = 1.83, p = 0.40), rolling-out difficulties (v2(2) = 4.39,p = 0.11), external design and development difficulties (v2(2) =3.07, p = 0.21), front-end analysis difficulties (v2(2) = 3.91,p = 0.14) and total scores (v2(2) = 4.56, p = 0.10). Almost all theparticipants (91%) stated their level of computer experiences asintermediate or advanced which can be inferred as the participantswere a homogeneous group in term of computer experience.Homogeneity of the group and similarity of the difficulties experi-enced in the processes can explain refusal of H1.

Table 2Internet experience and difficulties in the processes.

Variable Group N Ranks Df v2 p

External design and development difficulties Intermediate 32 38.22 2 6.44 0.04Advanced 31 33.94Expert 5 14.20

Rolling-out difficulties Intermediate 32 38.52 2 8.58 0.01Advanced 31 34.19Expert 5 10.70

Total Intermediate 32 37.78 2 8.45 0.01Advanced 31 35.03Expert 5 10.20

Table 3Having an Internet connection on their mobile devices and difficulties in the processes.

Variable Group N Ranks Total U p

Internal design and development difficulties Having 48 27.02 1297.00 23.00 0.05Not having 3 9.67 29.00

Front-end analysis difficulties Having 48 27.15 1303.00 17.00 0.02Not having 3 7.67 23.00

Ç. Güler et al. / Computers in Human Behavior 39 (2014) 128–135 133

5.2.2. Internet experience and difficulties in the processesTo test H2, a Kruskal Wallis test was conducted to examine the

differences between the difficulties that were experienced duringthe processes with regard to the participants’ Internet experience.The test results were given in Table 2.

The test results showed that there were significant differencesin terms of rolling-out difficulties (v2(2) = 8.58, p = 0.01), externaldesign and development difficulties (v2(2) = 6.44, p = 0.04) andtotal scores (v2(2) = 8.45, p = 0.01), which indicated that the partic-ipants who had an intermediate and advanced level of Internetexperience believed that the difficulties were similar, in contrastwith the participants who had expert levels of Internet experience.It can be inferred that the expert Internet users were aware of dif-ferences between the processes more than the intermediate andadvanced Internet users. Thus H2 was supported.

5.2.3. Owning a mobile device and difficulties in the processesTo test H3, a Mann Whitney U test was conducted to discover the

differences between the difficulties that were experienced duringthe processes with regard to the participants owning mobile devicesor not. The test results showed that there were no significant differ-ences in terms of internal design and development difficulties(U = 368, z = �.929, p = 0.51, r = �0.11), rolling-out difficulties(U = 392.00, z = �.588 p = 0.55, r = �0.07), external design anddevelopment difficulties (U = 381.00, z = �.745, p = 0.45,r = �0.09), front-end analysis difficulties (U = 424.00, z = -.128,p = 0.89, r = �0.01) and total scores (U = 362.00, z = �1.01, p = 0.31,r = �0.12). All of the participants had mobile devices and most of(75%) these devices were smart phones so the participants werehomogeneous in regard to this term. Thus H3 was not supported.

5.2.4. Having an educational software/application on mobile devicesand difficulties in the processes

In line with the H4, a Mann Whitney U test was conducted todiscover the differences between the difficulties that were experi-enced during the processes with regard to the participants havingan educational software/application on their mobile devices or not.The test results showed that there were no significant differencesin terms of internal design and development difficulties(U = 254.50, z = �1.15, p = 0.24, r = �0.16), rolling-out difficulties(U = 253.50, z = �1.17, p = 0.23, r = �0.16), external design anddevelopment difficulties (U = 300.00, z = �.288, p = 0.77,r = �0.03), front-end analysis difficulties (U = 302.00, z = �.24

p = 0.81, r = �0.3) and total scores (U = 275.50, z = �.756, p = 0.45,r = �0.10). Almost half of the participants (n = 31, 46%) had an edu-cational software/application in their mobile devices. These resultsimply that the experienced difficulties in design processes formobile and multimedia were not change whether the designershaving an educational software/application on their mobile devicesor not. As it was mentioned in Section 5.1.3 (Attewell, 2004; Hamatet al., 2012; Huang et al., 2008) the both processes were similar. Itappears this similarity does not change with regard to the partici-pants having an educational software/application on their mobiledevices (H4) and using their mobile devices for educational pur-poses (H5). Thus H4 were not supported.

5.2.5. Using mobile devices for educational purposes and difficulties inthe processes

To test the H5, a Kruskal Wallis test was conducted to analyzethe differences between the difficulties that were experienced dur-ing the processes with regard to the participants using their mobiledevices for educational purposes. The test results showed that therewere no significant differences in terms of internal design anddevelopment difficulties (v2(2) = 1.83, p = 0.40), rolling-out diffi-culties (v2(2) = 4.71, p = 0.09), external design and developmentdifficulties (v2(2) = 3.19, p = 0.20), front-end analysis difficulties(v2(2) = 2.39, p = 0.30) and total scores (v2(2) = 2.57, p = 0.26). Theparticipants (n = 31, 46%) who had an educational software/appli-cation on their mobile devices stated that they were using theirmobile devices for educational purposes at least a few times a week.Consistency between having an educational software/applicationon mobile devices and using the mobile devices for educationalpurposes reflected on the results. Thus H5 was not supported.

5.2.6. Having an Internet connection on mobile devices and difficultiesin the processes

In line with H6, a Mann Whitney U test was conducted to exam-ine the differences between the difficulties that were experiencedduring the processes with regard to the participants having anInternet connection on their mobile devices or not. The test resultswere given in Table 3.

The test results showed that there were significant differencesin terms of internal design and development difficulties(U = 23.00, z = �1.96, p = 0.05, r = �0.27) and front-end analysisdifficulties (U = 17.00, z = p = 0.02, r = �0.30), which indicated thatthe participants who had an Internet connection on their mobile

134 Ç. Güler et al. / Computers in Human Behavior 39 (2014) 128–135

devices believed that the difficulties were similar, in contrast withthe participants who did not have an Internet connection. It can beinferred that mobile technologies along with the Internet accessmight be very useful and effective in terms of interaction and coop-eration for project groups such as groups in this study. The projectswere executed by project groups in both processes, which requiredinteraction and cooperation among group members. In addition,there are some studies (Hashemi et al., 2011; Korucu & Alkan,2011; Kwon & Lee, 2010; Mohammad et al., 2012; Sandberget al., 2011) that emphasize the advantages of mobile technologiesand the benefits of having an Internet connection.

6. Conclusions

In this study, the difficulties that are experienced in the develop-ment processes of learning content for desktop computers andmobile devices were compared. The results can be interpreted thatthe difficulties in both processes tend to be similar. This interpreta-tion does not contradict previous studies in related literature(Bidaki et al., 2013; Huang e al., 2008; Kim & Albers, 2001; Luchiniet al., 2004; Nordin et al., 2010; Ozdamli, 2012). Although mobilecontent is technically multimedia content, the principles of devel-oping multimedia content may not fit in terms of developing mobilecontent, and mobile devices possess different hardware and soft-ware. In other words, both development processes have some sim-ilarities, where the development process of mobile content requiressome specific adjustments. On the other hand, it is strongly recom-mended that the development processes of mobile content shouldbe designed and proceed with regard to the special requirements(such as screen size, orientation, Internet quota, bandwidth, sup-ported platforms, and battery consumption), advantages and disad-vantages of mobile learning and mobile technologies.

Learners/students use mobile technologies and improve theirability to use them. With regard to this situation, educators shouldbe proficient in using mobile technologies and how to adapt anduse them in a learning process to make it more efficient (Hamatet al., 2012). Mobile learning technologies can be more successfulthan other learning technologies, especially in adult learning. How-ever, these technologies remove the boundaries of being in a class-room or in a face-to-face interaction with a teacher or otherstudents (Hashemi et al., 2011), which can cause some inefficientsituations (such as distraction) in a learning process. To make themobile learning process more efficient, mobile learninginstructional design is one of the most important issues thatshould be taken into consideration. An instructional design modelthat is specified for mobile learning and addresses these needs canmake mobile learning more effective and efficient. Future studiesthat conducted on developing instructional design model formobile learning can make important contribution to the existingliterature.

Another issue that needs to be addressed in the future studies isstandardization. There are many types of mobile devices that havedifferent hardware and software from each other. These differ-ences can cause some serious difficulties in the process of mobilecontent development. Studies on the standardization of contentdevelopment environments and content itself for mobile devicesmay address these difficulties. Using learning object or SCORMstandards might be helpful in such studies.

The development of mobile learning content or mobile learninginstructional design is a subject that remains to be studied. Instruc-tional designers from different disciplines may participate in suchstudies with multidiscipline contents. Also different softwaredevelopment environments such as MS Visual Studio C# may beconsidered as mobile content development tools in such studies.These types of studies are considered to contribute to mobile learn-ing and mobile learning instructional design.

References

Alvarez, C., Alarcon, R., & Nussbaum, M. (2011). Implementing collaborativelearning activities in the classroom supported by one-to-one mobilecomputing: A design-based process. Journal of Systems and Software, 84(11),1961–1976.

Attewell, J. (2004). Mobile technologies and learning. In N. Sweeney (Ed.), Atechnology update and m-learning project summary (pp. 25). London, UK:Learning and Skills Development Agency.

Bidaki, M. Z., Sanati, A. R., & Ghannad, F. R. (2013). Producing and introducingmobile books, as a new model of providing learning content in medical sciences.Procedia – Social and Behavioral Sciences, 83, 99–102.

Boot, E. W., van Merriënboer, J. J. G., & Theunissen, N. C. M. (2008). Improving thedevelopment of instructional software: Three building-block solutions tointerrelate design and production. Computers in Human Behavior, 24(3),1275–1292.

Cavus, N., & Uzunboylu, H. (2009). Improving critical thinking skills in mobilelearning. Procedia – Social and Behavioral Sciences, 1(1), 434–438.

Churchill, D., & Hedberg, J. (2008). Learning object design considerations for small-screen handheld devices. Computers & Education, 50(3), 881–893.

Clay, C. A. (2011). Exploring the use of mobile technologies for the acquisition ofclinical skills. Nurse Education Today, 31(6), 582–586.

Furió, D., González-Gancedo, S., Juan, M. C., Seguí, I., & Rando, N. (2013). Evaluationof learning outcomes using an educational iPhone game vs. traditional game.Computers & Education, 64, 1–23.

Gedik, N., Hanci-Karademirci, A., Kursun, E., & Cagiltay, K. (2012). Key instructionaldesign issues in a cellular phone-based mobile learning project. Computers &Education, 58(4), 1149–1159.

Hamat, A., Embi, M. A., & Hassan, H. A. (2012). Mobile learning readiness amongUKM lecturers. Procedia – Social and Behavioral Sciences, 59, 406–410.

Hashemi, M., Azizinezhad, M., Najafi, V., & Nesari, A. J. (2011). What is mobilelearning? Challenges and capabilities. Procedia – Social and Behavioral Sciences,30, 2477–2481.

Huang, C. (2005). Designing high-quality interactive multimedia learning modules.Computerized Medical Imaging and Graphics, 29(2–3), 223–233.

Huang, Y.-M., Kuo, Y.-H., Lin, Y.-T., & Cheng, S.-C. (2008). Toward interactive mobilesynchronous learning environment with context-awareness service. Computers& Education, 51(3), 1205–1226.

Hwang, G.-J., & Chang, H.-F. (2011). A formative assessment-based mobile learningapproach to improving the learning attitudes and achievements of students.Computers & Education, 56(4), 1023–1031.

Idrus, R. M., & Ismail, I. (2010). Role of institutions of higher learning towards aknowledge-based community utilising mobile devices. Procedia – Social andBehavioral Sciences, 2(2), 2766–2770.

Kim, L., & Albers, M. J. (2001). Web design issues when searching for information ina small screen display. In Proceedings of the 19th annual international conferenceon computer documentation (pp. 193–200). Sante Fe, New Mexico, USA: ACM.

Korucu, A. T., & Alkan, A. (2011). Differences between m-learning (mobile learning)and e-learning, basic terminology and usage of m-learning in education.Procedia – Social and Behavioral Sciences, 15, 1925–1930.

Kwon, S., & Lee, J. E. (2010). Design principles of m-learning for ESL. Procedia – Socialand Behavioral Sciences, 2(2), 1884–1889.

Li, L., & Leina, L. (2012). Designing principles of mobile learning in ESP course forChinese students. IERI Procedia, 2, 142–148.

Looi, C.-K., Wong, L.-H., So, H.-J., Seow, P., Toh, Y., Chen, W., et al. (2009). Anatomy ofa mobilized lesson: Learning my way. Computers & Education, 53(4), 1120–1132.

Luchini, K., Quintana, C., & Soloway, E. (2004). Design guidelines for learner-centered handheld tools. In Proceedings of the SIGCHI conference on humanfactors in computing systems (pp. 135–142). Vienna, Austria: ACM.

Martin, S., Diaz, G., Plaza, I., Ruiz, E., Castro, M., & Peire, J. (2011). State of the art offrameworks and middleware for facilitating mobile and ubiquitous learningdevelopment. Journal of Systems and Software, 84(11), 1883–1891.

Martin, F., & Ertzberger, J. (2013). Here and now mobile learning: An experimentalstudy on the use of mobile technology. Computers & Education, 68, 76–85.

Mayer, R. E. (2003). The promise of multimedia learning: Using the sameinstructional design methods across different media. Learning and Instruction,13(2), 125–139.

Mohammad, N. M. N., Mamat, M. N., & Isa, P. M. (2012). M-learning in Malaysia:Challenges and strategies. Procedia – Social and Behavioral Sciences, 67, 393–401.

Nordin, N., Embi, M. A., & Yunus, M. M. (2010). Mobile learning framework forlifelong learning. Procedia – Social and Behavioral Sciences, 7, 130–138.

Nunnally, J. C. (1978). Psychometric theory (McGraw-Hill Series in Psychology) (2nded.). New York: Mcgraw-Hill College.

Özdamli, F. (2011). The experiences of teacher candidates in developinginstructional multimedia materials in project based learning. Procedia – Socialand Behavioral Sciences, 15, 3810–3820.

Ozdamli, F. (2012). Pedagogical framework of m-learning. Procedia – Social andBehavioral Sciences, 31, 927–931.

Por, F. P., Mustafa, Z., Osman, S., Phoon, H. S., & Fong, S. F. (2012). Design anddevelopment of multimedia pronunciation learning management system for non-native English speakers. Procedia – Social and Behavioral Sciences, 64, 584–593.

Sandberg, J., Maris, M., & de Geus, K. (2011). Mobile English learning: An evidence-based study with fifth graders. Computers & Education, 57(1), 1334–1347.

Sharples, M. (2000). The design of personal mobile technologies for lifelonglearning. Computers & Education, 34(3–4), 177–193.

Ç. Güler et al. / Computers in Human Behavior 39 (2014) 128–135 135

Sun, P.-C., & Cheng, H. K. (2007). The design of instructional multimedia in e-learning: A media richness theory-based approach. Computers & Education,49(3), 662–676.

Sung, E., & Mayer, R. E. (2013). Online multimedia learning with mobile devices anddesktop computers: An experimental test of Clark’s methods-not-mediahypothesis. Computers in Human Behavior, 29(3), 639–647.

Triantafillou, E., Georgiadou, E., & Economides, A. A. (2008). The design andevaluation of a computerized adaptive test on mobile devices. Computers &Education, 50(4), 1319–1330.

Turner, J. S., & Croucher, S. M. (2013). An examination of the relationships amongUnited States college students’ media use habits, need for cognition, and gradepoint average. Learning, Media and Technology, 1–16.