Venkat Sastry talking on MoLE ProjectA Global Technology Initiative

Dr. Venkat V S S Sastry,

Jacob Hodges

13 November 2014 1ONRG Grant N62909-10-1-7121

21 countries

Chile

Bulg

aria

Canada

Azerb

aija

n

Fra

nce

Germ

any

Georg

ia

Jord

an

Mexic

o

Norw

ay

Pola

nd

Peru

Rom

ania

UK

US

Serb

ia

South

Afr

ica

Sin

gapore

sw

itzerland

Ukra

ine

UA

E

13 November 2014ONRG Grant N62909-10-1-71212

• MoLE Project was a two year initiative sponsored by

Coalition Warfare Program involving 24 countries

• Based on requirement by the Commander,

US Naval Forces Europe/Naval Forces Africa/Sixth Fleet

Deputy Director for Training and Readiness to

effectively operate in the largest maritime area of

operations where the most difficult

challenge is the ability to train and communicate

MoLE would leverage the global cellular

network infrastructure. Mobile

technologies and emerging mobile

applications/service models to build an m-

Learning capability

13 November 2014ONRG Grant N62909-10-1-71213

13 November 2014ONRG Grant N62909-10-1-71214

2mins 50 secs

Medical Content Working Group

Technology & Transition Group

Testing & Evaluation Working Group

13 November 2014ONRG Grant N62909-10-1-71215

MC

WG

: A

cco

mp

lish

men

ts-

To

ols

an

d C

on

ten

t

13 November 2014ONRG Grant N62909-10-1-71217

MC

WG

: A

cco

mp

lish

men

ts-

To

ols

an

d C

on

ten

t

13 November 2014ONRG Grant N62909-10-1-71218

Tec

hn

olo

gy &

Tra

nsit

ion

W

G

Blending “task” and

“learning” Initially, content

grouped by moment of

need

Mission Tools

A collection of interactive job aids and performance

support tools

Learning

The Learning section focuses on informal and formal

training experiences.

Library

The Library contains a multimedia collection of indexed

materials in different media (text, eBook, video) covering

topics relevant to the organization.

Mission Packs

Mission packs allows download of new content

Standards

The Standards section contains critical references

Network

Crowd-sourcing information on local resources and key

contacts.

Place holder for

future content

Most work based m-learning happens in stolen moments, which is why a smooth blend between learning, performance-support and productivity-tools is so key

Compelling m-learning is not just about the content. Or the technology. It is also about good learning design, good usability, and an app that you want to use

“

”

11

Converting and originating new content for MSO mobile learning

11

Task

Locate the pre-deployment

checklist and check off at

least 3 items for your

personal preparations.

Add one new item that you

might need, and save

14

Usability review October 2011

Proof of Concept trials April to June 2012 using “baked in” evaluation

layer

All content needed significant

“mobilization”

Blend of tools + courses worked well

Benefits of strong UX / user interface

Cross platform = reuse (content still in use)

Rich media / video very effective

Similar responses regardless of age,

device type, nationality, background

Testi

ng

& E

va

luati

on

Gro

up

Data collection plan• Demographic data

• Formulate research questions

Compliance with Research Ethics

Data analysis and interpretation

13 November 2014ONRG Grant N62909-10-1-712117

The Testing & Evaluation Process adhered to the

research protocol guidelines established by the

Ethics

1. US Department of Defense Directive

(DODD) 5400.11 (DOD Privacy Program,

DODD 3216.2)

2. (Protection of Human Subjects in DoD-

Supported Research, SECNAVINST

3900.39D, 32CFR219 (Common Rule),

3. European Union (EU) Data Protection

Requirements

4. UK Ministry of Defence JSP 536 (Research

Ethics Committee).

13 November 2014ONRG Grant N62909-10-1-712118

All the investigators need

to complete Research

Ethics courses online

Testi

ng

& E

va

luati

on

Gro

up

(a) What is the effectiveness or

practicality of using mobile

technologies to provide

training?

0

10

20

30

40

50

600

1

2

3

4

5

6

7

All Users Responses

19

(b) What is the benefit or availability

of using mobile devices in providing

training?

Testi

ng

& E

va

luati

on

Gro

up

20

(c) Do persons who have used the

mobile device provided training

believe they are capable to

perform a desired outcome?

Testi

ng

& E

va

luati

on

Gro

up

21

(d) What is the degree to which a

mobile training application is

available in austere environments to

as many Role players as possible?

Testi

ng

& E

va

luati

on

Gro

up

22

EVALUATION QUESTIONS Category

1.1 Rate your confidence in using the checklist. Self-efficacy

2.1 How useful are mobile devices for this type of training? Utility

2.2 How easy was the CTIP content to navigate? Accessibility

3.1 How useful is access to digital reference materials on a mobile device? Utility

4.1 How easily could you explain to a colleague how to look up NGO logos? Self-efficacy

5.1 How useful are videos as a tool to enhance your understanding of USAID?Usefulness

6.1 How helpful are collaboratively updated contact details? Self-efficacy

7.1 How useful are mobile devices for training? Utility

7.2 How useful are mobile devices for refresher training? Utility

7.3 How useful are mobile videos to enhance understanding? Utility

7.4 How useful are digital reference materials on mobile devices? Utility

7.5 How useful are collaborative contact pages to access real time information?Utility

7.6 How easy was the Global MedAid app to navigate? Accessibility

7.7Please write up to five single words which best describe your overall

experience of using the mobile device as a tool for learning.

7.8 What did you like most about the mobile device as a tool for learning?

7.9 What did you like least about the mobile device as a tool for learning?

8 Any additional comments

23

13 November 2014ONRG Grant N62909-10-1-712124

13 November 2014ONRG Grant N62909-10-1-712125

13 November 2014ONRG Grant N62909-10-1-712126

13 November 2014ONRG Grant N62909-10-1-712127

Course

content

Short quiz

Collapsible

content

13 November 2014ONRG Grant N62909-10-1-712128

End of

module

exercises

Hyperlinked

terminology

13 November 2014ONRG Grant N62909-10-1-712129

0 – no response

1 – not confident

at all

2 – not confident

3 – not very

confident

4 – neither

confident nor

unconfident

5 – quite

confident

6 – confident

7 – very

confident

30

• A “Welcome Email” was initially sent to

approximately 600 volunteer on the first

day of the MoLE Proof of Concept

• The announcement email provided

additional information on installation, the

user guide and an introduction video.

31

• Launched: 252

• Started Proof of Concept: 166 (65.9%)

• Completed Proof of Concept: 126 (50%)

32

Question Responses

Age less than 20, 20-29, 30-39,

40-49, 50+, no answer

Gender Male, Female

How proficient are you in English Beginner . . . Advanced

Are you using your own personal smartphone for the purpose of

this trial?

Yes, No, No Answer

How comfortable are you with using the mobile device that's

running the MoLE app? [Beginner to Advanced]

Beginner . . . Advanced

Have you previously been involved in humanitarian assistance or

disaster relief operations?

Yes, No, No Answer

What is your professional expertise? Medical, Rescue, Training,

E-learning, Other

Have you taken the Trafficking in Persons (CTIP ) course within

the last two years?

Yes, No, No Answer

33

Age Group Breakdown

< 20 years 5 1.87%

20+ years 51 19.03%

30+ years 83 30.97%

40+ years 75 27.99%

50+ years 52 19.40%

No Response 2 0.75%

34

Gender Breakdown

Male 181 67.54%

Female 87 32.46%

35

Expert

Entering

36

37

NewbyExpert and

above

38

39

40

41

42

43

Likert Rating Question All-Users Android iPhone

N/A or Skipped 8 3 5

1 Not useful at all 9 1 8

2 Not useful 9 4 5

3 Not very useful 20 14 6

4 Neither useful nor not useful 18 6 12

5 Quite useful 33 15 18

6 Useful 60 23 37

7 Very useful 22 5 17

n: 179

Minimum: 0

Maximum: 7

Mean: 4.7 +/-0.3

Median 5.4

Mode: 6

Standard Deviation: 1.9

Variance: 3.5

0102030405060

0

1

2

3

4

5

6

7

All Users Responses

0

5

10

15

20

250

1

2

3

4

5

6

7

Android Users Responses

0

10

20

30

400

1

2

3

4

5

6

7

iPhone Users Responses

44

Likert Rating Question All-Users Android iPhone

N/A or Skipped 18 5 13

1 Not useful at all 2 0 2

2 Not useful 4 2 2

3 Not very useful 6 2 4

4 Neither useful nor not useful 9 2 7

5 Quite useful 42 17 25

6 Useful 47 26 21

7 Very useful 51 17 34

n: 179

Minimum: 0

Maximum: 7

Mean: 5.1 +/-0.3

Median 6.0

Mode: 7

Standard Deviation: 2.1

Variance: 4.5

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

05

1015202530

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

13 November 2014ONRG Grant N62909-10-1-712145

A majority of the Android

participants were “confident “and

iPhone users were

“very confident” in using the

checklist to support pre-

deployment activities.

46

Likert Rating Question All-Users Android iPhone

N/A or Skipped 25 9 16

1 Not useful at all 1 0 1

2 Not useful 2 0 2

3 Not very useful 3 0 3

4 Neither useful nor not useful 9 2 7

5 Quite useful 35 19 16

6 Useful 45 17 28

7 Very useful 59 24 35

n: 179

Minimum: 1

Maximum: 7

Mean: 5.1 +/-0.3

Median 6.0

Mode: 7

Standard Deviation: 2.3

Variance: 5.4

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

05

10152025

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

47

Likert Rating Question All-Users Android iPhone

N/A or Skipped 24 9 15

1 Not useful at all 0 0 0

2 Not useful 5 1 4

3 Not very useful 9 1 8

4 Neither useful nor not useful 17 5 12

5 Quite useful 27 15 12

6 Useful 54 30 24

7 Very useful 43 10 33

n: 179

Minimum: 2

Maximum: 7.0

Mean: 4.8 +/-0.3

Median 6.0

Mode: 6

Standard Deviation: 2.3

Variance: 5.2

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

1015202530

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

48

Likert Rating Question All-Users Android iPhone

N/A or Skipped 30 10 20

1 Not useful at all 1 1 0

2 Not useful 5 0 5

3 Not very useful 16 6 10

4 Neither useful nor not useful 14 4 10

5 Quite useful 29 13 16

6 Useful 46 27 19

7 Very useful 38 10 28

n: 179

Minimum: 1

Maximum: 7

Mean: 4.5 +/-0.4

Median 5.0

Mode: 6

Standard Deviation: 2.4

Variance: 5.8

01020304050

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

05

1015202530

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

1015202530

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

49

Likert Rating Question All-Users Android iPhone

N/A or Skipped 43 20 23

1 Not useful at all 0 0 0

2 Not useful 0 0 0

3 Not very useful 7 1 6

4 Neither useful nor not useful 11 5 6

5 Quite useful 22 15 7

6 Useful 54 21 33

7 Very useful 42 9 33

n: 179

Minimum: 3

Maximum: 7

Mean: 4.4 +/-0.4

Median 6.0

Mode: 6

Standard Deviation: 2.7

Variance: 7.2

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

10152025

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

50

Likert Rating Question All-Users Android iPhone

N/A or Skipped 14 4 10

1 Not useful at all 2 1 1

2 Not useful 11 4 7

3 Not very useful 27 16 11

4 Neither useful nor not useful 23 6 17

5 Quite useful 26 14 12

6 Useful 38 20 18

7 Very useful 38 6 32

n: 179

Minimum: 0

Maximum: 7

Mean: 4.4 +/-0.3

Median 5.0

Mode: 6, 7

Standard Deviation: 2.0

Variance: 4.2

0

10

20

30

400

1

2

3

4

5

6

7

All Users Responses

0

5

10

15

200

1

2

3

4

5

6

7

Android Users Responses

0

10

20

30

400

1

2

3

4

5

6

7

iPhone Users Responses

51

Likert Rating Question All-Users Android iPhone

N/A or Skipped 5 1 4

1 Not useful at all 5 2 3

2 Not useful 22 8 14

3 Not very useful 6 3 3

4 Neither useful nor not useful 18 3 15

5 Quite useful 37 16 21

6 Useful 58 29 29

7 Very useful 28 9 19

n: 179

Minimum: 0

Maximum: 7

Mean: 4.8 +/-0.3

Median 5.3

Mode: 6

Standard Deviation: 1.8

Variance: 3.4

0102030405060

0

1

2

3

4

5

6

7

All Users Responses

05

1015202530

0

1

2

3

4

5

6

7

Android Users Responses

05

1015202530

0

1

2

3

4

5

6

7

iPhone Users Responses

‘They can because they

think they can’

52

Likert Rating Question All-Users Android iPhone

N/A or Skipped 33 12 21

1 Not useful at all 0 0 0

2 Not useful 2 0 2

3 Not very useful 7 3 4

4 Neither useful nor not useful 11 3 8

5 Quite useful 28 15 13

6 Useful 45 17 28

7 Very useful 53 21 32

n: 179

Minimum: 2

Maximum: 7

Mean: 4.7 +/-0.4

Median 6.0

Mode: 7

Standard Deviation: 2.5

Variance: 6.3

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

10152025

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

53

Likert Rating Question All-Users Android iPhone

N/A or Skipped 18 5 13

1 Not useful at all 2 0 2

2 Not useful 4 2 2

3 Not very useful 6 2 4

4 Neither useful nor not useful 9 2 7

5 Quite useful 42 17 25

6 Useful 47 26 21

7 Very useful 51 17 34

n: 179

Minimum: 0

Maximum: 7

Mean: 4.7 +/-0.3

Median 5.6

Mode: 7

Standard Deviation: 2.2

Variance: 4.6

0

10

20

30

40

50

600

1

2

3

4

5

6

7

All Users Responses

05

1015202530

0

1

2

3

4

5

6

7

Android Users Responses

0

10

20

30

400

1

2

3

4

5

6

7

iPhone Users Responses

54

280

113

42 33

59.8%

84.0%

92.9%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

0

50

100

150

200

250

300

350

400

450

Nu

mb

er

of

Refe

ren

ces

Categories

A mobile application framework that

consisted of (a) a mobile applications (for

iOS and Android mobile operating

systems) for the storage and display of

mobile learning content range of open

formats; and (b) an application

programming interface (API) describing

how interactive content (produced in

HTML) may interact with the application.

13 November 2014ONRG Grant N62909-10-1-712157

A web-based content management system

for the storage and management of

learning content, plus the associated

standards for 'packaging' and meta-data

describing learning content.

13 November 2014ONRG Grant N62909-10-1-712158

13 November 2014ONRG Grant N62909-10-1-712159

A 'mobile evaluation layer' for surveying

user feedback through the mobile

applications during the period of research

by (a) presenting users with contextually-

relevant questions during use of the mobile

applications; and

collecting and collating evaluation data in

a central repository, including techniques

for associating multiple evaluation

responses from single user and allowing

evaluations to be undertaken 'offline'

(without cellular or Wi-Fi internet access)

for asynchronous transfer to the repository.

13 November 2014ONRG Grant N62909-10-1-712160

All of the Mobile Learning Environment

(MoLE) source code for the iOS and Android

Apps and the online content repository has

been uploaded to Open Source Mobile

Learning Environment (OMLET) and is

available, free of charge, at

https://wss.apan.org/1539/JKO/mole/SitePa

ges/Home.aspx.

13 November 2014ONRG Grant N62909-10-1-712161

Mobile devices are practical and effective

Screen size is very important

Productivity tools enhance learning

experience

Mobile training applications are

appropriate in challenging environments

13 November 2014ONRG Grant N62909-10-1-712162

Book

The “Mobile Learning Environment (MoLE) Project: A Global

Technology Initiative”, is about a two-year training and education

project that was sponsored by the U.S. Department of Defense

(DoD)’s Coalition Warfare Program (CWP) and other research-

related organizations that focused on providing training/education in

areas with low-bandwidth and limited internet connectivity and

infrastructure.

This book is available in paperback and e-reader version via

Amazon. The paperback is also available via CreateSpace

(https://www.createspace.com/4174484)

It will be of interest to:

• Learning and Development Professionals

• Educators and Teachers

• E-Learning and m-Learning Professionals

• Evaluators

• Government and Non-Governmental Organizations (NGO)

• Information Technology (IT) Professionals

• Medical Professionals

• Quality Management Professionals

• Research, Development, Testing & Evaluation (RDT&E)

Professionals

• Scientist and Technology (S&T) Professionals

A blog has been created http://wp.me/p2ZlwE-4o for anyone

interested in reading the Preface.

A brief overview of the MoLE Project via Slideshare

(http://www.slideshare.net/jrhodges1972/tracking/new#)

13 November 2014ONRG Grant N62909-10-1-712164

Backup Slides

(Not used in the presentation)

Learning is strongly influenced by

motivational factors and context

Measurement of learning is hotly debated

the extent to which technology enhanced

instruction facilitates learning adds to this

debate

13 November 2014ONRG Grant N62909-10-1-712165

Plan – 80 minutes

Play Video to begin with (3 mins)

Background to project; sponsors

Early designs (from Tribal slides)

Example content 15 mins

• Medical

• MATTs

• Evaluation

Challenges 10 mins

• Technological

• Cultural; expectations; linguistic

• Ethical compliance; IRBs

Data Collection Plan 15 mins

• Informed consent

• Questionnaire

13 November 2014ONRG Grant N62909-10-1-712166

• Data Analysis 15 mins

• Demographic data

• Exploratory data analysis

• Analysis of text; word co-

occurrence

• Final Report

• Major outputs

• Include Jake’s booklet on

Amazon

• Concluding Remarks

• Offer Live Demo

• Quiz NGO etc

67

• A “Welcome Reminder Email”

was initially bi-weekly to remind

the volunteers.

• The reminder email also

provided additional information

on installation, the user guide

and an introduction video.

68

69

Likert Rating Question All-Users Android iPhone

N/A or Skipped 30 11 19

1 Not useful at all 0 0 0

2 Not useful 2 0 2

3 Not very useful 6 1 5

4 Neither useful nor not useful 12 4 8

5 Quite useful 20 10 10

6 Useful 53 27 26

7 Very useful 56 18 38

n: 179

Minimum: 2

Maximum: 7

Mean: 4.9 +/-0.4

Median 6.0

Mode: 7

Standard Deviation: 2.5

Variance: 6.1

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

1015202530

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

70

Likert Rating Question All-Users Android iPhone

N/A or Skipped 23 10 13

1 Not useful at all 2 0 2

2 Not useful 2 0 2

3 Not very useful 8 4 4

4 Neither useful nor not useful 14 7 7

5 Quite useful 34 9 25

6 Useful 46 25 21

7 Very useful 50 16 34

n: 179

Minimum: 1

Maximum: 7

Mean: 5.0 +/-0.3

Median 6.0

Mode: 7

Standard Deviation: 2.3

Variance: 5.2

01020304050

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

10152025

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

71

Likert Rating Question All-Users Android iPhone

N/A or Skipped 33 12 21

1 Not useful at all 0 0 0

2 Not useful 2 0 2

3 Not very useful 7 3 4

4 Neither useful nor not useful 11 3 8

5 Quite useful 28 15 13

6 Useful 45 17 28

7 Very useful 53 21 32

n: 179

Minimum: 2

Maximum: 7

Mean: 4.7 +/-0.4

Median 6.0

Mode: 7

Standard Deviation: 2.5

Variance: 6.3

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

10152025

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

72

Likert Rating Question All-Users Android iPhone

N/A or Skipped 41 18 23

1 Not useful at all 0 0 0

2 Not useful 0 0 0

3 Not very useful 4 2 2

4 Neither useful nor not useful 8 2 6

5 Quite useful 15 7 8

6 Useful 47 21 26

7 Very useful 64 21 43

n: 179

Minimum: 3

Maximum: 7

Mean: 4.7 +/-0.4

Median 6.0

Mode: 7

Standard Deviation: 2.7

Variance: 7.5

0

20

40

60

80

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

01020304050

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

10152025

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

73

Likert Rating Question All-Users Android iPhone

N/A or Skipped 42 19 23

1 Not useful at all 0 0 0

2 Not useful 0 0 0

3 Not very useful 5 3 2

4 Neither useful nor not useful 16 7 9

5 Quite useful 27 15 12

6 Useful 34 11 23

7 Very useful 55 16 39

n: 179

Minimum: 3

Maximum: 7

Mean: 4.5 +/-0.4

Median 5.0

Mode: 7

Standard Deviation: 2.7

Variance: 7.3

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

0

5

10

15

20

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

74

Likert Rating Question All-Users Android iPhone

N/A or Skipped 43 20 23

1 Not useful at all 0 0 0

2 Not useful 0 0 0

3 Not very useful 4 3 1

4 Neither useful nor not useful 10 3 7

5 Quite useful 22 9 13

6 Useful 46 24 22

7 Very useful 54 12 42

n: 179

Minimum: 3

Maximum: 7

Mean: 4.6 +/-0.4

Median 6.0

Mode: 7

Standard Deviation: 2.7

Variance: 7.5

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

01020304050

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

10152025

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

75

Likert Rating Question All-Users Android iPhone

N/A or Skipped 44 21 23

1 Not useful at all 1 0 1

2 Not useful 2 1 1

3 Not very useful 4 2 2

4 Neither useful nor not useful 13 5 8

5 Quite useful 25 11 14

6 Useful 35 15 20

7 Very useful 55 16 39

n: 179

Minimum: 1

Maximum: 7

Mean: 4.4 +/-0.4

Median 6.0

Mode: 7

Standard Deviation: 2.8

Variance: 7.6

0102030405060

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

0

10

20

30

40

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

10152025

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

76

Likert Rating Question All-Users Android iPhone

N/A or Skipped 46 22 24

1 Not useful at all 1 0 1

2 Not useful 3 2 1

3 Not very useful 12 3 9

4 Neither useful nor not useful 22 11 11

5 Quite useful 21 7 14

6 Useful 44 23 21

7 Very useful 30 3 27

n: 179

Minimum: 0

Maximum: 7

Mean: 4.0 +/-0.4

Median 5.0

Mode: N/A or

Skipped

Standard Deviation: 2.6

Variance: 6.9

01020304050

Skipped orN/A

1

2

3

4

5

6

7

All Users Responses

05

1015202530

Skipped orN/A

1

2

3

4

5

6

7

iPhone Users Responses

05

10152025

Skipped orN/A

1

2

3

4

5

6

7

Android Users Responses

13 November 2014ONRG Grant N62909-10-1-712177

13 November 2014ONRG Grant N62909-10-1-712178

0

1

2

3

4

5

6

7

<20 20+ 30+ 40+ 50+ <20 20+ 30+ 40+ 50+

0

3.9

1.41.1

3.4

6.8

6.3

5.24.8

3.9

Series1

As a learning platform

As a collaborative platform

As a problem solving tool

As a data acquisition device

As a computational resource (e.g. run

simulations)

As a communication device

13 November 2014ONRG Grant N62909-10-1-712179

an evaluation strategy must be cognizant

of desired learning outcomes for the

intended learner, and should take into

account the combination of instructional

strategies that include delivery media

13 November 2014ONRG Grant N62909-10-1-712180

learning effectiveness is a function of

effective pedagogical practices

smartphone technologies facilitate

constructive engagement with the learner

that in turn influences the pedagogic

practice

13 November 2014ONRG Grant N62909-10-1-712181

The internal and cognitive information

processing events include temporary

storage, encoding and retrieval.

The external instructional events include

gaining attention, drill and practice, and

feedback

13 November 2014ONRG Grant N62909-10-1-712182

Individuals who proactively approach their learning tasks• Personal initiative, perseverance, adaptive skills

Self-regulated processes help learners acquire knowledge and skills more effectively

Self-regulation of learning has been suggested to refer to self-directed processes that help learners learn more effectively

13 November 2014ONRG Grant N62909-10-1-712183

Item Instrument; Remarks

Background knowledge,

prior skills, abilities

Conduct a pre-test

Self-regulation in learning

context(Toering, Elferink-

Gemser, Jonker, Van

Heuvelen, & Visscher,

2012)

Use of tools that support self-regulation; Collect Self-regulation of Learning and

Self-Report Scale prior and post learning intervention. The online form is available

at

https://docs.google.com/spreadsheet/viewform?formkey=dHFwNnBiRjY1QWM0eE5

ielA5X0pVdXc6MQ.

Learning content Analysis of learning content provided for the learner;

Learning support tools Use of auxiliary support tools online, on the device; usage by the learner;

measurement of learner engagement; learner analytics

Learning transfer Conduct post-tests, apply knowledge to specific tasks that map to learning

objectives; these tests include, quizzes, case studies, written assignments and peer

review

Learner feedback Qualitative feedback and textual analysis

Learning style/preferences Learning modality preference inventory and Kiersey Temperament Inventory. It is

anticipated that the learning style has second order effects on learning. (see also

(Neuhauser, 2002)). Learning style may have stronger influence on learner

engagement and attention span.

Time on task Record of tasks completed and the use of tools. This data may be collected on the

device (aka MoLE).

Instructional method Types used and the nature of the tasks

Teacher effects Controlled experiment

13 November 2014ONRG Grant N62909-10-1-712184