PI/Programmer Interaction

Critical elements that must be decided by theresearcher before working with the IT staff who areinvolved in preparing the tablets and programmingthe survey instrument.

5-7 Tests in the Development Lifecycle

Programmer and project team should thoroughlytest the tablet-based survey instrument throughoutthe development lifecycle of the survey instrument.

Enumerator Training

Training the enumerators on the paper-basedsurvey instrument, as well as the tablet-basedinstrument, is critical to identify problems that ariseonly in the field and to ensure accurate datacollection practices.

Equipment & MDM Strategies

Equipment selection, preparation and logisticsmust be included in the timeline. Don’t forget yourmobile device management strategy!

Tablet-based Data Collection:

A Blueprint for a Successful Data Collection Project

Tim Van AckerIT Director, Carolina Population Center

Acknowledgments2

CPC Staff members who helped me write the original document by this name:

• Mary Jane Hill: QDS, CSPro, Blaise, SAS programmer• Mandy Monath: RedCap, ODK, SAS, Stata Programmer• David Perrin: Linux Administrator/ODK programmer• Joyce Tabor: Add Health data manager, data entry supervisor

Neil Hendrick, UC Berkley Human Rights Center• (https://opendatakit.org/help/training-guides/)

John Garcia, Frank Porter Graham Child Development Center• Windows 8.1 imaging advice and hardware to make it happen

PI/Programmer Interaction3

Communication between the PI and programmer is crucial throughout the development lifecycle

• Programmer reviews survey instrument and prepares questions for PI.• Clarification regarding how the PI wants to recreate a table in tablet format.• PI should specify the following: skip logic, constraints, variable naming

convention, hints for interviewer or respondent, calculations, which fields are required, choices for select_one and select_multiple questions, how many questions per screen, use “select all that apply” or ask “yes/no” for each item.

• Discuss pros and cons of using open-ended questions and text boxes such as “Other, specify.”

• Discuss coding and standardization issues so that all common responses (e.g., yes/no) are coded in the same way.

• Discuss how the PI wants to handle “don’t know, unsure, refused to answer.”• Agree on a timeline.

PI/Programmer Interaction (cont.)4

PI/Programmer Interaction (cont.)5

Paper-based survey instrument should be 95% complete before tablet-based programming begins

• The paper-based survey instrument should be vetted/tested by project team members, including review of the text and skip logic, and trial runs should be done by doing mock interviews using the paper-based survey. This will reveal problems before the programming begins.

• The Rolling Stones had ‘time on their side’ – programmers don’t!• If a language other than English is being used on the tablet administration,

the translation needs to be completed and vetted (ideally by a local speaker) before programming begins.

• Interviewer instructions on how to administer the questions (e.g., what gets read, what is an instruction to the interviewer, how tables are completed) must be developed before programming begins.

PI/Programmer Interaction (cont.)6

Programming should be done based on paper survey instrument

• Q & A via email, as well as informal meetings in the hall, are helpful to understand what the PI wants, but are difficult to track and incorporate during the programming stage, and nearly impossible to verify during the testing stages!

• All decisions/changes should be incorporated into the official paper-based survey.

• It is wise to have a paper-based instrument as a backup in the field: programming to the paper survey will ensure the two are in-step.

• PI or a designate needs to edit the paper instrument to reflect all decisions and changes to the instrument, providing the programmer with updated and ultimately the final version of the survey instrument.

• All changes should be made using Track Changes.

PI/Programmer Interaction (cont.)7

A timeline for completion should be established and agreed upon

• How long will it take to program the survey?

• When will enumerator testing take place?

• When will data collection begin?

• The timeline should be reviewed frequently and updated as necessary: no one likes surprises!

• Factor in time zone differences and work hours to the communication pipeline. These two items will introduce lags in communication, especially during final field testing and any cycle of last minute tweaks!

PI/Programmer Interaction (cont.)8

The tablet-based survey instrument should be thoroughly tested

• Use 5-7 stages of testing• Verify survey compiles correctly on tablet• Text is spelled correctly• Questions are in proper order• Choices are spelled correctly and in proper order• Hints are included where necessary to give the enumerator and/or

respondent clear instructions on what is being asked• Relevant fields are entered accurately where necessary so skip/branching

logic works as intended• Constraints are entered for fields when necessary• Constraint messages are short and clear• Calculations work as intended• All required fields are flagged as required• Data looks like you expect it to look

PI/Programmer Interaction (cont.)9

Programming tips

• Program the survey instrument in sections:• Save each section as a separate file,• Use “notes” to include reminders regarding skip logic between sections,• Allows others to test while programmer is working on other sections.

• Extract dummy data and deliver to PI for review and approval.

• Modify completed sections based on feed-back from other testers.

• Allow time to combine all sections into one survey instrument and complete more testing!

PI/Programmer Interaction (cont.)10

Implementation, follow-up, and additional forms

• Have a Plan B: electricity and the Internet don’t always work as smoothly/consistently as we may like!

• Paper-based survey instruments should be made available in case of emergency.• Car chargers or solar-powered chargers should be available to keep tablets charged.• Enumerators should be thoroughly familiar with the paper survey.• PI and programmer should keep in contact regarding data being uploaded.• PI or field supervisor should monitor progress of uploads to ensure enumerators are

each doing approximately the same number of surveys as one another.• “Are we done yet?”

• Invariably, someone will come up with ideas for additional forms that are needed (e.g., non-completion form, follow-up form, etc.)

• Media: do you want pictures, video or audio?• Communication between PI and programmer is crucial in the final stages since the

programmer may need to begin work on someone else’s project. Communicate!

5-7 Tests in the Development Lifecycle11

1. Paper testing: Work out as many kinks as possible on the paper instrument (95%)

2. Development testing: Program a question or two and then test on the tablet

3. Pre-testing: Someone other than the programmer should test on a tablet

4. Enumerator pre-testing: Test on individual sections

5. Enumerator testing: Test on combined instrument

6. Field testing: Whole survey field-tested with real people not in the study location

7. Final testing (ODK only): Final field testing with survey on ODK Aggregate server

Enumerator Training12

• During enumerator training, enumerators look for the following and report back to the group after each exercise listed below:• Typos• Errors in flow of questions• Errors in skip patterns• Duplicate questions• Questions to be added• Problems with language translation

• Paper instrument training• 2-10 days (based on the size of the survey instrument)• Understand questions• Understand flow/skip logic• Understand how to ask the questions• Read through each question as a group, individually, and with a partner

Enumerator Training (cont.)13

• Tablet training• Basics: Turn on/Turn off/Logon/Logoff/Navigation• Sample Survey: 10-15 sample/fun questions

• How to load survey• How to navigate through the survey• How to save results mid-survey/end of survey• How to backup/send survey to server• What can possibly go wrong? And how to recover!

• Real survey• Read around: take turns reading questions while everyone inputs data• Self-administer: everyone reads each question aloud and enters their

responses• Buddy system: odd/even questions or alternate sections• Dry-run (pre-test) in the field (buddy system, alternating sections)

Equipment and MDM Strategy14

• Equipment: recommendations, logistics, and time to order, receive, configure and ship must be included in the timeline.

• Purchase equipment as far in advance as possible to allow for issues of backorder, configuration, shipping to the field.

• Plan on 2-3 hours per tablet for unpacking, labeling, imaging, encrypting, enrolling in MDM, testing, providing paperwork for customs and re-packaging for mailing or transport.

• Check tablets out each morning and back in each evening!

Equipment and MDM Strategy (cont.)15

• Android devices:• Nexus 7• Galaxy Tab (AirWatch/Samsung have a partnership, so tighter integration of

security controls)

• Windows devices:• Dell Venue 11 Pro• Dell Venue 8 Pro• Asus T100

• Miscellaneous• Adapters for plugs in-country• Car chargers• Solar-powered chargers• Surge Protectors• Purchase extra tablets (20% for breakage, theft, etc.)• Provide paperwork for customs

Equipment and MDM Strategy (cont.)16

Mobile Device Management• AirWatch• Meraki

Research Centers at UNC-CH are conducting ongoing

discussions related to tablet-based data collection.

Topics:• Review what different units have used for tablet-based data collection.

• Think about ways we might share experiences, software, and potentially staff.

• Demonstrations of various programs (e.g., Qualtrics, RedCap, ODK, CSPro,

Blaise).

• Discussion regarding advantages/disadvantages of various programs.

If you would like to join us, contact cpchelp@unc.edu

with subject heading “tablet-based data collection

meetings.”

17