HuskyHunt Conceptual Design Document

The Practicum

My intention with the practicum is to leverage the existing HuskyHunt framework and

measure the effects of micro-credentials/badges on game completion and player engagement. The assessment will be done using the quantitative and qualitative assessments described below.

Analysis and Design of Technologies, Lesson content and Pedagogy

HuskyHunt

The university recognizes that its students do not have sufficient understanding of the personal security and behavioral risks associated with their activities on the Internet (e.g., copyright infringement, phishing, electronic banking security). There is also concern regarding the students general apathy related to personal privacy. Because mandatory training was discussed but dismissed, HuskyHunt was created to provide a learning format that students would opt into.

HuskyHunt is a web based application designed with four specific goals:

Teach students practical lessons about Internet and information security awareness topics that they can relate to personally

Utilize gaming mechanics to increase engagement and participation Leverage Social media, specifically Facebook and Twitter to reinforce learning that

occurred in the game modules.

University of Connecticut

Formative Assessment of HuskyHunt

Pufahl, Jason8-12-2014

http://huskyhunt.uconn.edu/


Leverage social media to increase the likelihood that UConn students not participating in HuskyHunt are exposed to the security training.

HuskyHunt is designed to be delivered at a large scale and has no technical limitations to the number of individuals that can participate in an available and active course. The framework aligns most closely to an xMOOC. An xMOOC is geared towards an individual seeking an achievement, is likely offered in addition to other complementary education and is designed to be repeated (“elearnspace › MOOCs are really a platform,” n.d.).

The application was originally designed with no specific pedagogy in mind, but upon reflection I feel the online content and assessment framework most closely resembles an objectivist learning theory (Duffy & Jonassen, 1992). HuskyHunt was developed and implemented to deliver “the entities, relations and attributes that the learner must ‘know’” (Duffy & Jonassen, 1992 location 192). It also shares similarities with a current standards based curriculum concepts (Carr-Chellman, 2011) leveraging directed instruction and assessments. HuskyHunt establishes clearly communicated learning objectives and leverages online multiple choice assessments designed to test specifically what was taught in the lesson.

There game has incorporated, inadvertently, some of Gagne’s 9 events of instruction (Pastula, 1999):

Marketing and prizes were used to gain potential players attention Materials to be learned were presented in chunks Learning was assessed

Recognizing that the primary online teaching module of HuskyHunt does not actively engage students in the learning process, the application has a secondary, real world module that leverages the concept of practice fields (Jonassen & Land, 2012). The secondary module is designed to engage players in practical activities that reinforce concepts taught during the online portion of the game.

The application has been marketed to students primarily as a game and secondarily as a security awareness education tool. The distinction was made with the hope that playing a game containing tangible prizes (e.g. cash rewards, coupons at local merchants, swag) would incentivize students to engage in security awareness training that isn’t mandatory. We originally underestimated the students desire to learn and over the last few years have modified the marketing efforts to more clearly position the game as a learning opportunity with prizes.

The application employs three basic game mechanics:

There are Rules There is a scoring system There are rewards throughout and during the game for participants and winners

1

http://huskyhunt.uconn.edu/game-rules/


Learning modules are presented and assessed using multiple choice questions and answers. As students complete modules they earn points which ultimately positions them to win the grand cash prize. Players are also able to earn points by participating in secondary modules designed to support the learning objective of the primary module. Player motivation occurs primarily through a points system designed around module completion and secondary activities participation. Additional in game prizes such as merchant coupons are also available periodically throughout the game.

Social Media is used in two ways:

Allow the game master to communicate game events to player. Allow players to share security awareness information to their

friends/followers.

All announcements utilize the hashtag “#HuskyHunt” for consistency.

Micro-Credentials

Micro-Credentials, also known as digital badges, are designed to provide confirmation for the successful achievement of particular activities. Digital badges are conceptually similar to the physical badges (i.e., insignia) used in the military. Digital badges serve a variety of functions, including (“Digital badges - Wikipedia, the free encyclopedia,” n.d.), (“About | Open Badges,” n.d.)

To provide a reward To convey some relevant information To establish a goal To identify someone as a member of a group To provide status

Badges will be added to the existing game framework as a mechanism to provide increased incentives for players to participate in and complete the game.

Expected Outcomes and Objectives

The general instructional objective of the project will be to determine the effect that the opportunity to earn and collect digital badges has on participation and completion rates.

The information gained from the study will be used to infer whether or not digital badges can improve access to information. However, it will not specifically assess whether or not learning occurred. It is expected that the players engaged in the application version that leverages badges will progress further and more quickly through the game than those players not receiving badges. It is also expected that the players receiving badges indicate that they enjoy the game more.

2


Digital Badge Implementation

Clear, concise badge design will be essential for establishing motivators that sufficiently contribute to completion and engagement. Badge design has not yet occurred, but I am initially considering the following badges (Badge titles are currently descriptive only):

Fastest to Answer Primary Module:Awarded to the first Player that completes a primary learning module. There will be an opportunity to earn this in each primary learning module.

Fastest to Answer Secondary Module:Awarded to the first Player that completes a secondary learning module. There will be an opportunity to earn this in each secondary learning module.

Security Ninja:Awarded to anyone completing the primary and all secondary daily modules.

Fastest to Answer ALL Primary Modules:Awarded to the individual that was quickest, on average, to complete all Primary modules.

Most recruits:Awarded to the player that recruits the most players to join.

Gone Phishing:Awarded to players passing the Phishing module the first time.

Spread the word:Awarded to players posting game provided security messages to Facebook/Twitter.

Game Implementation

The current application and gameplay will be modified to ensure that the data collected will be useful for analysis. The following are key decision points:

The game will be run as online-only, omitting the real world scavenger hunt. The game will permit user created accounts and no longer rely solely on NetID for

access. This change should improve player demographics by enabling players to be recruited both internally and externally to UConn.

The game will have two versions, one with badges and scoring and the other with scoring only.

3


The games will be limited to one week, but two game sessions will be run. This will permit the collection of data over multiple sessions and is expected to provide more reliable results.

The application will be designed to log additional data points to improve analytical options. It is currently expected that the following data will be collected: User demographic information, Time spent on each page, page transition order, assessment statistics (e.g., success, failure, and attempts), social media participation. It is likely more data will collected as assessments are refined.

The game will include a video introduction by me. This will provide a human element to the game and hopefully establish trust in the game, the creator, and the purpose.

The most significant challenge will be recruiting players. Unfortunately, I believe that utilizing the more common method of announcing an incentive driven research study participant opportunity (e.g., via listserv, daily digest, posters) would likely produce players that felt some requirement to complete the game. The major consideration is collecting participants that want to play and do not feel that participation or game completion is in any way mandatory. I am currently evaluating ways to leverage security contacts across Connecticut Higher Education to assist in identifying potential new player pools.

Once potential players are identified, they will be sent instructions describing the game, the learning objectives, the account creation process, and the game start and end dates. As players sign up, they will be randomly added to game control groups. One control group will have opportunities to earn both points and digital badges and one control group will only earn points. Players will have an opportunity to recruit other players and will receive points and/or badges for their recruiting effort. Players will only be able to recruit until the time the game begins.

The game will be run with a primary daily lesson and associated assessment. The assessments are expected to remain multiple choice. There will be multiple secondary lessons that occur each day. These lessons will be offered at random times during the day and will likely be available only for short intervals. Both primary and secondary lessons will provide opportunities for players to earn points and badges if applicable. It is expected that there will be time limits to earn maximum points and that point values will decay to an established minimum greater than zero as time to completion increases. It is important that players always have the opportunity to earn some points so there is some incentive to complete each module.

As points and badges are earned they will be displayed on a leaderboard. All players will be able to view their current position in relationship to other player at any time.

The game will end with a final assessment likely to contain:

Multiple choice assessment utilizing questions drawn from each module assessment Multiple choice questions relating to each module, but not drawn directly from

module assessments Practical test to see if players respond to a phishing message

4


Practical test to see if players provide username and password information when asked for it

Players will be presented with a short optional survey at the end of the game and asked if they would be willing to participate in a short interview.

Evaluation

Assessments will be developed that produce quantitative statistics to measure completion rate, engagement, and skills achieved. Qualitative data will be gathered and used to measure proficiency in new technology literacies and attitudes towards digital badges and personal information security concepts.

Completion RateCompletion rate will be evaluated using the following statistics:

% of players completing each module % of players remaining after each module % of players completing all modules % of players abandoning game at some point % of players abandoning by module

EngagementEngagement rate will be evaluated using the following statistics:

Average elapsed time between lesson release and assessment completion % players posting to social media Recruitment of new players % of players completing post-game survey

Skills Learned Average times player required to answer an assessment question correctly Average times player required to answer final assessment question correctly % of people responding to spam email test % of people providing username and password

Attitudes Shift and New Learning Literacies A post-game survey will be used to obtain a greater understanding of the following

information:

Player’s thoughts and knowledge about Internet and information security before and after the game. A pre-game interview will not be conducted, so player attitude changes will need to be assessed only through a post-game interview.

Player’s thoughts on how badges influenced their desire to play the game. Players attitudes towards the technologies and concepts used in the game:

5


o Twitter/Facebooko Badgeso Mobile application accesso Points and scoringo Google Hangoutso Google Forms or Socrative

The post-game survey will also be used to identify players that would be willing discuss, via Google Hangouts, their perspective on the game and its technologies. These informal interviews will be used as supporting data.

About | Open Badges. (n.d.). Retrieved August 9, 2014, from http://www.openbadges.org/about/

Carr-Chellman, A. A. (2011). Instructional design for teachers: improving classroom practice.

New York: Routledge.

Digital badges - Wikipedia, the free encyclopedia. (n.d.). Retrieved August 9, 2014, from

http://en.wikipedia.org/wiki/Digital_badges

Duffy, T. M., & Jonassen, D. H. (Eds.). (1992). Constructivism and the technology of

instruction: a conversation. Hillsdale, N.J: Lawrence Erlbaum Associates Publishers.

elearnspace › MOOCs are really a platform. (n.d.). Retrieved from

http://www.elearnspace.org/blog/2012/07/25/moocs-are-really-a-platform/

Jonassen, D. H., & Land, S. M. (Eds.). (2012). Theoretical foundations of learning environments

(2nd ed.). New York: Routledge.

Pastula, P. (1999). Applying Learning Theories to Online Instructional Design. Pastula.com.

Retrieved August 9, 2014, from

6


http://www.patsula.com/usefo/webbasedlearning/tutorial1/learning_theories_full_version

.html

7

Documents

HuskyHunt Conceptual Design Document