Understanding Assessment Types - Evalion · interactive item types, leader boards, badges, levels, and multimedia.9, 10 Increased engagement from using these tools is predicted to

10

Back to Table of Contents

© 2017 Gartner, Inc. and/or its affiliates. All rights reserved. CEB171969 Detail about CEB Inc. and its subsidiaries, now part of Gartner, can be found at gartner.com/ceb-offices.

Understanding Assessment Types

https://www.cebglobal.com/about/contact-us/registration.html

11



Definitions of Assessment Types

Source: CEB analysis.

Newer assessment concepts such as gamification and serious games are often used interchangeably, but there are important differences that have implications for evaluating these tools’ usage in selection contexts. The table below describes these new assessment concepts within the context of traditional and multimedia-based assessments.

Applies one or more game elements to nongame contexts, such as assessments, to motivate and engage test takers

Gamified Assessments

Definitions of Assessment Types

Multimedia-based assessments designed to create a virtual environment to test real-life job situations; by simulating work behaviors and contexts, simulations can also provide a realistic preview of the job

Multimedia Simulations

Uses a game for purposes other than pure entertainment, such as for training, instructional purposes, or to achieve a real-life goal

Serious Games

Online or paper-pencil assessments of abilities or self-reported measures of attributes or behaviors; examples include cognitive ability tests, personality assessments, biodata, and situational judgment tests

Traditional Assessments


12



Gamification covers a wide range of tools and techniques to create an immersive, engaging assessment experience, or to improve the effectiveness of existing assessment methods.

Examples of game elements that can be applied to traditional assessments include storylines, progress indicators, feedback mechanisms, interactive item types, leader boards, badges, levels, and multimedia.9, 10

Increased engagement from using these tools is predicted to lead to achieving important business outcomes, such as employee knowledge retention, applicant attraction, market penetration, product awareness, and talent measurement.11

While gamification is designed to enhance traditional assessment methods by adding one or more game elements, serious games use many game elements that provide unique ways of assessing individual characteristics. For example, instead of asking candidates what they would do or what characteristics are most like them, a serious game uses “stealth assessment” to measure actions performed in a game environment.12

While gamification is designed to enhance traditional assessment methods by adding one or more game elements, serious games use many game elements

that provide unique ways of assessing individual characteristics.

Introduction to Gamification


13



The Value of Gamification

Organizations are attracted to gamified assessments, and they believe that these methods will increase candidate engagement, compared to traditional selection assessments.

Nevertheless, because gamification can incorporate different things, there is no simple way to tell how valuable it is. Research on gamification in selection contexts is new, and there are still many unanswered questions.

It’s also unclear what is measured in a game or whether it yields useful information about a candidate. For instance, some gamified assessments in the market allege that dozens of behavioral traits can be measured, such as risk propensity or perseverance. The measurement of such traits may take the form of a game where a candidate can either pump a balloon to potentially make more (fake) money or cash out before the balloon pops. This type of game is used as a way of understanding a candidate’s risk propensity. There are several issues with this approach.

First, it is not clear whether this measurement of risk propensity correlates with risk propensity indicators performed on the job, given the contrived nature of the assessment. This type of measure is also characterized as a “maximal” performance measure rather than a “typical” performance measure, because of the high stakes and perceived consequence. In most cases, organizations are interested in what a candidate is usually like, rather than what level of risk he or she will take in a very particular situation. It is

also unclear whether behavioral traits measured in a gamified assessment are good measures of these traits, which would normally be established through correlating these measures with established, valid measures. Finally, even though gamification may help overcome response biases commonly observed on self-report measures from high-stakes settings, a savvy job candidate could still guess what a gamified test is measuring and alter his or her responses accordingly.

For instance, if a candidate perceives risk propensity to be desirable for a particular job, he or she may continue to pump the balloon and exhibit “risky” behavior. There's still a lot to be discovered in this aspect of assessment methodology.

Research on gamification in selection contexts is new, and there are still

many unanswered questions.


14



Return on InvestmentGamification can be relatively inexpensive to implement, if the number and type of game elements are kept to a manageable level. Researchers suggest that adding a few targeted game elements to an existing assessment will probably produce a greater return on investment than producing an entire digital, serious game for the same purpose, because serious games require a gaming platform.9, 13

Candidate EngagementThere is insufficient research to conclude that candidates are more engaged or motivated to perform well on a gamified assessment, which would be the key reason to justify their use. For example, one CEB study applied several game elements to an inductive reasoning test: an immersive storyline with 3-D animation, immediate feedback during the test, and a drag-and-drop, interactive item type.14 The study found no differences in test scores due to gamification, and test takers in highly gamified conditions were not more motivated to complete the assessment than those in less gamified conditions. Instead, participants in the most gamified conditions showed less concentration and higher anxiety than candidates in less gamified conditions, which indicates that gamification can be taken too far. The researchers concluded that providing negative feedback while taking a selection assessment may increase anxiety, lower the opportunity to show their skills, and may reduce candidate motivation. Similarly, another study found that when cognitive assessments were framed as games, the perceived length of the test was reduced, but those who perceived the testing time to be shorter also reported lower test-taking motivation.15 These candidate reactions must be carefully considered when implementing new assessment methods.

Gamification can be relatively inexpensive to implement, if the number and type of game elements are kept to a manageable level.


15



Candidate Reactions

Perceived Fairness

Research shows that candidates’ perceptions of fairness in the selection process have important implications (e.g., job acceptance intentions, likelihood to recommend the organization to others, intention to pursue legal action, post-hire job satisfaction).16, 17, 18

The type of test in a selection process can affect perceived fairness of the process. Job relatedness (whether the test is perceived to measure constructs relevant to the job) is one of the most important aspects of perceived fairness.18 Candidates generally prefer job- related methods, which affect overall positive reactions.16, 17, 19 The perceived opportunity for candidates to demonstrate their knowledge, skills, and abilities is another important determinant of perceived fairness.20 Gamified assessments that bear little or no resemblance to the job run a higher risk of negative reactions, including perceived unfairness and inability to demonstrate job-related skills.

Our 2016 Candidate Experiences and Preferences Survey indicates that less than 10% of candidates indicate that they would prefer to take a gamified assessment as a part of their next hiring process. This percentage is significantly lower than the 50%–60% of candidates who prefer to complete assessments that are clearly job related, such as job knowledge tests and work simulations. This trend is present across all generations of

candidates, including millennials, who show a similar level of preference to gamified assessments as older generations.

Assessment LengthOrganizations often want to implement a short process to reduce the effort required from the candidate. However, research shows that assessments that are too short are generally seen as less credible than longer ones and may undermine the candidate’s perception of the opportunity to perform. Candidates are likely to feel that they have a better opportunity to perform if the assessment length sufficiently allows for their knowledge, skills, and abilities to be measured.21 In fact, individuals actually react more favorably to longer cognitive assessments than shorter ones.22 Specifically, longer cognitive assessments are perceived to be fairer than shorter ones, positively relate to organizational

Gamified assessments that bear little or no resemblance to the job run a

higher risk of negative reactions.


16



attractiveness, and positively relate to job offer acceptance. This finding is likely due to longer tests’ correlation with the perception of job relatedness and opportunity to perform, which are both important factors of candidate reactions. And all of these findings above suggest that candidates are willing to complete a more lengthy assessment process in exchange for the chance to demonstrate job-related skills and receive fair consideration.

Recently, the CEB 2016 Candidate Experiences and Preferences survey revealed that candidates prefer assessments that are not too long but not too short. Candidates most prefer assessments that take 10 to 30 minutes to complete and least prefer short assessments (e.g., less than five minutes) or very long assessments (e.g., over an hour).

Research shows that assessments that are too short are generally seen as less credible than longer ones and may undermine the candidate’s perception of theopportunity to perform.


17



of job candidates believed that completing a test on a mobile device would provide a fair, accurate testing

23%

experience.

Delivery As ownership of smartphones increases, so has the desire for organizations to implement mobile-enabled assessments.22 The device on which assessments are offered is an important consideration. Some potential benefits of offering mobile-enabled assessments include reaching more diverse candidates and making the testing process more convenient.24 Early research on candidate reactions to mobile assessment showed that complex, high-stakes assessments are perceived less favorably when completed on a mobile device than on a computer. One study found that various tests were perceived to be less fair and more difficult when taken on a mobile device than on a computer.25

Furthermore, less than a quarter (23%) of job candidates believed that completing a test on a mobile device would provide a fair, accurate testing experience.26 Participants reported more positive reactions overall,27 greater ease of the testing experience, and a better chance to perform when completing a test on a computer rather than a mobile device.5

The CEB 2016 Candidate Experiences and Preferences Survey supports the finding that candidates prefer to complete assessments on a computer more than on a mobile device. Specifically, candidates most prefer to complete traditional, questionnaire-style assessments on a computer. The least-preferred delivery method mixes two types of technology, such as gamified assessments delivered on a mobile device. This trend is stable across generations, with millennials exhibiting the same preferences as older generations.

More recently, mobile assessment has evolved to use more innovative, mobile-first strategies for assessment design. This change is intended to overcome some of the unfavorable candidate reactions to early mobile assessments that were simply sized down to fit a smaller display. New methods of mobile assessment utilize native mobile functionality (e.g., swiping) and more familiar, app-like features to present a positive candidate experience. Research using mobile-first design features has recently shown positive candidate reactions and psychometric equivalence with traditional (e.g., computer-based) approaches.28

Mobile assessment has evolved to use more

innovative,mobile-first strategiesfor assessment design.


18



Validity of Gamified Assessments

Validity is another key consideration when evaluating new assessment methods for use in selection contexts.

Before using a gamified assessment for hiring purposes, it is important to gather evidence that performance on the assessment will lead to desired business outcomes.

The validity of a gamified assessment depends, in part, on how the assessment is designed and which game elements are used. When packaging valid, established assessment content into a more engaging experience, the validity lies in the underlying content and is not likely to change with different packaging. Examples of innovative, realistic job previews that apply game elements in this manner include New Zealand Air’s situational judgment test and Heineken’s interactive recruitment video, which includes personality items. People tend to react more favorably to packaging like Heineken and New Zealand Air than to multiple-choice items, as long as the content is job relevant.

Game elements also affect validity. When selecting or designing a gamified assessment, define the business objectives by using the assessment (e.g., attracting candidates best fit to the job), identify the target behaviors of the test taker and how the behaviors will be measured, and select the appropriate game elements and tools. Following these steps will help you meet the desired objectives and foster the target behaviors.10

Selecting a game element that counters the business objectives or target behaviors could have a negative impact on the validity of an existing assessment. For example, game elements designed to trigger feelings of competition (e.g., leader boards that rank candidates) may increase candidate levels of anxiety, which could affect how the candidate performs on the assessment. Researchers also suggest that the number of game elements can have a measurement impact. The more game elements are employed, the higher the risk of measurement contamination, and the harder it can be to interpret behavior from the assessment.29 There is a greater probability of introducing error into assessment scores when technology provides greater levels of autonomy to candidates.30 This probability is one reason that serious games may pose a greater risk to organizations when used for high-stakes assessment—compared to applying a few targeted game elements to established assessments.

http://goplaces.theheinekencompany.com/en/age-gate

https://quiz.weirdlyhub.com/quiz/airnz/be-the-next


19



Serious Games

Developing a serious game to directly observe behaviors is more complicated and resource intensive than including gamified elements in a proven assessment.

Historically, some of the most common business applications of serious game designs have been training and skill development; behavior change (e.g., promoting exercise, buying a product); or problem solving and innovation.10

Less is known about the utility and predictive value of serious games in a selection context. Developing a serious game requires substantially more cost, effort, and a lot of data to provide evidence that a new and innovative “game” works to predict job performance. In fact, some foundational design elements of serious games may be counterproductive in high-stakes selection contexts. One such measurement challenge from serious games is practice effects.

Practice EffectsOne core design characteristic of serious games is that it encourages trial-and-error learning with problem-solving strategies,30 which is often accomplished by allowing for repeated play. What we know from research and experience is that a person tends to get better at games after playing them several times. When performance improves with practice, it makes for a poor assessment because the scores are unstable. This instability is also problematic because practice effects could lead candidates to change their test-taking strategies in order to “game the system” and

Gamified Assessments Versus Serious Games

Gamified Assessments are traditional assessments enhanced with one or more gamified elements (e.g., storylines, progress indicators, feedback mechanisms, interactive item types, leader boards, badges, levels, multimedia).In contrast, serious games provide new ways of measuring behaviors and individual characteristics in a game environment. Serious games often share similar characteristics as entertainment video games but are designed for non-entertainment purposes. For example, serious games can involve quests, challenges, or interaction with the game environment to progress through various levels.


20



improve performance, to increase their chances of being hired.

There is also a potential for bias in favor of those who regularly play digital games. If we developed a game similar to other games in the market, people who play these similar games often may score higher than those who do not play digital games. The objective of selection assessments is to evaluate candidates on the competencies, skills, or abilities that lead to desired business outcomes (e.g., successful job performance, hiring candidates who fit the organization, candidates who remain on the job). The game design should not lead to an advantage or disadvantage for any subset of candidates based on characteristics that are not related to job performance, such as frequent game use in non-work contexts.11

If we developed a game similar to other games in the market, people who play these similar games often may score higher than those who do not play digital games.

Return on InvestmentMany organizational users expect assessments to be tailored or customized to a very specific purpose, (e.g., the job context, the specific requirements of the job, the company values and mission, the company’s product portfolio). The pace of change in organizations and rapidly changing technology, however, may result in a serious game having a limited shelf life. These types of tools require continual updates and investment, to ensure they continue to reflect the current organizational environment and remain fit for purpose. The rate of change and the high cost needed to continually update gamified assessment make the investment very substantial for serious games and gamified assessment.


21



Failure to implement updates to reflect changes in the job could

It would be

to culturally adapt a serious game.

of the assessment.

challenging and expensive

reduce the life-span

The expectation is that games will be slick and fun, so there is a high design standard and high costs to keep up with continuing technology advancements. The development cost will be affected by the need for game designers and the type of media format used. Multimedia technology can have a short shelf life, requiring frequent updates. For example, America’s Army (www.americasarmy.com), a successful serious game designed as a realistic job preview for the US Army, has had numerous updates since it was first developed.

While some updates in serious games may be needed to reflect more modern expectations for digital game interfaces and animation, in high-stakes assessment, revisions may also be required to reflect adaptations to changes in the job or work environment if the game is meant to be highly realistic. To minimize the need for such frequent revisions, a serious game in high-stakes assessment would need to be generic and agnostic to the job, to use it across many jobs. However, this generic format could result in perceived lower job relevance which, in turn, may result in poor user reactions. Failure to implement updates to reflect changes in the job could reduce the life-span of the assessment.

In addition, it would be challenging and expensive to culturally adapt a serious game and all elements of the game in a way that maintains equivalency for how the competencies, skills, or abilities are measured.11 In comparison, repackaging an existing assessment into a gamified assessment that includes one or a few game elements would be more straightforward and less resource intensive.


22



Validity of Serious Games

Some researchers argue that the large number of data points obtained from serious games or highly game-like assessments will allow for better alignment between predictors and criteria.1

However, there is little research showing success in using serious games to predict job performance. It is likely possible but would require R&D time, a significant investment, and some risk (e.g., poor reactions), so we cannot make a definitive ROI case for pursuing serious games.

The jury is still out on adverse impact and incremental validity of serious games over traditional or gamified assessments. To reduce the risk of adverse impact, minimize potential demographic differences in selection scores when using serious games. Reduce complexity in the game structure and user interface, and make it easy to learn how to play the serious game; doing so will help you reduce the potential bias of prior gaming experience and decrease the probability that differences across demographic groups will be an issue.10

Right now, there is more favorable support to gamify an assessment with established validity than using serious games for selection. Assessment providers and developers need to look into the technology and research based on this method before adopting this technique, given the lack of evidence and high development costs. Lack of familiarity with the serious games interface could result in poor assessment results, even if the candidate would ultimately perform well in the job.


23



Media Format Considerations

The choice of media to use can affect various assessment properties.

People tend to associate serious games or gamified assessments with animated visuals.

However, there are several media formats that can be used. For example, a serious game could be entirely text based with key elements of scenes being described and the player responding to the situation by typing what action they would take. This format was used in some of the early adventure games available for personal computers in the 1980s. The main media options available are text, audio or voice-over, still images (e.g., photographs, drawings, computer-generated images), and multimedia formats. Multimedia formats include 2-D and 3-D animation and live-action video. Each format brings its own advantages and disadvantages to the table. While it is possible for these formats to be mixed and matched within a single assessment, discernment should be used to avoid creating a “Frankenstein” assessment that lacks a unifying feel.

The choice of media to use can affect various assessment properties, and organizations may place differing priorities on these features.31 So before deciding on a media format, it is important for organizations to prioritize which properties of an assessment are the most important. When deciding on which media format to adopt, the impacts of the format on these various features need to be weighed against each other and balanced with the costs. At times, a particular media format may enhance several of the priority properties, while in other situations, a format may enhance one important feature but deter from another. Some of the key properties are highlighted below.

In this section, we describe different ways in which media are embedded in assessment and considerations for the use of various formats. The use of media in assessment may allow organizations to achieve their goals of high validity and candidate reactions.

Psychometric PropertiesPsychometric considerations focus on whether a particular media format will contribute to or detract from the assessment’s ability to accurately measure interest and to effectively predict performance. As a general principle, if a media format introduces an irrelevant element (e.g., one that influences the candidate’s performance on the assessment but doesn't predict the outcome of interest), then that format is likely to deter accurate measurement. On the other hand, if a media format removes an irrelevant feature, then it is likely to enhance measurement accuracy. For example, performance on text-based assessments can be influenced by the candidate’s reading speed and comprehension.32 If the behavior the assessment is designed to measure does not require a high level of reading comprehension (e.g., a retail sales role), then


24



The use of

audio, animation, or live-action videocould reduce the reading demands of gamified assessment.

text-heavy gamification elements may hamper accurate prediction. Text-heavy gamification elements could include introductive or transitional narratives, detailed scoreboards, or performance feedback. The use of audio, animation, or live-action video could reduce the reading demands of gamified assessment. However, these media types could introduce other irrelevant factors. The use of visual media requires that the demographics of characters in assessment scenarios be defined. A text-based scenario could simply refer to a customer entering the store. If the customer is portrayed in an animation, then the race, gender, age, style of dress, body type, and hair color all need to be presented and carefully considered, as any one of these features could influence how the candidate responds to the scenario. The media format chosen should minimize irrelevant features.

ValiditySome limited research indicates whether media-based assessment produces the same or higher levels of validity compared to traditional assessment. One study found that video-based, situational judgment tests had higher validity in predicting job performance than text-based versions of the same content.33 This increased validity may be due to video-based situational judgment tests looking more like the job than text-based assessments. Less research is available on comparing the validity of assessments that use animation.


25



Candidates generally prefer assessments that include

multimedia components.

Adverse ImpactVideo-based, situational judgment tests have been found to have a lower correlation with cognitive ability assessments than text-based situational judgment tests. This gap could be due to the lower reading demands that video-based assessments place on the candidate. This lower association appears to result in video-based tests having less adverse impact on minority groups than text-based assessments.32 While more research is needed in this area for animated assessments, to the extent that they also reduce cognitive demands on candidates, it is expected that they would also show less adverse impact.

Candidate EngagementAfter viewing the same situational judgment item in three media-rich formats (e.g., video, 2-D and 3-D animation), people rated the video-based version as more engaging than both the 2-D- and 3-D-animated versions. They did not indicate a difference between the two animated versions.34 This finding suggests that to increase candidate engagement in the assessment process, organizations may want to consider using video rather than animation.

Candidate Reactions to Media FormatsAs discussed above, job relatedness affects candidates' perception of an assessment's fairness. It has been found that for situational judgment tests, people viewed questions presented in a 3-D animation format to be more job related than the same questions presented in computer-based text. When the assessments using different media-rich formats were compared with each other, video-based assessments were perceived as more job related than either 2-D- or 3-D-animated assessments.

For job relatedness, no differences were seen for 2-D- or 3-D-animated assessments.33

Another aspect of a candidate’s perception of fairness is the opportunity to demonstrate his or her knowledge, skills, and abilities. Research has found that use of 3-D animation did not increase candidates’ perception of their opportunity to perform on an assessment; even so, including a 3-D-animated assessment in a test battery with a personality assessment increased candidates’ perception of their opportunity to perform on the personality assessment (compared to including a computer-based text assessment with the personality assessment).34 Based on recent, limited research, there is some information indicating that including a media-rich assessment in a battery of other assessments may improve candidate perceptions of text-based personality assessments.

Various studies have indicated that candidates generally prefer assessments that include multimedia components rather than only text-based assessments.35 When asked about their general preference among the various types of media-rich assessments, people ranked video-based assessments highest, followed by 3-D-animated and then 2-D-animated assessments. The same pattern was found for the degree of realism created and the positive impression of the company using the assessment.34


26



Face ValidityFace validity refers to the extent that an assessment appears relevant to the situation. Research has indicated that the perception of face validity of situational judgment tests is higher when they are delivered in a video format rather than in text.32 Based on a composite of ratings on positive impression, information provided, realism, and general preference, live-action videos were ranked first and 3-D animation second, both being ranked above text-based content.35 So, in many cases, multimedia formats could enhance face validity. However, if a job is very text oriented (e.g., report writing) a text-based assessment may hold more face validity.

AccessibilityIn the interest of fairness, diversity efforts, and compliance with relevant laws, organizations need to consider what modifications may be necessary to make an assessment accessible to individuals with disabilities. With text-based assessments, an accommodation might be able to be achieved via the use of a screen reader. Still images, animations, and live-action videos may add challenges to the accommodation process, perhaps requiring elements such as closed captioning or audio descriptions of scenes and actions portrayed in these media.

Practical ConsiderationsThe process for designing multimedia assessment content is more involved than designing text-based assessment content, as the multimedia content must be “brought to life” with scripts, visual, and audio details. Once the script is created, the media must be produced, which involves selecting actors and creating scenes. This process is generally done in collaboration with live-action or animation production specialists. These challenges add time and expense to the assessment development process. Many situations

can drive the need to modify content, such as localizing for another culture, updating actor or avatar clothing to match changing fashion trends, and custom branding. Text-based assessments are generally easiest to update, as the visual and audio elements do not need to be addressed. Animation tends to allow more flexibility in updating content than live-action video.

Company ImageOrganizations may expend considerable effort and expense to create and maintain a desired public image. A company’s image can influence its attractiveness to potential candidates.36 A company may choose to use a particular media format because it aligns with its broader corporate image. Some companies may think that text-based assessments make them look outdated, while other companies may hold the perspective that text-based assessments communicate a message of stability. What candidates expect in an assessment also affects what to consider when it comes to the company image. Candidates may not think twice about taking a text-based assessment to join a legal firm but would be surprised if given such an assessment for joining a company in the forefront of technology.

screen reader.

With text-based assessments, an accommodation might be able to be achieved via the use of a


27



Diversity RepresentationIf companies want to recruit from diverse demographics, they should want their assessments to reflect that diversity. While text-based assessments can refer to demographic characteristics in narrative-based questions, the multimedia formats lend themselves a bit more naturally to reflecting diversity. With multimedia formats, actors or avatars can be selected to represent diversity of race, gender, age, and other characteristics without having to make an explicit reference to it. “Background” diversity can be added with secondary characters in the background of the scene. Here, animation has an advantage over live-action video, as background characters can be added without having the expense of additional acting talent. However, when using animation to represent diversity, take care that the avatars do not default to unfavorable stereotypes of diverse characteristics.


28



Conclusion

Innovations in technology-enabled assessment and the rise of the candidate centric market have spurred organizations to rethink their talent assessment practices. Gamified assessment offers promise for engaging and retaining candidates, imparting a positive employer brand, and tapping predictive information about candidate potential. The research, however, is still in its infancy and does not yet fully support several of the key criteria for judging the usefulness of assessments.

Even though gamified assessment is interactive or engaging, it is more costly to develop and maintain, and users should be aware that the benefits may not be commensurate with the cost. In certain situations (e.g., realistic job previews), gamification is a feasible, useful way of engaging candidates and providing a positive experience.

Research has yet to strongly support the significant investment and use associated with gamified and serious games. Technology-enabled assessment is a highly dynamic field. As more research accumulates on gamified assessment, more definitive conclusions can be drawn regarding its effectiveness.

Multimedia assessments provide an option to balance candidate engagement with sound measurement, compared to other newer, untested assessment types. In fact, multimedia assessments offer many proven advantages. For example, video- based, situational judgment tests offer stronger prediction of job performance than text-based versions of the same content.

We encourage test users and consumers to inquire and evaluate the effectiveness of various assessment tools and ensure they address candidate expectations for a fair, relevant, and efficient assessment experience. They should also evaluate requirements for predictive, legally defensible, affordable, and scalable testing programs.


29



Our Recommendations

Source: CEB analysis.

Analyze further to ensure job-related skills are being measured.

Gamified Assessments

• Validity: Validity of underlying assessment content is likely to remain when packaged with gamified elements.

• Adverse Impact: More research is needed.• Cost: Adding gamified elements to

existing assessments can be fairly inexpensive.

• Reactions: Gamification may motivate and engage candidates but can also increase anxiety. More research is needed.

Proceed. Simulating work behaviors and contexts provides a realistic preview of the job.

Multimedia Simulations

• Validity: Simulations can increase face validity and predictive validity.

• Adverse Impact: Audio, animation, or video minimizes reading demands and is likely to reduce adverse impact.

• Cost: Multimedia simulations are resource intensive to develop and maintain.

• Reactions: Candidates generally prefer multimedia assessments and perceive them as more job related.

Caution! Untested validity is risky, especially in high-stakes assessment. Early research leaves many questions unanswered.

• Validity: Little is known about the utility and predictive value of serious games for selection. Ensure the game directly measures the competencies, skills, or abilities that lead to the desired business outcome.

• Adverse Impact: More research is needed.• Cost: Serious games are costly to develop

and have a shorter shelf life. Benefits may not be worth the costs.

• Reactions: Low job relevance is likely to lead to poor candidate reactions.

Serious Games

Evolve current approaches that are backed by extensive research on validity, adverse impact, cost, and candidate reactions, to incorporate technology and create the experience candidates expect today.

Traditional Assessments


Documents

Understanding Assessment Types - Evalion · interactive item types, leader boards, badges, levels, and multimedia.9, 10 Increased engagement from using these tools is predicted to