1.Improving Training Outcomes Using Pre-training Scripts a Theory of Planned Behavior Approach

7/31/2019 1.Improving Training Outcomes Using Pre-training Scripts a Theory of Planned Behavior Approach

1/18

Information and Organization 12 (2002) 135152www.elsevier.com/locate/infoandorg

Improving training outcomes using pre-trainingscripts: a theory of planned behavior approach

R. Santhanam

DSIS Area, School of Management, Gatton School of Business and Economics, University of Kentucky,

Lexington, KY 40505-0034, USA

Abstract

Prior research has shown that high levels of behavioral intentions to use a system lead toincreased use of information systems. Therefore, researchers have emphasized the need toidentify methods that can enhance users intentions to use a system. Training programs areviewed as one avenue to improve users intentions to use a system. Two important goals ofa training program are to enhance a trainees intentions to use the target system and improvetheir level of system proficiency. This study postulates that the Theory of Planned Behaviorcould be used to guide the development of belief manipulation pre-training scripts. These pre-training scripts can be used to positively influence trainees beliefs regarding the target systemleading to higher intentions to use the system and in improved levels of system proficiency.The results of a controlled laboratory experiment indicate that subjects who received the beliefmanipulation pre-training script showed significantly higher intentions to use the target systemwhen compared to subjects who did not. The belief manipulation pre-training scripts, however,did not seem to improve the level of system proficiency of trainees. 2002 Elsevier ScienceLtd. All rights reserved.

Keywords: Training; Scripts; Theory of Planned Behaviour

1. Introduction

Training programs are seen as an important avenue to improve effective use ofInformation Systems (IS) in organizations. The main objective of these training pro-grams is to enhance end-user motivation and develop their skills to use the system

Tel.: +1-859-257-4397; fax: +1-859-257-8031.

E-mail address: [email protected] (R. Santhanam).

1471-7727/02/$ - see front matter 2002 Elsevier Science Ltd. All rights reserved.

PII: S 1 4 7 1 - 7 7 2 7 ( 0 2 ) 0 0 0 0 3 - 9


2/18

136 R. Santhanam / Information and Organization 12 (2002) 135152

effectively (Nelson, Whitner, & Philcox, 1995; Compeau, Olfman, Sein & Webster,

1995). Large organizations expend millions of dollars for employee training with a

majority of these expenditures dedicated to training in IS (Van Buren, 2000). Despite

the large investment of resources, empirical studies indicate that these training pro-grams have had limited success because, in their jobs, trainees only use a small

fraction of the skills that they learned (Olfman & Bostrom, 1991; Olfman & Mandvi-

walla, 1995a; Venkatesh, 1999). One possible reason is that research studies on end-

user training have focused primarily on identifying methods to improve a traineessystem proficiency rather than their intentions to use the system (Davis and Bostrom,1993; Compeau and Higgins, 1995; Simon, Grover, Teng, & Whitcomb, 1996; Wied-

enbeck & Zila, 1997; Lim, Ward & Benbasat, 1997).

Although a trainees intention to use the system is often mentioned as a desirableoutcome (Sein, Bostrom, & Olfman, 1987; Bostrom, Olfman, & Sein, 1990), limited

attention has been paid to enhancing a trainees intention to use the system. Bothresearchers and practitioners have concluded that enhancing a trainees intention touse the system must be a part of the training program (Shayo & Olfman, 1993;

Hubbard, 1999). Improving a trainees intentions to use the system is considered acritical outcome because it directly impacts the acceptance and usage of the system

(Davis, Bagozzi, & Warshaw, 1989; Szajna, 1996). Thus, a training program must

not only enhance trainees system proficiency skills but also their intention to usethe system thereby leading to effective use of the target system. Research on several

other IS topics have also stressed the importance of improving users intentions to

use a system. Theoretical representations that predict IS usage patterns, such as theTechnology Acceptance Model (TAM) and the Theory of Planned Behavior (TPB),

posit a direct relationship between behavioral intentions to use a system and actual

usage of a system (Azjen, 1985; Davis et al., 1989; Mathieson, 1991; Taylor &

Todd, 1995). Derived from the theory of reasoned action, the TPB proposes that a

persons behavior (of using the system) is directly related to their behavioral inten-tions (to use a system). Behavioral intentions are a function of attitudes toward thesystem, subjective norms and perceived behavioral control, which are in turn a func-

tion of behavioral, normative and control beliefs (Fishbein & Azjen, 1975; Azjen &

Fishbein, 1980; Azjen, 1985). From the TPB, it is clear that if users can be influenced

to acquire high levels of positive behavioral, normative and control beliefs regardinga system, they will develop high intentions to use the system and thus increase their

actual usage of the system. According to Petty and Cacioppo (1981), The theoryof reasoned action makes it clear that any influence attemptwhether the goal is tochange attitude, norm, intention or behaviormust always be directed at one ormore of the individuals beliefs. (p. 200)

A belief is defined as what a person knows about something; it can be changedby eliminating old beliefs or by introducing new beliefs (Petty & Cacioppo, 1981).

Users enroll in a training program to gather new information about a system. The

trainers and the training program are viewed as credible sources for providing this

information. Hence, a training program provides an ideal opportunity to inform usersabout the benefits and possible outcomes of using the system and influence theirbeliefs about the system. The information provided in the training program could


3/18

137R. Santhanam / Information and Organization 12 (2002) 135152

lead users to develop high positive beliefs about the system, which in turn could lead

to higher intentions to use the system. Furthermore, having high levels of behavioral

intentions to use a system may also serve as a motivating factor to learn to operate

the system more effectively. Noe (p.g 737, 1986) describes motivation in a trainingenvironment as a force that energizes and influences enthusiasm about the programand directs participants to learn. Hence, users can be expected to pay greater atten-tion to training, learn more about the system operations and develop high levels of

system proficiency. We therefore expect that the TPB can be utilized to developmessages that can be provided to users in a training program to influence their beliefs,their intentions to use the system and their system proficiency levels.

2. Theoretical background

2.1. Prior research

Empirical research has constantly reinforced the conclusion that end-user training

is a key factor influencing the effective use of IS (Thompson, Higgins, & Howell,1991; Lee, Kim, & Lee, 1995). Based on general models of training, Bostrom et al.

(1990) identify two immediate outcomes of an IS training program, namely, perform-

ance and motivation. The performance outcome deals with improving a traineessystem proficiency skills while the motivational outcome deals with enhancing a

trainees intention to use the system. The goal of an end-user training program is toproduce a motivated user who has the basic skills to use the system effectively(Compeau et al., 1995). Although research has identified at least two explicit out-comes, end-user training programs have traditionally been defined in terms of skill(Sein, Bostrom, & Olfman, 1999). Consequently, most of the research on IS training

has also focused on enhancing a trainees level of system proficiency. However,several studies have begun to highlight the value of investigating motivational factorsin training (Olfman & Bostrom, 1991; Webster & Martocchio, 1992; Davis & Bos-

trom, 1994; Compeau & Higgins, 1995; Webster & Martocchio, 1995; Venkatesh,

1999; Venkatesh & Speier, 1999). This is based on the premise that motivational

factors not only impact performance outcomes of training but also affects continueduse of the system in the workplace (Davis et al., 1992).

Several different approaches and outcomes have been utilized to examine the role

of motivational factors in IS training. Research has shown that individual trainee

characteristics such as microcomputer playfulness and cognitive spontaneity have a

motivational effect and can positively impact the learning outcomes of training

(Webster & Martocchio, 1992; Davis & Bostrom, 1994). Similarly, a direct manipu-

lation interface as opposed to a command driven interface provides greater intrinsic

motivation to the trainees and has a positive effect on learning outcomes (Davis &

Bostrom, 1994). Behavior-modeling-based training methods are effective in improv-

ing a trainees perceptions of self-efficacy and can enhance a trainees skilled per-formance (Compeau & Higgins, 1995). Providing opportunity scripts informing sub-

jects that using microcomputers would enhance their job opportunities can result


4/18


in a higher learning performance (Martocchio, 1992). However, optimistic training

previews carrying the message that the software would be easy to learn and prove

useful in their job increased a trainees attention to performance but did not impact

learning outcomes (Webster and Martocchio, 1995).Beyond the link between motivation and performance outcomes, research has

started to examine whether training sessions can be used as a direct platform to

enhance a users motivation to use the software. Venkatesh and Speier (1999) foundthat trainees with a positive mood, induced by watching humorous commercial video

clips during training, developed short bursts of greater intrinsic motivation and higherintentions to use a technology. The TAM model (David Bagozzi & Warshaw, 1989)

has been used to guide the development of training interventions based on the prem-

ise that a trainees system related beliefs can be influenced in a training session,which in turn can impact their intentions to use a system (Olfman, Bostrom & Sein,

1991; Venkatesh, 1999). Olfman et al. (1991) found that subjects who received for-

mal training in the use of an e-mail system formed higher perceived usefulness

beliefs about the system than those who did not receive formal training. Venkatesh

(1999) developed a special system to increase trainees perceived ease-of-use beliefsabout the system. This special system permitted the inclusion of game-based training

methods. Results showed that subjects in the game-based training group developed

higher ease-of-use beliefs and higher intentions to use the system than subjects in a

traditional training group. However, subjects in the game-based training group did

not exhibit higher learning performance.

These findings suggest that trainees intentions to use a system can be influencedduring a training session and this could also result in positive performance outcomes.The evidence however, is not conclusive. The TAM model could be used to guide

the development of training interventions. However, this approach requires the

manipulation of perceived usefulness and ease-of-use beliefs that are dependent on

the design features of the system. It should be noted that the system used by Venka-

tesh (1999) was developed in-house and specifically designed to facilitate theinclusion of a game-based training environment that would increase a users ease-of-use perceptions. Commercially available systems typically do not have such

design features and in general it may be difficult to modify design features of existing

systems. Instead, organizations may have greater latitude in providing recognitionand support services to facilitate the use of existing systems. Hence, other types of

training interventions that can be easily implemented and applied to existing systems

and to any training environment must be developed. The use of the TPB to develop

training interventions that can influence intentions to use a system is an alternativethat deserves consideration.

2.2. Research model and hypotheses

The TPB proposes that an individuals behavior can be predicted by his or her

intentions to behave in a specific manner (Azjen, 1985; Azjen, 1991). Research hasshown that the TPB is able to effectively model and predict end-user behavior per-

taining to system use (Mathieson, 1991; Taylor & Todd, 1995). The next logical


5/18


step would be to utilize the TPB to develop interventions that can influence an indi-viduals beliefs in order to increase their intentions to use the system (Hill, Smith, &Mann, 1987).

TPB posits that intentions to behave are influenced by an individuals 1) Attitudetowards the behavior (A), which is a direct function (sum product) of his/her

behavioral belief and outcome evaluation of that belief; 2) Subjective Norms (SN),

which is the sum product of his/her normative belief and motivation to comply with

the wishes of referent others; and 3) Perceived Behavioral Control (PBC), which is

a sum product of control beliefs about the availability of certain resources such ashelp centers and the perceived facilitation of resources important in using the system.

Because the attitude toward the behavior is a sum product of the behavioral belief

and outcome evaluation of all beliefs, it will be dominated by the behavioral belief

and outcome evaluation pairs that have the highest values. Hence, if we are able to

identify those outcomes that individuals most value, and if we are able to induce

high levels of behavioral belief for these outcomes, the resulting attitude toward the

behavior will also be very high. Similar reasoning applies to increasing normative

and control beliefs regarding the target system. High levels of behavioral, normative,

and control beliefs can result in high behavioral intentions.

It is important to note that for purposes of enhancing behavioral intentions to use

a system, which is the objective of this study, it is not necessary to assess the levels

of attitudes, subjective norms and perceived behavioral control. If the relevant

behavioral beliefs, normative beliefs and control beliefs are high, then according to

the tenets of TPB, it should lead to high intentions to use a system. According toPetty and Cacioppo (1981), to change an attitude, it is necessary to modify theinformation on which the attitude rests. It is generally necessary to change a person sbeliefs (p. 184). Further, an alternative procedure for assessing attitudes, then,would be to measure the salient beliefs that a person has about the attitude object(p. 194, Petty & Cacioppo, 1981).

It has been noted that other people/external sources comprise one important sourceof user beliefs and that beliefs formed in this manner are similar to those formed

by personal observation (Fishbein & Azjen, 1975; Salancik & Pfeffer, 1978). Results

of studies on Persuasion theory and attitude change suggest that messages people

receive from a credible source tend to have a high persuasive effect (Zimbardo &Ebbesen, 1970; Petty & Cacioppo, 1981). Users attend a training program because

they are interested in learning how to use a specific system and the trainer is viewedas a credible provider of information about the system. Hence, a training program

provides an ideal opportunity to provide information that persuades users to formcertain beliefs about a system.

Training programs convey their message either verbally (through the trainer) or

through written material. Such written material are often referred to as scripts, labels

and previews (Martocchio, 1992; Webster & Martocchio, 1993; Webster & Martoc-

chio, 1995). In this study, we use the term scripts. These scripts can be provided

either within the training session (during the study phase) or prior to training as pre-training scripts (forming part of a pre-training strategy) (Baldwin & Magjuka, 1997).

Compared to using scripts within the training session, using pre-training scripts pro-


6/18


7/18


lead to outcomes they value or find attractive (Burton, Chen, Grover, & Stewart,1993; Snead & Harrell, 1994). Based on this, it can be expected that trainees who

have high behavioral intentions to use a system will expend greater effort to learn

to use the system because it will help them to engage in an outcome they value,namely, the behavior of using the system. Hence, they can be expected to attach

greater importance to and pay greater attention to training material than those who

do not have high intentions to use the system. If such effort is supported by good

training, these trainees can be expected to perform well in the training program.

Hence, we also test the following hypothesis:

H4: Subjects in a training program who receive belief manipulation pre-training

scripts intended to enhance their behavioral, normative and control beliefs about

a system will exhibit higher learning performance than subjects who receive a

pre-training script that does not seek to influence their beliefs.

3. Research method

3.1. Procedures

A laboratory experiment was conducted to investigate the proposed hypotheses.

As described earlier, the objective of the experiment is to manipulate the beliefs of

the trainees regarding the target system by using a pre-training script. It was thereforeimportant to identify a target system that could gauge the subjects outcome expec-tations and perceived facilitation factors prior to training. It also had to be a system

whose features were such that subjects could relate to the messages in the script. E-

mail systems with their many functions fit these criteria and have been used in severalIS training research studies (Santhanam & Sein, 1994; Lim et al., 1997). Further,

the Vax e-mail system used in this study provided an easy way to capture all thesubjects keystroke operations, which were later scored manually to measure theperformance outcome. The trainees were students enrolled in an introductory IS class,

which provided a large pool of subjects.

In order to develop the belief manipulation pre-training script, it was necessaryto identify trainees most valued outcomes, the groups that they had a motivation tocomply with, and the resources that they considered most important in effectively

using the technology. Prior to the actual experiment, an open-ended survey was

administered to a group of undergraduate IS students. Fifty-four items were identifiedfrom this survey and a second survey was developed using these items. The second

survey was administered to a group of 120 students who were all undergraduate IS

students and very similar to the target population. Using a 5 point Likert scale rang-

ing from Not at all important to Extremely Important, the respondents rated theimportance of these items in relation to the use of e-mail. Those items receiving an

average score greater than 4.0 were selected as the most valued outcomes. The surveyidentified the following six items as the most valued outcomes for the target popu-lation:


8/18


1. The ability to communicate with the instructor outside of class or office hours,2. The ability to submit assignments from home,

3. The ability to get information regarding the class at home,

4. The ability to receive messages without being present,5. The ability to send messages to people in other parts of the world at no cost, and

6. The ability to receive information regarding the class, without attending class.

The survey identified five groups of people as those with whom subjects had ahigh motivation to comply with:

1. Instructors,

2. Superiors at work,

3. Group or project members,

4. Prospective employers, and

5. Subordinates at work.

Finally, the survey identified the following five resources as important facilitatingfactors:

1. Support Services,

2. Good manuals,

3. Training,

4. Easy to use software, and5. Continuous Availability and Access.

These 16 items represented the basis for the manipulation of beliefs of trainees

in the actual experiment. The belief manipulation pre-training script emphasized the

importance of the 16 identified items as they related to e-mail. For example, one ofthe most valued outcomes identified in the target group was the ability to receivemessages without being present. The belief manipulation pre-training script included

sentences such as Unlike telephone conversations, one does not have to be present toreceive messages to describe a feature the e-mail system provided. Another sentence

relating to the valued outcome of sending messages without any cost, read as: Mess-ages that are e-mailed to any part of the world generally cost nothing to the user.Such sentences in the script would serve the purpose of increasing the subjectscorresponding behavioral belief. Another pre-training script describing the general

history of computers served as a control script. This control script did not refer to

any of the items identified in the surveys. The two scripts were of equal length, andthe time required to read the scripts was approximately the same.

The experimental procedure was as follows: Under-graduate business students who

were enrolled in an introductory IS class were requested to volunteer for the experi-

ment. Subjects who volunteered for the experiment were randomly assigned to either

the control group or the treatment group. A background questionnaire was firstadministered to gather biographical information and identify each subjects priorexperience with e-mail and computers. After completing the background question-


9/18


naire, the subjects in the treatment group were given the belief manipulation pre-

training script while the subjects in the control group were given the control script.

Note that both groups were in the same room and were identified only by different

computer account numbers. Subjects were not aware that they were receiving differ-ent scripts. The trainers also were not aware which subject received a specific typeof script.

After the trainees had read the scripts, they were asked to complete a questionnaire.

Two different types of questionnaires were used. The first type of questionnaire(TPB1 survey) consisted of sixteen questions that measured trainees relevant beliefsregarding the system and three questions that measured their intentions to use the

system, prior to training. The questions were anchored on a five point Likert scale.This was administered to subjects in both the control and treatment groups. To ensure

that there was no confounding effect and to gauge the true effects of the treatment,

a Dummy questionnaire was used as an experimental control to eliminate the possi-

bility that in asking questions regarding beliefs about the system, the questionnaire

itself could influence subjects beliefs. The Dummy questionnaire consisted of a setof true/false questions followed by the same three questions as in the TPB1 survey.

The true/false questions in the Dummy questionnaire were based on the type of

script (treatment or control) the subject had received. The Dummy questionnaire was

administered to 20 randomly chosen subjects from each group.1

The subjects were then provided with training on the e-mail system. A training

script was provided to all subjects. This script contained a conceptual model of the

system, procedural methods to operate the system, a list of commands frequentlyused in the Vax e-mail system and a list of frequently occurring error messages. A

conceptual model describes the components of the system and their inter-relationship

using metaphors or an abstract hierarchy. For example, a conceptual model could

describe the Vax e-mail system as something similar to a filing cabinet containingfile folders that, in turn, contain messages. Procedural methods describe commandsto execute a task, such as commands to send mail, open folders, and read mail. Bothconceptual and procedural method training was used in this study. Although prior

studies have identified training methods using conceptual models as being moreeffective than those using procedural methods, both methods were used in this study

to prevent any confounding effects that may arise due to the selection of a singlemethod (Santhanam & Sein, 1994; Olfman & Mandviwalla, 1995b). The subjects

were then asked to perform several training tasks related to e-mail and the use of

folders in e-mail. This training session lasted twenty minutes.

After completing the training session, subjects were asked to perform a series of

hands-on tasks using the e-mail system. These consisted of four near-transfer and

three far-transfer tasks. Near-transfer tasks are simple tasks very similar to the train-

ing tasks. Far-transfer tasks are complex, novel tasks not executed during training

that require subjects to reason to complete the task. As stated earlier, specially

designed software was used to continuously monitor and create a log of the subjects

1 Questionnaires, instruments and other experimental materials are available from the author.


10/18


actions and their outcomes. This log was evaluated and scored using a standard

grading scheme to provide each subjects task performance scores. Following thecompletion of these tasks, the subjects completed a five question paper and pencilsystem comprehension test to determine how well they understood the system.

Finally, the subjects were administered a survey questionnaire to measure their belief

levels and intentions to use. This survey was identical to the one given prior to

training (TPB1) and hereafter will be referred to as TPB2 survey. A graphical

description of the procedures used in the study is shown in Fig. 1.

The actual experiment was conducted in eight different sessions. The same

researcher and two assistants conducted every session. About twenty subjects took

part in each session. Subjects were not permitted to communicate with other subjects

or to look at another subjects work. All subjects were given 2% grade points for

participating in the experiment. Subjects who did not participate in training wereprovided with other assignments to obtain the 2% grade points.

Fig. 1. Experimental procedures.


11/18


3.2. Measure of dependent variables

The TPB1(administered prior to training) and TPB2 (administered after training)

surveys were identical and were used to measure a trainees beliefs and intentionsto use. A subjects identification of the six most valued outcomes of e-mail, the fivereferent others with whom they were motivated to comply, and the five necessaryfacilitating factors had been identified earlier. TPB1 and TPB2 were designed tocapture the subjects relevant behavioral, normative and control beliefs in relationto the above items. Subjects responses to the sixteen questions were obtained anda sum of all their responses provided an aggregate measure of their beliefs. Intentions

to use measured by three questions on the survey, were very similar to those used

in prior research (Mathieson, 1991; Taylor & Todd, 1995). A factor analysis was

conducted on this measure. In all cases, the factor loadings were greater than 0.70,

which is greater than the generally accepted levels (Comrey, 1973). The Cronbachsalpha in all cases was also greater than the generally acceptable level of 0.80(Nunnally, 1967).

The level of system proficiency achieved by trainees is typically measured byevaluating their performance on hands-on near-transfer and far-transfer tasks and

paper and pencil system comprehension tests (Bostrom et. al., 1990; Santhanam and

Sein, 1994; Simon et al., 1996). The subjects performance on the near-transfer andfar-transfer tasks was used to compute the near-transfer task and far-transfer task

scores respectively, providing the task comprehension score. Performance on the

paper-and pencil system comprehension test was used to compute the system com-prehension score. The scores were computed on a 0 to 10 scale (where a score of

0 was assigned to an incorrect answer and a score of 10 was assigned to a correct

answer with the correct sequence of operations). The subjects level of system pro-ficiency (the dependent variable) was measured by the near transfer task score, thefar transfer task score, and the system comprehension score.

4. Results

Among the subjects who participated, 38 were dropped from further analysis fora variety of reasons (including mal-functioning equipment, software problems,

incomplete questionnaire, etc.), resulting in a usable pool of 124 subjects. Of the

124 subjects, 56 subjects were in the treatment group and 68 in the control group.

There were no significant differences in age, prior email experience, grade pointaverages and other demographic variables, between subjects in the treatment and

control groups. A t-test comparing the average belief score after training (TPB2)

between subjects who received the TPB1 questionnaire and those who received the

Dummy questionnaire was not significant (p0.67). Thus, it was concluded thatadministering TPB1 (the belief questionnaire) in itself did not result in an increase

in subjects belief score and in all further analysis, the subjects scores are analyzedsolely based on whether they belonged to the treatment or control group.

It was hypothesized that the mean belief score of those subjects who received the


12/18


13/18


group is, significantly higher than the mean intention to use score of the subjects inthe control group, both before (p-value 0.01) and after (p-value 0.02) training.

These results provide support for Hypothesis 3.

Finally, Hypothesis 4 postulated that the mean performance scores of the subjectsreceiving the treatment would be higher than the mean performance score of the

subjects not receiving the treatment. As indicated earlier, three different performance

outcome measures (Near-transfer task score, Far-ransfer task score, and System Com-

prehension score) were used. As is to be expected, the three measures were highly

correlated. Given the high correlation between the dependent measures, the appropri-ate analysis in this case was a Multivariate Analysis of Co-Variance with Email

experience and GPA used as covariates. The results of this analysis are provided in

Table 3. It is seen that the hypothesis regarding performance outcomes (Hypothesis

4) is not supported.

5. Discussion

Improving end-users intentions to use a system is a critical issue in IS researchbecause of its direct impact on actual use of a system. The results of this study

provide convincing evidence to support our premise that behavioral intentions to use

the system can be influenced by using TPB based belief manipulation pre-trainingscripts in a training program. The results, also suggest that such pre-training scripts

may not improve performance outcomes of training.The results shown in Table 1 provide evidence to suggest that a simple belief

manipulation script can be used to positively influence the trainees beliefs regardingthe system. The mean belief scores after training increased for both groups. The

results also indicate that the effects of the treatment persisted and had an effect over

and above the effects of the training program. If the training program alone were

adequate to influence trainees beliefs, then the belief score of the treatment andcontrol groups after training would not be significantly different. The results indicatethat this is not the case. Even after training, the belief score of the treatment group

is significantly higher than that of the control group. This suggests that the belief

manipulation script had a positive influence on the trainees beliefs over and aboveany influence that the training program alone would have had.

Table 3

Performance comparison of treatment and control

Group Mean performance scoresa

Near transfer task Far transfer task System comprehension

Treatment 28.67 15.3 20.51

Control 29.01 15.5 21.01

a P-value for MANOVA comparison of means=0.97.


14/18


Similar results are observed for intentions to use the system. Even after training,

the mean intention-to-use score of the treatment group is significantly higher thanthe mean intention to use score of the control group. This indicates that, in addition

to any increases in the level of beliefs that can be attributed to the training program,the belief manipulation script results in a significant increase in intention to use.Taken together, these results provide strong evidence that a belief manipulation pre-

training script can be used effectively to increase trainees intentions to use the sys-tem.

The results also indicated (see Table 3) that the belief manipulation scripts do notseem to have a significant impact on increasing the system proficiency scores.Research studies have clearly identified the type of training method used as a signifi-cant factor in enhancing trainees learning performance, i.e., their level of systemproficiency (Santhanam & Sein, 1994; Compeau & Higgins, 1995; Olfman & Mand-viwalla, 1995b; Simon et al., 1996; Davis & Wiedenbeck, 1998). Therefore, it is

possible that the lack of learning performance difference between the groups may

be due to the fact that system proficiency levels are impacted to a greater extent bythe training method than they are by the trainees intentions to use the system. Inthis study, subjects in both groups received the same type of training method. Hence,

even if subjects in the treatment group (the subjects with the high intentions to use

the system) expended greater effort in training, the fact that both groups received

the same type of training might have equalized the extent of understanding about

the system resulting in no learning/performance differences. It is interesting to note

that these results are very similar to findings from several other research studiesthat have investigated the role of trainee motivation. These studies have found thatmanipulations to improve trainee motivation may not result in increased learning

outcomes (Webster & Martocchio, 1995; Venkatesh, 1999). Webster and Martocchio

(1995) found that optimistic training previews did not relate directly to trainee learn-

ing, while Venkatesh (1999) found that game-based training methods created favor-

able user perceptions of the system but did not lead to increased trainee learning.Taken together, these results suggest that the role of motivational factors in training

is important and its role in improving a trainees level of system proficiency needsfurther investigation, as suggested by other researchers (Colquitt, Lepine and Noe,

2000).

5.1. Limitations of the study

In this study we did not measure actual usage of the system. The link between

behavioral intentions to use a system and actual usage has been investigated and

documented (Mathieson, 1991; Azjen & Driver, 1992; Taylor & Todd, 1995; Sutton,

1998; Venkatesh & Speier, 1999). Since a large number of studies have already

validated this link, we did not attempt to address this particular issue.

The difference in intention-to-use scores between the treatment and control groups

was significant but not large. Perhaps this was due to the fact that e-mail was afamiliar system, and several subjects in the control group also knew about e-mail

and had some prior experience. However, the fact that we could signi ficantly influ-


15/18


ence the beliefs of our subjects about a well-known system such as e-mail provides

support for the use of belief manipulation pre-training scripts.Some subjects had

prior experience with e-mail and this was statistically controlled. We did not view

prior experience with the system as a disadvantage because for the purposes of thisexperiment, it was important to identify the subjects valued outcomes prior to theexperiment. We also had to facilitate their ability to relate to messages received from

the pre-training script. A totally new system that users had no idea about would have

made these tasks difficult.

5.2. Implications for practice and future research directions

Only 10% of training leads to a change in the behavior on trainees jobs(Venkatesh, 1999). Hence, practitioners are constantly searching for ways to improve

training and the subsequent systematic use of technology. The most recent report on

industry practices indicates that technology related training expenditures continue toincrease as organizations spend enormous sums of money to send employees to

training programs (Van Buren, 2000). Most trainers and trainees think that these

training programs are in place for the purpose of learning about a piece of software.

Our results indicate that developing a high level of system proficiency may not corre-late with high levels of intention to use the system and, hence may not lead to

increased use of the system. The results of this study suggests that pre-training stra-

tegies could help to influence users intentions to use the system and improve the

long-term organizational effects of a training program. By using such belief manipu-lation scripts as a part of their training program, organizations could highlight how

learning to use the system can help employees get their regular jobs done easily and

how their use of the system can impress their peers and bosses. The scripts could

also specifically inform employees on the types of resources such as help centers,manuals, etc. that the organization would be willing to provide to facilitate the use

of the system. The results of this study suggest that this should result in increasinga trainees intentions to use the system. Given earlier results that show a strongrelationship between intention to use and actual usage, we believe that such a strategy

will also result in higher actual usage of the system.

One natural extension of this research is to identify whether or not beliefs andintentions to use a system can be influenced by methods other than pre-trainingscripts. User manuals are important documents in the hands of end-users and could

be examined as an option (Olfman & Mandviwalla, 1995b). The effects of delivering

the messages via other media such as videos or computer-based methods could also

be investigated. We also did not investigate the impact of different types of training

methods and their potential interaction with the belief manipulation script. These are

factors that could influence the formation of beliefs and deserve further investigation.The role of motivational factors in influencing system proficiency levels has shownmixed results and needs to be researched further. Recent research suggests that learn-

ing to use a system may also be dependent on the shared histories and social interac-tions that occur within the communities of practice (George, Iacono & Kling, 1995;Cook & Brown, 1999). Therefore, the use and impact of such social theories of


16/18


learning in improving intentions to use a system must be explored, perhaps using

qualitative research paradigms.

References

Azjen, I. (1985). From intentions to actions: A theory of planned behavior. In J. Kuhl, & J. Beckmann

(Eds.), Action-control: From cognition to behavior (pp. 1113). Heidelberg: Springer. London.

Azjen, I. (1991). The theory of planned behavior. Organizational Behavior and Human-Decision Pro-

cesses, 50(1), 179211.

Azjen, I., & Driver, B. L. (1992). Application of the theory of planned behavior to leisure choice. Journal

of Leisure Research, 24(3), 207218.

Azjen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood

Cliffs, NJ: Prentice-Hall.

Baldwin, T. T., & Magjuka, R. J. (1997). Training as an organizational episode: Pretraining in fluenceson trainee motivation. In J. K. Ford, S. W. J. Kozlowski, K. Kraiger, E. Salas, & M. S. Teachout

(Eds.), Improving training effectiveness in organizations (pp. 99127). New Jersey: Hillsdale.

Bostrom, R. P., Olfman, L., & Sein, M. K. (1990). The importance of learning style in end-user training.

MIS Quarterly, 14(1), 101119.

Burton, F. G., Chen, Y., Grover, V., & Stewart, K. A. (1992/93). An Application of Expectancy Theory

for Assessing User Motivation to Utilize an Expert System. Journal of Management Information Sys-

tems, 9(3), 183198.

Colquitt, J. A., Lepine, J. A., & Noe, R. A. (2000). Toward and integrative of training motivation: A

meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85(5), 678707.

Compeau, D. R., & Higgins, C. A. (1995). Application of social cognitive theory to training for computer

skills. Information Systems Research, 6(2), 118143.

Compeau, D., Olfman, L., Sein, M. K., & Webster, J. (1995). End-user training and learning. Communi-cations of the ACM, 38(7), 2426.

Comrey, A. L. (1973). A first course in factor analysis. New York: Academic Press.

Cook, S. D. N., & Brown, J. S. (1999). Bridging epistemologies: The generative dance between organiza-

tional knowledge and organizational knowing. Organization Science, 10(4), 381400.

Davis, S. A., & Bostrom, R. P. (1993). Training end-users: An experimental investigation of the roles of

the computer interface and training methods. MIS Quarterly, 17(1), 6185.

Davis, S. A., & Bostrom, R. P. (1994). The effects of an intrinsically motivating instructional environment

on software learning and acceptance. Information Systems Journal, 4(1), 326.

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A

comparison of two theoretical models. Management Science, 35(8), 9821003.

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1992). Extrinsic and intrinsic motivation to use computers

in the workplace. Journal of Applied Social Psychology, 22(14), 11111132.Davis, S., & Wiedenbeck, S. (1998). The effects of interaction style and training method on end -user

learning of software packages. Interacting with Computers, 11(2), 147172.

Fishbein, M., & Azjen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and

research. Reading, MA: Addison-Wesley Publishing Company.

George, J. F., Iacono, S., & Kling, R. (1995). Learning in context: extensively computerized work groups

as communities-of-practice. Information and Organization, 5(3/4), 185202.

Hill, T., Smith, N. D., & Mann, M. F. (1987). Role of efficacy expectations in predicting the decision

to use advanced technologies: The case of computers. Journal of Applied Psychology, 72(2), 307313.

Hubbard, A. (1999). Motivating learners. Mortgage Banking, 59(7), 113115.

Kanfer, R. (1990). Motivation theory and industrial and organizational psychology. In M. D. Dunnette, &

L. M. Hough (Eds.), Handbook of industrial psychology (Vol. 1) (pp. 75170). Palo Alto, CA: Con-

sulting Psychologists Press.

Lee, S. M., Kim, Y. R., & Lee, J. (1995). An empirical study of the relationships among end-user infor-

mation systems, training and effectiveness. Journal of MIS, 12(2), 189202.


17/18


Lim, K. H., Ward, L. M., & Benbasat, I. (1997). An empirical study of computer system learning: Com-

parison of co-discovery and self-discovery methods. Information Systems Research, 8(3), 254272.

Martocchio, J. (1992). Microcomputer usage as an opportunity: The influence of context in employee

training. Personnel Psychology, 45(3), 529552.Mathieson, K. (1991). Predicting user intentions: Comparing the technology acceptance model with the

theory of planned behavior. Information Systems Research, 2(3), 173191.

Nelson, R. R., Whitner, E. M., & Philcox, H. H. (1995). The assessment of end-user training needs.

Communications of the ACM, 38(7), 2739.

Noe, R. A. (1986). Trainees attributes and attitudes: Neglected influences on training effectiveness. Acad-

emy of Management Review, 11(4), 736749.

Nunnally, J. C. (1967). Psychometric theory. New York: McGraw Hill.

Olfman, L., & Bostrom, R. P. (1991). End-user software training: An experimental comparision of

methods to enhance motivation. Journal of Information Systems, 1(2), 249266.

Olfman, L., & Mandviwalla, M. (1995a). An experimental comparison of end-user software training

manuals. Information Systems Journal, 5(1), 1936.

Olfman, L., & Mandviwalla, M. (1995b). Conceptual vs. procedural software training for graphical userinterfaces: A longitudinal field experiment. MIS Quarterly, 18(4), 405426.

Olfman, L., Bostrom, R. P., & Sein, M. K. (1991). The effects of training on motivation to use and self-

reported use of an electronic mail system. In J. M. Carey (Ed.), Human factors in information systems:

An organizational perspective (pp. 309320). New Jersey: Norwood.

Petty, R. E., & Cacioppo, J. T. (1981). Attitudes and persuasion: Classic and contemporary approaches.

Dubuque, IA: Brown Company Publishers.

Salancik, G. R., & Pfeffer, J. (1978). A social information processing approach to job attitudes and task

design. Administrative Science Quarterly, 23(2), 224253.

Santhanam, R., & Sein, M. K. (1994). Improving end user proficiency: Effects of conceptual training and

nature of interaction. Information Systems Research, 5(4), 378399.

Sein, M. K., Bostrom, R. P., & Olfman, L. (1987). Training end users to compute: Cognitive motivational

and social issues. INFOR, 25, 236255.

Sein, M. K., Bostrom, R. P., & Olfman, L. (1999). Rethinking end-user training strategy: Applying a

hierarchical knowledge-level model. Journal of End-User Computing, 1/3, 3239.

Simon, S. J., Grover, V., Teng, J. T., & Whitcomb, K. (1996). The relationship of information system

training methods and cognitive ability to end-user satisfaction, comprehension and skill-transfer: A

longitudinal field study. Information Systems Research, 7(4), 466490.

Shayo, C. & L Olfman. (1993). Is the effectiveness of formal end-user software training a mirage? Pro-

ceedings of the ACM group on Computing and Personnel Research , ACM Press, New York, 8899.

Snead, K. C., & Harrell, A. M. (1994). An application of expectancy theory to explain a managers

intention to use a decision support system. Decision Sciences, 25(4), 499513.

Sutton, S. (1998). Predicting and explaining intentions and behavior: How well are we doing? Journal

of Applied Social Psychology, 28(15), 13171322.Szajna, B. (1996). Empirical evaluation of the revised technology acceptance model. Management Science,

42(1), 8592.

Taylor, S., & Todd, P. A. (1995). Understanding information technology usage: A test of competing

models. Information Systems Research, 6(2), 144176.

Thompson, R. L., Higgins, C. A., & Howell, J. M. (1991). Personal computing: Toward a conceptual

model of utilization. MIS Quarterly, 15(1), 125143.

Van Buren, B. E. (2000). The 2000 ASTD State of the Industry Report, Press Release from the American

Society of Training and Development, March, 14.

Venkatesh, V. (1999). Creation of favorable user perceptions: Exploring the role of intrinsic motivation.

MIS Quarterly, 23(2), 239260.

Venkatesh, V., & Speier, C. (1999). Computer technology training in the workplace: A longitudinal

investigation of the effect of mood. Organizational Behavior and Human Decision Processes, 79(1),

128.

Vroom, V. H. (1964). Work and motivation. New York: John Wiley.


18/18


Webster, J., & Martocchio, J. J. (1992). Microcomputer playfulness: Development of a measure with

workplace implications. MIS Quarterly, 16(2), 201226.

Webster, J., & Martocchio, J. J. (1993). Turning work into play: Implications for microcomputer software

training. Journal of Management, 19(1), 127147.Webster, J., & Martocchio, J. J. (1995). The differential effects of software training previews on training

outcomes. Journal of Management, 21(4), 757787.

Wiedenbeck, S., & Zila, P. A. (1997). Hands-on practice in learning to use software: A comparision of

exercise, exploration and combined formats. ACM Transactions on Computer-Human Interaction,

4(20), 169196.

Zimbardo, P., & Ebbesen, E. B. (1970). Influencing attitudes and changing behavior. Reading, MA:

Addison Wesley.

Documents

1.Improving Training Outcomes Using Pre-training Scripts a Theory of Planned Behavior Approach