_organizational_performance_in_hospitals_in_Suriname._A_test_of_an_extended_version_of_DeLone_and_McLean_s_IS_Success_Model_2003_by_DeLano_Gefferie_MBAIV

Maastricht School of Management FHR Lim A Po Institute

MBA IV

Master of Business Administration Program

2007 – 2009

Maastricht School of Management

“Evaluating the Effectiveness of Information Systems on

organizational performance in hospitals in Suriname”

“A test of an extended version of DeLone and McLean’s IS Success Model 2003”

By

Delano C. Gefferie

Suriname

May 2009

Supervised by: Prof. Dr. Kami Rwegasira PhD

This thesis was submitted in partial fulfillment of the requirements for the degree of Master of

Business Administration (MBA) at Maastricht School of Management (MsM), Maastricht, the

Netherlands

MBA Thesis SRFHR0407023

ii

ACKNOWLEDGEMENT

This educational journey has been the most challenging and invigorating experience of my live. I

can only acknowledge a few of many people and organizations on this page who were there along

the way to help me in my growth. Doing research is especially a social process. The support and

advice of other people is crucial in order to carry out an enormous project like this. However, I like

to extend my thanks to several people. I’m grateful for the support of my peers in the MBA course.

They inspired me to work hard to persevere in spite of challenges we faced. My sincere appreciation

goes to the Urmila Monorath, Ria Wong A Fa, and Ismanto Adna who finally got me over the extra

mile.

The Ministry of Health of Suriname and the hospitals in Suriname are acknowledged for the

mandate to conduct this study and their support to the project throughout. The coordinators and other

staff of the Ministry Health and hospitals are recognised for their great effort in preparing the

different information documents and time obtained for distributing en generating the questionnaires

to conduct this study, without which it would not to be possible.

My supervisor Professor Dr. Kami Rwegasira PhD played a key role in maintaining the scientific

rigour for the study and his dedication is appreciated. FHR Lim A Po Institute for Social Studies and

Dr. S. DeBono are credited for the direction and support they provided during the early stages of the

project development, planning and finalization.

I can not forget Mrs. Dr. Ollye Chin A Sen and Mrs. Urmila Monorath who have motivated me to do

the MBA course as well as my friend and fellow Mr. Frits Konigferander who was always there for

me.

Most important is the appreciation directed to all co-ordinators at the Ministry of Health and the

hospitals: Mrs. A. Fitz Jim, Manager Planning and Control of the Diakonessenhuis, Mr. R. Jiwalal,

System Manager of the s’Lands Hospital, Mr. O. Lalay, Financial Manager of Streekziekenhuis

Nickerie and Mrs. Dr. M. Algoe MD, Manager MIS Ministry of Health Suriname and other staff

members of the Department of Health and the hospitals who participated in the study, helping to

make it a success.


iii

DEDICATION

My family and friends have sustained me through many transitions in my life but my wife, daughters

and brother Karel, deserve the deepest gratitude for being there every step of the way. Karel had

faith in me when I doubted and was my daily cheerleader and drill sergeant when it was hard to

continue. I dedicate this study to my wife, daughters and brother Karel who made this success

possible.


iv

ABSTRACT

Organizations are very dependent on information systems when evaluating their performance and

there are many indications that these trends will continue in the future. Consequently managements

in organizations in Suriname are fully cognizant of this potential and thereby employ ICT to support

their activities.

The aims of this study are to determine what factors are influencing the effectiveness of Information

Systems and to examine their impact on organizational performance in the hospital environment in

Suriname.

The approach in this study was a multiple case study using surveys and archival data while the

instrument for data collection was a questionnaire. Data were collected from three general hospitals

in Suriname. A set of 75 items, based on the research model, were developed, and aggregated into

four scales for measuring the use of information systems in the hospitals, and six scales for

measuring the effectiveness or success of information systems on organizational performance.

Regression and Structural Modeling Techniques (SEM) were applied to data collected from

questionnaires answered by 99 users of Information Systems and IS support personnel at three

general hospitals in Suriname with more than 100 personnel and an identified automation

organization.

Correlation analysis reveals a statistically significant relationship between both sets of variables,

specified in an IS success model and a behavior model of use. Results show that the strongest or

main factors influencing the effectiveness on organizational performance in the hospitals in

Suriname are User Satisfaction and the Benefits of Use directly and Information Quality indirectly.

In this study those factors seemed to be the strongest predictors of Net Benefits in the hospital

environment of Suriname, and thus performance.

Further results show that the use of information system by the core professions in the three hospitals

is underdeveloped and there is a significant difference in approach and culture between private and

public hospitals.


v

TABLE OF CONTENTS ACKNOWLEDGEMENT ................................................................................................................. ii

DEDICATION ................................................................................................................................... iii

ABSTRACT ....................................................................................................................................... iv

TABLE OF CONTENTS................................................................................................................... v

GLOSSARY .....................................................................................................................................viii

LIST OF ABBREVIATIONS............................................................................................................ x

LIST OF FIGURES........................................................................................................................... xi

LIST OF TABLES............................................................................................................................ xii

CHAPTER 1 INTRODUCTION .................................................................................................. 1

1.1 Introduction ........................................................................................................................ 1

1.2 Problem definition.............................................................................................................. 2

1.2.1 Research objectives .................................................................................................... 2

1.2.2 Research questions ..................................................................................................... 3

1.3 Research methodology ....................................................................................................... 5

1.3.1 Introduction ................................................................................................................ 5

1.3.2 Research Design.......................................................................................................... 6

1.3.3 Sample and Setting..................................................................................................... 8

1.3.3.1 Questionnaire Design ............................................................................................. 8

1.3.3.2 Sample and Data Collection Procedure.............................................................. 10

1.3.3.3 Data analysis ......................................................................................................... 11

1.4 Research material............................................................................................................. 13

1.5 Scope of research .............................................................................................................. 13

1.6 Chapter Outline................................................................................................................ 14

1.7 Problems and limitations ................................................................................................. 14

1.8 Summary ........................................................................................................................... 15

CHAPTER 2 BACKGROUND INFORMATION.................................................................... 16

2.1 Introduction ...................................................................................................................... 16

2.2 Hospitals in Suriname ...................................................................................................... 16

2.2.1 Information Systems in hospitals in Suriname...................................................... 17

2.2.2 Justification of research design ............................................................................... 17

2.2.3 Attributes which characterize a hospital IS........................................................... 19

CHAPTER 3 LITERATURE REVIEW.................................................................................... 20


vi

3.1 Introduction ...................................................................................................................... 20

3.2 Information Systems ........................................................................................................ 20

3.3 ERP Systems ..................................................................................................................... 21

3.4 Approaches and methods in use for the evaluation of Information Systems.............. 22

3.4.1 Introduction .............................................................................................................. 22

3.4.2 A conceptual hierarchy of system objectives ......................................................... 23

3.4.3 Relevant approaches to evaluate Information Systems ........................................ 24

3.5 Approaches to the evaluation of Medical Information Systems (MEIS) .................... 32

3.6 Measurement .................................................................................................................... 34

3.7 Organizational issues ....................................................................................................... 35

3.8 Summary ........................................................................................................................... 36

CHAPTER 4 HYPOTHESES DEVELOPMENT AND PROPOSED MODEL .................... 37

4.1 Introduction ...................................................................................................................... 37

4.2 Comparison of Evaluation Approaches ......................................................................... 37

4.3 Applying the IS Success Models in research context .................................................... 38

4.3.1 Operationalization of variables and proposed IS Success Model ........................ 40

4.3.1.1. Variables of IS Success Model................................................................................ 40

4.3.1.2 Variables of Behavior Model of Use ............................................................... 43

4.4 Measures of IS Success Model and proposed Framework ........................................... 45

CHAPTER 5 RESEARCH METHODOLOGY........................................................................ 53

5.1 Procedures and Sample.................................................................................................... 53

5.2 Case Evidence ................................................................................................................... 56

5.2.1 Introduction .............................................................................................................. 56

5.2.2 Case Background‘s Lands Hospital........................................................................ 57

5.2.3 Case Background Diakonessenhuis ........................................................................ 59

5.2.4 Case Background Streekziekenhuis Nickerie ........................................................ 60

5.3 Respondent Characteristics............................................................................................. 61

5.4 Research Instrument and Variable Measurement ........................................................ 61

5.4.1 Research Instrument................................................................................................ 61

5.4.2 Variable measures .................................................................................................... 62

5.5 Data Analysis and Discussion.......................................................................................... 63

5.5.1 Model testing............................................................................................................. 63

5.5.2 Quality dimensions in Proposed IS Success Model ............................................... 65

5.5.3 Net Benefits of Use in Proposed IS Success Model................................................ 66


vii

5.5.4 User Satisfaction in Proposed IS Success Model ................................................... 67

5.5.5 Beliefs in Proposed IS Success Model..................................................................... 67

5.5.6 Net Benefits in Proposed IS Success Model ........................................................... 68

5.5.7 Summary ................................................................................................................... 68

CHAPTER 6 CONCLUSIONS AND RECOMMENDATIONS ............................................. 69

6.1 Conclusions ....................................................................................................................... 69

6.2 Recommendations ............................................................................................................ 70

6.3 Limitations ........................................................................................................................ 73

6.4 Implications for Future Research ................................................................................... 73

BIBLIOGRAPHY ............................................................................................................................ 75

APPENDIX 1: Description of data set.......................................................................................... 78

APPENDIX 2: Questionnaire; English version ........................................................................... 80

APPENDIX 3: Questionnaire; Dutch version.............................................................................. 86

APPENDIX 4: Results pre-test ..................................................................................................... 93

APPENDIX 5: Indication of human resources in health sector in Suriname........................... 94

APPENDIX 6: Hospital sample..................................................................................................... 95

APPENDIX 7: Hypothesized relationships and correlation..................................................... 103

APPENDIX 8: Overview Alternative Hypothesis and testing results ..................................... 106

APPENDIX 9: Descriptive statistics ........................................................................................... 108

APPENDIX 10: Overall results final tests Structural Equation Model .................................... 109

APPENDIX 11: Indication of distribution of spending by payer .............................................. 113

APPENDIX 12: Indication of Health Status and Outcomes Vital Statistics............................. 114

APPENDIX 13: Direct, indirect and total effects of constructs ................................................. 115

APPENDIX 14: Research Model with hypotheses ...................................................................... 116


viii

GLOSSARY

This is a listing of a definition of some terms used in this thesis.

Definitions:

A system is a set of two or more interrelated components that interact to achieve a goal (Marshall B.

Romney and Paul J. Steinbart, 2006)

An Information System (IS) can be defined technically as a set of interrelated components that

collect (or retrieve), process, store, and distribute information to support decision making and

control in an organization (Kenneth C. Laudon and Jane P. Laudon. 2000).

A Management Information System (MIS) is a formalized computer information system that can

integrate data from various sources to provide the information necessary for management decision

making (J. Hicks, 1990).

Accounting Information System (AIS) is a system that collects, records, stores and processes data

to produce information for decision makers (Marshall B. Romney and Paul J. Steinbart, 2006).

Medical Information Systems (MEIS) is an Information System that provides examples of what

health care providers should do and how medical informatics should be of use in order to gain public

trust (Japan Association of Medical Informatics Commission for the definition of Electronic

Medical Records).

Information is data that have been processed and organized into output that is meaningful to the

person, who receives it. Inventory can be mandatory, essential or discretionary (Marshall B.

Romney and Paul J. Steinbart, 2006).

Data are characters that are accepted as input to an information system for further storing and

processing. After processing the data become information (Marshall B. Romney and Paul J.

Steinbart 2006)


ix

IS Success is a measure of the degree to which the person evaluating the system believes that the

stakeholder (in whose interest the evaluation is being done) is better off. Logically, if Net Benefits

could be measured with precision, IS Success would be equivalent to Net Benefits. However, IS

Success also has political and emotive overtones of “we won” about it, which are less evident in Net

Benefits (Seddon and Kiew (1997).

A Health Information and Management System (HIMS) is epidemiology a combination from

various sources used to derive information about health status, health care, provision and use of

services and health impact Informatics, McGraw-Hill Concise Dictionary of Modern Medicine, by

The McGraw-Hill Companies, Inc, 2002

A Hospital Information System (HIS) is a system that provides information management features

the hospitals need for daily business; Features: patient tracking, billing and administrative programs;

may include clinical features, McGraw-Hill Concise Dictionary of Modern Medicine, by The

McGraw-Hill Companies, Inc, 2002


x

LIST OF ABBREVIATIONS

AIS Accounting Information System

AMOS Analysis of Moment Structures

ANOVA Analysis of Variance

ARKIS Anaesthesia Record-keeping Information System

AZV Algemene Ziektekostenregeling (National Medicare)

CEO Chief Executive Officer

CIS Computerized Information Systems

D&M DeLone and McLean

D&M 1992 Delone and McLean IS Success Model 1992 (original)

D&M 2003 Delone and McLean IS Success Model 2003 (updated)

DKH Diakonessenhuis (private hospital)

ERP Enterprise Resource Planning

GOVT Government

HIS Hospital Information System

ICT Information Communication Technology

IS Information System

ISs Information Systems

LISREL Linear Structural Relationships

LH ‘s Lands Hospital (public hospital)

MRIS Medical Record Information Systems

MIS Management Information System

NHIS National Health Information System

NIS Nursing Record Information System

POP Population

SEM Structural Equation Model

SZN Streekziekenhuis Nickerie (public hospital)

SPSS Statistical Package for Social Science

TAM Technology Acceptance Model


xi

LIST OF FIGURES

Figure 1: Extended DeLone and McLean Success Model 2003 5

Figure 2: The Technology Acceptance Model (TAM) 12

Figure 3: Information System 20

Figure 4: D&M IS Success Model 1992 28

Figure 5: Updated D&M Model 2003 30

Figure 6: Seddon IS Success Model 1997 (a Re-specified version of D&M Model 1992) 31

Figure 7: Levels of clinical efficacy related to use of diagnostic technology 33

Figure 8: Path Analysis Results 65


xii

LIST OF TABLES

Table 1: Capacity of the three hospitals in the study 8

Table 2: Overview of test group in DKH 9

Table 3: Overall Information Systems hospitals 18

Table 4: Comparison of IS Evaluation Approaches in conceptual hierarchy 39

Table 5: Research hypotheses 52

Table 6: Results of Factor Analyses of construct scales 55

Table 7: Descriptive statistics and Correlation matrix 64


xiii

This page is intentionally left blank


1

CHAPTER 1 INTRODUCTION

1.1 Introduction

The relationship between the use of Information Technology and organizational performance has widely

been researched in recent years. The results have shown a significant and positive correlation between

IT and organizational performance. This thesis deals with the evaluation of the effectiveness of

information systems (ISs) in hospitals in Suriname. Organizations depend largely on information

systems when evaluating their performance and there are many indications that these trends will

continue in the future. However, there is concern among CEOs and top managers that the investments in

ISs are not yielding the anticipated outcomes. In recent years, Information, and Communication

Technology (ICT) has played a pivotal role in the digital economy. It has become one of the core

elements of managerial reform around the world. Without a doubt, hospital management in Suriname are

fully cognizant of this potential and therefore employ ICT to support hospital activities. Hence, ISs have

emerged. Medical Information Systems (MEIS) were improved the last five years by most of the

hospitals by introducing Microsoft Business Solution Navision, an Enterprise Resource Planning (ERP)

system which should help them to manage their organization efficiently and effectively.

This study aims to understand the fundamental factors that influence the effectiveness of information

systems in hospitals in Suriname, testing an extended version of the Delone and McLean’s IS Success

Model 2003. It reports the results of an empirical study evaluating the contribution of ISs in the

effectiveness on operational performance of hospitals based on literature review and it includes research

of many subject areas related to the evaluation of ISs, such as models and frameworks as well as

literature review on the characteristics of medical informatics and health economics. The thesis

measures the factors, determining information systems’ effectiveness in Surinamese hospitals with more

than 100 employees, having an identified automation organization and using ERP systems by

investigating data collected from users and IS support staff who have experienced ISs. These variables

are presumably system decision performance: System Quality, Information Quality and Service

Quality and the measures for effectiveness: Intention to Use (System Usage), Benefits of Use (Benefits

of use from end-users’ view), User Satisfaction and Net Benefits (Net value from organization’s view)

that are considered the most effective variables to predict organizational performance. In conjunction

with the results of the empirical study conclusions have been drawn and recommendations have been

formulated for hospital management in Suriname.


2

1.2 Problem definition

Organizations generally implement Information Systems after advise of experienced IS users and

consultants but owing to the complexity of today’s ISs the management is often unsatisfied when

comparing the effectiveness of their IS to those of competitors and/or others. To improve business and

so organizational performance, organizations need an efficient planning and control system that

synchronizes planning of all processes across the organization. The key to competitiveness nowadays is

a solid IS seamlessly (dovetailed) aligned with the core business processes developed to deliver high

quality products and services to customers efficiently and effectively. These demands have prompted

more and more firms and organizations to shift their ICT strategy from developing in-house IS to

purchasing applications, such as ERP to generate synergies and enhance operation efficiency (Hong and

Kim, 2002). ERP systems as an integrated IT solution have been extensively adopted since the late

1900s by organizations to help improve both the efficiency and effectiveness of business processes.

However, it usually takes time to implement ERP systems. Moreover, many ERP solutions are

expensive projects in which a considerable number of diverse experts and different agendas are often

involved.

The problem that organizations face after implementing an ERP system and Information Systems in

general is to determine to what extent they lead to organizational success. Hence the central research

question of this thesis is:

“What are the main factors influencing effectiveness of information systems on the organizational

performance of hospitals in Suriname?”

1.2.1 Research objectives

Under the assumption that effectiveness of Information Systems improves organizational performance,

the following research objectives have been formulated:

1. determination of the factors that influence effectiveness of ISs on organization performance;

2. determination of ISs hospitals in Suriname have in place and the level of their effectiveness;

3. determine the relationship between use of ISs and performance of hospitals in

Suriname.


3

The Information Systems of the three hospitals involved as a whole is the unit of analysis in this study.

This due to the fact that their ERP system is not in the same stage of development and by doing so facts

can be found to compare findings to help them evaluate how implementation of their ERP system

among other ISs have in the first place benefited them selves, and secondly in comparison with other

hospitals involved in the study. Although it is very important to evaluate the success of ERP

implementation projects since a lot of financial and human resources have been invested, Brandford and

Sandy (2002) reported that 57% of the companies they have interviewed launched no assessment on the

performance of their ERP systems. Lack of empirically effective evaluation models is one of the reasons

for not measuring ERP performance. This is also one of the reasons that motivate the final selection of

the hospitals involved in the study. Hospitals are parent organizations with more than one health care

facility. As a contribution, this study is the first study aimed to understand organizational performance in

hospitals in Suriname by using the Delone and McLean IS Success Model 2003 extended with a partial

behaviour Model of Use derived from the Technology Acceptance Model (Davis et al, 1989) which was

elaborated by Seddon et al.’s in Seddon IS Success Model 1997. As such, it is one of the few studies that

empirically confirm the excellence of an extended version of DeLone and McLean’s IS Success Model

2003 in predicting net benefits from IS use. It also provides a theoretical foundation for researching ISs

effectiveness in the future worldwide.

This study accordingly attempts to test a success model for post-implementation ERP systems among

other ISs and to empirically investigate the multidimensional relationship between the success measures.

In other words, the purpose of this study is to identify the variables for success measurement of post–

implementation of ERP systems particularly and ISs in general. In addition, the relationships between

the success measures are also empirically tested.

To the practitioners (management, users, IS support staff and government), this study provides a useful

guideline for achieving better ISs and increasing benefits from use by identifying specific benefits

factors, which are simple and easy to understand, and can be manipulated through system design and

implementation. It thereby assists hospital management in considering the findings for development and

evaluation of ISs.

1.2.2 Research questions

In this thesis I have, particularly, tested the relationship between the effectiveness of IS and the

organizational performance of hospitals involved. So, the aim of this thesis is to empirically test whether


4

the use of ISs in organizations leads to productivity improvement or not. The main research questions

derived from the objectives set forth in thesis are:

1. What factors influence effectiveness of Information Systems?

2. Which Information Systems hospitals in Suriname have in place and what is the level of

their effectiveness?

3. What is the relationship between the effectiveness of Information Systems and the

performance of hospitals in Suriname?

A careful scan of the literature shows that researchers in similar studies generally apply statistical and

mathematical models, such as regression analysis, correlation analysis, and data envelopment analysis.

In this study I will apply correlation and path analysis to study the relationship between IS usage and

organizational performance. In order to answer the research questions I found among other models (see

Chapter 2 and 3) DeLone and Mclean’s model a strong theoretically based and empirical tested model to

support my research. I have conducted an empirical study in three hospitals using Computerized

Information Systems (CIS) and implemented an ERP system to analyse the relationship between the

variables with an extended version of DeLone and McClean’s IS Success Model 2003 (see Figure 1).

The aim of the research is not a formal test of the proposed research model. So, my research is an

applied analysis of the model and not a theoretical instrument evaluation exercise.


5

Figure 1: Extended DeLone and McLean IS Success Model 2003 (modified)

Source: Shih-Wen Chien, 2004

1.3 Research methodology

1.3.1 Introduction

As the D&M Model is a multidimensional model which need to be analyzed with multivariate

techniques the stage six-step approach to Structural Equation Modeling (SEM) analysis was conducted.

These six stages are:1

1. Defining individual constructs (see Appendix 1);

1 Hair et al. (2005)

4. Other Measures of Net

Benefits of IS Use

2. General Perceptual Measure

of Net Benefits of IS Systems

Benefit

of Use

User

Satisfaction

Net Benefits

3. Partial Behavioral Model of IS Use

Perceived

Ease of Use

Perceived

Usefulness

Intention

to Use

1. Measures of Three

Quality Dimensions

System

Quality

Information

Quality

Service

Quality


6

2. Developing overall measuring model (see Figure 1);

3. Designing a study to produce empirical results (see 1.3.2);

4. Assessing the measurement model validity (see 1.3.3.1, 5.1, 5.4.2 and 5.5.1);

5. Specifying model; hypotheses validity (see 1.3.3.1, 5.4.2, 5.5.1 and Appendix 7, 8, 10);

6. Assessing structural model validity (see 1.3.3.1, 1.3.3.3, 5.5.1 and Appendix 7, 8, 10)

1.3.2 Research Design

In order to address the main research questions in this study that may provide valuable insight for

researchers and for hospital management of which factors influence the effectiveness of ISs in general

and the ISs in operation in the hospital environment and therefore its impact on the organizational

performance (net benefits), I have intended to use a quantitative approach. By choosing a quantitative

approach outcomes of net benefits could be predicted the best, then when using a qualitative approach.

Furthermore results are comparable in time while trends could be analyzed and declared.

I specially adopted an extended version of the Updated DeLone and McLean (2003) IS Success Model

to look at the factors that influence organizational performance. According to the proposed model those

factors for the IS Success Model are: System Quality, Information Quality, Service Quality, Benefits of

Use (System Usage), User Satisfaction and Net benefits, while for the behavioral part of this model,

namely System usage/ Benefits of use those factors are Perceived Ease of Use, Perceived Usefulness

and Intention to Use ISs; I have also included behavior and demographic factors that may influence net

benefits based on literature review.

To address the three main research questions of this study mentioned in paragraph 1.2.2 I will discuss

the type of data analyzed in this study and how it will contribute to addressing the research question

involved:

1. What factors influence effectiveness of Information Systems?

Factors that influence the effectiveness of Information Systems have been determined by literature

review and interviews with IT-managers and focus groups. These factors are: System Usage, Benefit of

Use and Users Satisfaction directly. From these factors System Usage has been identified as a behavior

and not a success variable. Both, Benefits of Use and User Satisfaction influence organizational

performance and thus Net Benefits, directly while the quality variables System Quality, Information

Quality and Service Quality influence the effectiveness of IS indirectly (see Appendix 13). Studies have

shown that the more users use a system the more they are satisfied with it, and the more benefits are


7

resulted from. To elaborate System Usage as a behavior the D&M Model was extended with the

behavior model of use, which consists of the variables Perceived Ease of Use, Perceived Usefulness and

Intention to Use as shown in Figure 1, section 3. All these variables were measured with the

questionnaire instrument shown in Appendix 3, part 1 to 10, while the hypotheses H1a to H9 were

formulated as depicted in Table 5, page 52.

2. Which Information Systems hospitals in Suriname have in place and what is the level of

their effectiveness?

This research question has been addressed by identifying which information systems (see Table 2) the

selected hospitals have installed and in what structure. Furthermore information was gathered to

determine how long these ISs were implemented because the course of implementation is an important

factor which has to be taken in consideration when measuring ISs in a post-implementation stage for a

single hospital or for comparison purposes. Measuring the level of effectiveness of this infrastructure

was achieved by measuring the extent to which System Quality, Information Quality and Service

Quality (part 1 to 3 of the questionnaire contribute to Benefit of Use and User Satisfaction. To test these

relationships hypotheses H1a to H3b were formulated

3. What is the relationship between the effectiveness of Information Systems and the performance of

hospitals in Suriname?

The relationship between the effectiveness of Information Systems and the performance of the hospitals

in Suriname is depicted in the relation between measurements of effectiveness as stated in point 1 and 2,

namely Benefits of Use and User Satisfaction directly and indirectly through the relation between the

Quality measures and Net Benefits. This relation is hypothesized in hypotheses H4b-c and H5b-d, while

Net Benefit from its side contributes to Intention of Use for which relation hypothesis H9 was

formulated.

The character of the research is the application of theoretical concepts and thus deductive. Using an

extended version of the DeLone and McLean IS Success model 2003 (see figure 1), I have derived the

research questions and formulated the hypothesis to be used in the thesis (see for discussion Chapter 4)

to test the relationships in the proposed model and by doing so I have addressed the research question of

this study.

In order to test the content validity of the proposed research model, a series of in-depth interviews were

conducted with representative respondents at the hospitals involved in the study. The capacity of the


8

different hospitals in terms of licensed beds, category of personnel, and users of IS and IS support

personnel are stated in Table 1.

Personnel Target group Hospital

Name

Beds

Physicians Nurses Others Total Users of

ISs

Support staffTotal

LH2 368 19 339 328 686 70 2 72

DKH3 217 23 220 358 601 150 3 153

SZN4 97 9 121 214 344 40 2 42

Total 682 51 680 900 1631 260 7 267

Table 1: Capacity of the hospitals in this study as per December 31, 2008

Source: Hospitals in study

1.3.3 Sample and Setting

1.3.3.1 Questionnaire Design

The survey items were adapted from previous studies regarding the D&M Model and the TAM Model

with some modification to reflect the context of the study. The measures of continuance

intention to use were adapted from the measures of Intention to Use in the Technology Acceptance

Model (TAM) with modification to reflect the intention to continue using IS. Each item was rated on a

Likert scale from 1 to 7 (Strongly Disagree to Neutral to Strongly Agree). The questionnaire of which

the Dutch version stated in Appendix 3 was used in the surveys is stated in Appendix 2.

The questionnaire was originally designed in English (see Appendix 2) and then translated into Dutch

for the survey. Afterwards the Dutch version of the questionnaire was checked and translated back into

English by an independent translator to ensure there was no loss of meaning during the translation as

suggested by Zikmund, (2003). The process showed there were no significant discrepancies between the

2 S’Lands Hospital 3 Diakonessenhuis 4 Strrekziekenhuis Nickerie


9

two versions. In addition to the back-translation process, the validity and reliability of the measurement

instrument were confirmed. The validity of the questionnaire, the ability to measure (Zikmund 2003),

was strengthened through an extensive review of IS and statistical literature.

In addition, a pre-test through 10 convenience samples was applied to determine if the intended

participants had any difficulty understanding the questionnaire, the length of the questionnaire, the

sequence of questions, the sensitivity of any of the items, if the time needed to complete it was sufficient

and whether there were ambiguous or biased questions (Cooper, 2003). Based on the feedback of the

pre-test, some small adjustments were made and the variable Use was increased from two to four items

to ensure its validity at the test. Draft questionnaires were sent to ten potential participants divided over

the different units of the potential participants in order to have a good representation of the target group.

The participants were users of IS of the Diakonessenhuis and IS support staff divided according to Table

2. The other two hospitals were not ready to participate when the pre-test was performed.

The pre-test was successful. Descriptive statistics were analyzed for any significant kurtosis or skew.

Scales means were tested on their values close to minimum or maximum, even as Confirmatory Factor

Analysis/test (CFA). 8 out of the 10 participants have responded with good feedback to apply some

small modifications regarding the comment of the respondents in the questionnaire instrument and

prepared the final draft. After modification of the questionnaire it was distributed among the three

hospitals. There was only one questionnaire for all participants. This in the context of developing a

more standardized questionnaire for the extended D&M Model which already have shown great

contribution to IS evaluation.

Department N users and IS staff Test group %

Account receivables 7 1 14

Purchasing 3 1 33

IT (IS support) 4 1 25

Finance 7 1 14

Others 129 6 5

Total 150 10 6

Table 2: Overview of test group in DKH

Source: Own research and information Planning and Control Department DKH

The survey questionnaire used in this study contains 11 parts capturing information for 10 latent


10

constructs and 1 part capturing demographical information of respondents containing 75 indicator items

specifically addressing the fifteen propositions formulated in the study. Part 11 contains 13 questions

capturing respondents’ demographic information, such as age, gender, department, highest educational

level and positions. The remaining parts comprised of 56 items on IS success (see Appendix 2 and 3).

All the items in part 1 to 10 were measured using a 7-point Likert scale ranging from 1 = strongly

disagree to 7 = strongly agree.

1.3.3.2 Sample and Data Collection Procedure

The sample of interest of this study is users of IS and people involved in the system development

process. I have chosen these respondents because they could provide valid information concerning

both the system development process and the quality of the final system. A number of steps have been

carried out to identify such a group of respondents. First, hospitals were selected based on the criteria

that the number of employees should be more than 100 and the existence of an appropriate IT

environment. Second, direct discussions with CIO, or IT department representatives were made,

introducing the study’s objectives and inviting appropriate candidates of their organization to

participate in this study. The reasons of contacting the IT department representative is that they are the

people who know which employees of their company have participated in system development and are

using the system. Sets of questionnaire, including a cover letter explaining the purpose of the study

were mailed to the IT department representatives. The questionnaire was administered through e-

mailing, including two e-mail reminders, internal mailing, and face to face distribution in the hospitals

in the week of 26 April to May 1, 2009. There was no incentive provided to the participants. Due to

the scale and stage of presenting the questionnaire users and support staff participating in the pre-test

were not excluded from questionnaire distribution. Participants at the pre-test got the revised

questionnaire just like other potential respondents to the survey. The IT department representatives

would then transfer the materials to the appropriate respondents. One and two weeks after the

questionnaires were sent a follow-up call was made to the IT representatives. As direct follow-up to

the potential respondents is not possible, this has affected the response rate of this study.

The questions were formulated in such a way that they were easily understood and will require 15 to 20

minutes to be completed. To ensure consistency and reliability the definition of the ISs under

examination was given in the cover letter of the questionnaire. Finally, the usable questionnaires will be

processed in SPSS. The population consist of 260 members and the sample size was set at 155 (60% of


11

population) because of feasibility reasons and to achieve the theoretical minima for both multiple

regression and SEM analyses. Response rate will be yielded. Respondents were divided by demographic

factors. According to Hair et al. (2006) the minimum sample size was set at 50 based on requirement for

multivariate regression while taking in consideration that the D&M model is a tested measurement

instrument and a participation of more than 15 respondents per indicator item is the general accepted

ratio. A sample size of 150 to 200 would be sufficient based on literature review but was not feasible

taking the circumstances in the hospital environment in consideration. So the focus was set a minimum

of 100 returning questionnaires yielding a planned response rate of 65%.

1.3.3.3 Data analysis

The proposed model has been tested using both conventional ordinary least square (OLS) regression

(descriptive analysis, data screening and a LISREL-like Structural Equation Modeling (SEM)

techniques with SPSS for Windows statistical application were used for data analysis.

SEM multivariate statistical techniques, which incorporate and integrate multiple regression and factor

analysis, were used. It is divided into two steps: confirmatory factor analysis (CFA) measurement model

estimation and SEM structural model analysis. CFA measurement model estimation has been performed

to ensure the reliability and validity of measures and constructs and SEM structural model analysis to

estimate casual relationships among constructs. It is critical that measurement of each construct is

psychometrically sound before testing the casual relationship among constructs.

Descriptive analysis

Minimal sample for sound estimations size should be 50 for multiple regressions, while SEM requires

150 to 400 respondents for studies with more than five constructs to get representative results out of the

survey at 95% significance and an alpha of 0.05.

Validation

Factor analysis has been performed to check for construct validity. Descriptive statistics is also

performed by computing an ANOVA and t-test was used for significance between groups of IS users

but due to lack of response of the government hospital result are not valid. Hypothesis testing is

performed by using linear regression analyses. Lastly, qualitative analysis of an open-ended question

included in the measurement instrument will also be conducted for triangulation purposes. All

quantitative and qualitative results are presented in the thesis.


12

Because the D&M Model is a multidimensional and interactive model I have used Structural Equation

Model (SEM), with SPSS statistic application as analyses tool. AMOS and LISREL were options too,

but lack of availability forced me to use SPSS to do the analysis. In the D&M Success Model, “system

quality” measures the technical success, “information quality” measures the semantic success, while

“service quality” measures IS support personnel productivity (Deming, 1981-82); Based on both process

and causal considerations the six dimensions of the D&M Model are interrelated. For example higher

system quality is expected to lead to higher user satisfaction and use leading to positive impacts on

organizational improvement. Data was collected from User and IS support staff. The relations identified

in the research Model are depicted in Figure 1, page 5. Descriptive analysis, data testing and Structural

Equation Modeling (SEM) techniques have been used for data analyses. The factors that affect users’

preferences for IS use were also examined on there impact on intention to use and usage of IS. In order

to enhance acceptance and increase usage of IS, it is important to understand how users experience and

decide on the selection and use of IS. So the relationship between IS characteristics, behavior beliefs

and behavior intention was examined to generate thoughts and provide recommendations for system

designers to improve IS system design, development and implementation. According to Fishbein and

Azjen’s Theory of Reasoned Action (TRA), Davis proposed the Technology Acceptance Model (TAM),

which aims to predict system acceptance and to diagnose design problems. According to TAM (figure 2,

page 12), user acceptance of any technology, measured by a person’s intention to use the technology, is

determined by two beliefs, namely: perceived ease of use and perceived usefulness, which mediate the

effects that external variables have on usage intention. In addition, perceived ease of use and use also

influences perceived usefulness.

Figure 2: Technology Acceptance Model

Source: Technology Acceptance Model (TAM) by Davis, 1989


13

Measurement Model estimation

Through data screening I searched for missing data. There are 75 items and 1 open ended question

measuring 10 constructs and I have searched for type of distribution and outliers. In order to ensure that

measures are psychometrically sound, item and constructs reliability, as well as convergent and

discriminant validity, has been assessed. Squared factor loading (SFL) have been used to test item

reliability, while Cronbach’s α and composite reliability ρ were used to test construct reliability. The

criteria which were set to be met are for SFL ≥ 0.50, Cronbach’s α > 0.70 and composite reliability ρ ≥

0.70. All constructs AVEs are presented in a “discriminant validity table” (see Table 7, page 64).

Structural Model Analysis

Path coefficients between constructs will be computed based on maximum likelihood estimation. Chi-

square (X2), Chi-square X2/df, Goodness of fit (GFI), Adjusted Goodness of Fit (AGFI), Norm Fit Index

(NFI), Comparative Fit Index CFI, Root Mean Square Residual (RMSR), and Root Mean Square Error

of Approximation (RMSEA) will be used to evaluate model fit.[1] The results of casual relationship with

estimate path coefficients is presented in a figure of structural model analysis in which the effects among

the different constructs will be presented in a descriptive and correlation matrix (see Table 7, page 64,

Appendix 8, 10 and 13).

1.4 Research material

There are different approaches for the evaluation of ISs. I have analyzed the different models and try to

determine if they are sufficient to solve my research questions (see Chapter 3). After literature review I

have found an extended version of the D&M IS Success Model 2003 the most appropriate model to

perform the evaluation of ISs I want to conduct. To determine the factors influencing the effectiveness

of IS I have used different previous studies and literature regarding ISs and the evaluation of ISs

1.5 Scope of research

The research of this thesis has been conducted in the period January 2009 up to May 2009. The study is

conducted within the framework of the master thesis project for the Master of Business Administration

(MBA) at the FHR Institute of Social Studies in co-operation with the Maastricht School of

Management, with specialization: Management and Accounting. The focus of the thesis is on the

evaluation of the effectiveness (success) of ISs in a hospital environment. The result of this study is an

http://webmail.msm.nl/exchange/rwegasira/Drafts/RE: Questionnaire2 (Delano).EML/1_multipart/1_text.htm#_ftn1


14

evaluation of ISs in use at hospitals which will enable hospital management to improve the effectiveness

of the hospitals in general and the ISs in particularly.

1.6 Chapter Outline

This thesis is divided into 6 chapters. As noticed chapter 1 describes the design of the study, the research

problem and issues, framing this problem within the existing literature pointing out deficiencies in the

literature, and targeting the study for stakeholders. The remainder of this thesis is organized as follows.

Chapter 2 reports the background of the study. Chapter 3 reports the literature review and relevant

approaches for IS evaluating studies and highlights the research motivation. Chapters 4 provides an

overview of DeLone and McLean success framework as well as that of Seddons’ contribution to it and

concludes with the presentation of the proposed model for evaluating the effectiveness of ISs in

hospitals in Suriname in the current study. Chapter 5 describes the operationalization of the success

model in terms of methods, measures, findings and analyses (methodology). In Chapter 6 the emphasis

is on the final analysis, conclusions and recommendations. Finally this Chapter focuses on problems and

limitations of the study.

1.7 Problems and limitations

The present study only sampled users and IS staff from the three hospitals in the study, rather than all

users and IS support staff of all five general hospitals in Suriname. By surveying more users and IS

support staff in other hospitals with the same instrument may obtain more generalized data. In addition,

the research model should be tested for reliability and validity with different setting with various

information seeking contexts.

The research objective is evaluating the effectiveness of IS in a developing country. Taking this in

consideration the result of this study may not be applicable in general but under the given circumstances

and stage of development as Suriname and eventually countries in the same stage of development.

Data was generated by a survey and interview process. Although a quality control was used to identify

inconsistencies in the responses. Data are subject to the limitations of different levels of users and

support staff within and between the three hospitals, including respondent misunderstanding of survey

items, respondent fatigue, or forced-choice responses which may distort survey responses. Random

errors may also be a contributing factor.


15

1.8 Summary

The present chapter addresses the need for a conceptual framework for evaluating Information Systems’

effectiveness or success, which will help management to measure the impact of usage of information

systems an organizational performance.

The contribution of this study will be meaningful for both academics and business practices. First, it

addresses a gap by discussing the test of the D&M Model in a developing country and secondly is one

of the few studies testing an expended version DeLone and McLean (2003) IS Success Model

eliminating the attitude construct in the behaviour model. Thirdly, the findings of this study will be

valuable for staff of organizations involved in the study, because they provide guidance for customizing

and personalizing.


16

CHAPTER 2 BACKGROUND INFORMATION

2.1 Introduction

There is a lot of debating going on regarding the validity of various information systems in Suriname.

The fit, match and congruence between an information system and its organizational context will be of

increasing importance as information systems become more and more integral parts of organizations.

The primary objective of implementing information systems in health care is to help the organization

achieve its goals (Watson, 1993). Gory and Scot Morton (1971) suggest that the primary objective of an

IS in an organization is to support the decision making process.

The impact of information technology on value creation is essential in any organization either through

increasing revenues at marginal cost, or through reducing costs at marginal changes in revenue, and thus

enhancing operating benefits. Measuring the return on investment (R.O.I.) in information systems is

being debated in the IS literature. This debate is also growing in the business community. Evaluating IS

investment therefore is an important issue in organizations, but it is often overlooked. There is emerging

widespread a growing concern in organizations that IS investment does not deliver value and that many

objects do not meet business objectives (Fitzgerald, 1998).

2.2 Hospitals in Suriname

There are five general hospitals (three public and two private) in Suriname of which four are located in

the capital Paramaribo and one in the western district of Nickerie, the second largest city of Suriname. In

November 2008 three of the hospital CEOs visited hospitals and health care institutions in the

Netherlands to evaluate the problems and needs of their hospitals and see how Dutch health care

institutions could support this with their Dutch counterpart “Stichting Equally Equipped”5. Stichting

Equally Equipped” is a foundation founded in 2007 to facilitate community services to Suriname and to

Suriname itself as a whole, with support of hospitals in the Hague area.

In his end of the year speech the Minister of Health, Dr. Celcius Waterberg announced renewing of

medical equipment for hospitals and policlinics, the creation of more training opportunities for health

care workers on different levels and the construction of a new kidney haemodialysis centre in 2009. 5 De Ware Tijd of November 11, 2008


17

According to the minister there is a lot planned for his ministry in 20096. He mentioned infrastructure

which will be financed by the French Developing Fund. New hospitals, policlinics en health care centres

in all districts and the interior (rural areas) will be financed with this fund, while renovation of the

historical part of the “Sint Vincentius Ziekenhuis”, a private hospital, and property of the Catholic

Church will be financed with the Sector Fund.

Minister Waterberg also mentioned private initiative for the construction of a hospital in the district of

Wanica, and elsewhere in the country, medical service centres and health care hotels. He was full of

praise to the preparation of the National Medicare, “Algemene Zorgverzekering (AZV)”. He believes

that the population as a whole will benefit from this system and that co-operation and teamwork of all

stakeholders in the health care industry will lead to positive results.

2.2.1 Information Systems in hospitals in Suriname

I had planned to examine all five general hospitals in Suriname. Afterwards I got approval from four,

but one (St. Vincentius Ziekenhuis) did not meet the IT standard set for this research, so it was not

included in the study. The three hospitals left have some modules of an ERP-system:”Microsoft

Business Solution Navision” installed in the last five years. Since the stage of development of the ERP

systems is not in the same stage, in the sense of modules operationalized I decided to do an overall

evaluation rather than an evaluation of separate modules, which match better with the time given to

complete the study. An overview of the ISs the hospitals have installed is given in Table 3.

2.2.2 Justification of research design

The purpose of the second research question was to investigate which ISs the hospitals in Suriname have

in place in comparison with the international standards and to evaluate how to design the study. I think

that by going wide when it comes to operationalize the constructs of the D&M Model in this initial stage

of evaluation enough interest is created for further research of the separate modules. So I have

concentrated on an overall evaluation approach in this study, to get the opinion of users and IS support

staff.

6 De Ware Tijd of December 31, 2008


18

Hospital Module operationalized Program Implemented

LH 1. Finance

2. Medical Record System

( in-patients, admissions and

operations)

3. Pharmacy system

4. Laboratory system

(only for outpatients)

5. Usual standard applications

1. Navision (ERP)

2. Navision (ERP)

3. Aposys

4. In-house

5. Microsoft Office

2004

DKH 1. Finance (inter-faced with MRS)

2. Medical Record System (MRS)

(in-patients, admission and

operations)

3. Accounts Receivables

(customer items)

4. Liabilities = Suppliers

5. HRM

6. Purchasing

(inventory and stock-keeping)

7. Archive

8. Outpatient System

9. CRP = Central Registration Point

(overall laboratory and roentgen

registration = highest yield for

Outpatients = 1st stage)

10. First aid/Emergency admissions;

1. Navision (ERP)

2. Navision (ERP)

3. Navision (ERP)

4. Navision (ERP)

5. In-house

6. In-house

7. In-house

8. In-house

9. In-house

10. In-house

2006

SZN 1. Finance

2. Medical Record System

(in-patients, admissions and

operations)

1. Navision (ERP)

2. Navision (ERP)

2006

Table 3: Overall Information System of Hospitals

Source: Hospitals in study

Overall ISs contributes to organizational performance. As Overall ISs I consider the “Microsoft

Business Solution Navision”, consisting of the following modules: Finance, Medical Record System (in-

patients, intake and operations), Accounts Receivables, and Out-patient System are part of it and all

other Information Systems operationalized in the three hospitals.

Because of the size of the hospitals involved and their stage and level of automation it takes a lot of

efforts getting them in the study and even after getting approval of their management to do the study


19

their information was flowing very slowly, except from Diakonessehuis. Diakonessenhuis which is a

private hospital in comparison with the other two which are government owned is in a further stage of

automation so it was easier for them to participate optimally in this study.

2.2.3 Attributes which characterize a hospital IS

Regarding attributes, which characterize a hospital ISs I can say that related to de D&M model, which is

a generally used model in the model itself the attributes are almost the same when it comes to the design

of the study. The industry specifics are in the demographics. In content it could differ, but that’s more

on the details, e.g. hospitals consider utilization (occupancy) of beds as their main performance

indicator, which can also be applicable for any organization, when considering occupancy. So thinking

in an overall evaluation (assessment) of applications there is no difference. To distinguish the context of

the hospital environment the questions or statement in the questionnaire could illustrate the context but

due to the difference in stage of implementation, structure and culture in the three hospitals I was forced

to develop a more generalized applicable questionnaire, which covers the overall ISs of the hospitals in

the context and due to these circumstances became applicable also in all kinds of organizations and

industries taking the specifics of health care industry and the hospital environment also in consideration.

That’s why I have formulated all statements to be responded to on a standard Likert scale instead of

having a mixed of scalings, descriptive and open end question. The only descriptive part of my

questionnaire is the demographic part. All other information is quantitative and phrase as a statement on

the Likert scale using the scale of 1 to 7 due to a mix of experience, non-experience and professional

users and IS support personnel working in different fields and different levels. According to literature

study on a larger scale respondent will indicate more precisely what they think about the statement than

on a shorter scale of 5 or 3, which is more for a group of respondents which is highly experienced or for

professionals involved with the topic. Another argument is that IT using ERP systems is the fulfillment

of high level of standardization and so understanding or translating all specifics of any organization and

industry in one language.


20

CHAPTER 3 LITERATURE REVIEW

3.1 Introduction

This Chapter deals with the theoretical background of ISs and the Evaluation of these Systems. First, a

general overview of ISs is given. Then, background information is given on Medical Information

Systems (MEIS) as one of most critical information systems for hospitals followed by relevant

evaluation approaches and methods.

3.2 Information Systems

An Information System (IS) can be defined technically as a set of interrelated components that collect

(or retrieve), process, store, and distribute information to support decision making and control in an

organization7. In addition to support decision making and control, information systems may also help

managers and workers to analyze problems, visualize complex subjects, and create new products.

Information Systems contain information about significant people, places, and things within the

organization or in the environment surrounding it (see Figure 3). Information is data that have been

shaped into a form that is meaningful and useful to human beings. Data in contrast, are streams of raw

facts representing events occurring in organizations or the physical environment before they have been

organized and arranged into a form that people can understand and use.

Figure 3: Information System

Source: Kenneth C. Laudon and Jane P. Laudon, 2000

7 Kenneth C. Laudon and Jane P. Laudon, Management Information Systems, 6th Edition, Organization and

Technologogy in the networked enterprise, Prenhall 2000

INPUT OUTPUT PROCESS

FEEDBACK

IINNFFOORRMMAATTIIOONN SSYYSSTTEEMM

EENNVVIIRROONNMMEENNTT

CCuussttoommeerrss SSuupppplliieerrss

RReegguullaattoorryy SSttoocckkhhoollddeerrss CCoommppeettiittoorrss AAggeenncciieess

OORRGGAANNIIZZAATTIIOONN


21

3.3 ERP Systems

Enterprise Resource Planning Systems (ERP) system are defined as “configurable information systems

packages that integrate information and information-based processes within and across function

organization” (Kumar and Hillegensberg, 2000).8 They promise the seamless (dovetailed) integration of

all the information flowing through an organization – financing and accounting information, supply

chain management, human resource information, customer information and the like. For managers who

have struggled with incompatible information systems and inconsistent operating practices, these

organizations-wide systems hold the promise of integrating all aspects of information and process within

and around the organization. The strategic value of ERP systems and the resources organizations invest

in then make evaluating and monitoring of their success important to both practitioners and researchers.

This is particularly true in view of the many reported cases of failures in implementing such systems.

There is a small but growing literature on the impact of ERP systems. The few studies are interviews,

case studies, and industry surveys (Tasi et al, 2005). The companies involved reported substantial

performance improvement in several areas thanks to their ERP systems, such as their ability to provide

information to customers, cycle times, and on-time completion rates. Based on Galbraith’s information

processing perspective (1973), Gattiker and Goodhue (2000) group ERP benefits into four categories:

1. Improvement of information flow across subunits through standardization and integration of

activities;

2. Enabling centralization of administrative activities such as accounts payable and payroll;

3. Reducing IS maintenance cost and increase the ability to deploy new IS functionality;

4. Facilitation transformation from inefficient business process toward accepted best of practice

processes.

Since the realization of ERP benefits depends on individual practices, studies on individual impacts are

conducted to measure the influence of ERP on employees and their productivity. Individual impact is

measured in terms of effectiveness, quality of work, and decision-making performance. For the ERP

impact on organizations, as Umble et al. (2003) suggest, it is important adopting effective measures

indicating how the ERP affects on-time deliveries, gross profit margin, customer order-to-ship time,

inventory turns, and so on. Therefore, organizational performance is measured in terms of reduction of

8 Moshe Zvan, Faculty of Management, Tel Aviv University, Israel, User Satisfaction in ERP systems: Some Empirical

Evidence, Working paper no 22/2003, Research no 07010100, (2003)


22

inventory cost, decrease in order-processing time, improvement on sales margin, and real-time sharing

of transaction information. Meanwhile, it is also imperative to measure the business process

improvement because ERP systems are often implemented to take advantages of best practice solutions

to enhance business operations.

3.4 Approaches and methods in use for the evaluation of Information Systems

3.4.1 Introduction

Evaluation represents a critical issue in ISs research and practice. Unfortunately in searching for a

success measure, rather than finding none, there are nearly as many measures as there are studies. The

reason for this is understandable when one considers that ”information”, as the output of and

information system of hospitals, can be measured at different levels, including technical level as the

semantic level and the effectiveness level. Shannon and Weaver (1949) defined the technical or system

level of communication as the accuracy and efficiency of the communication system which produces the

information. In this context, the semantic level as the success of the information system in conveying the

intended meaning; the effectiveness level, on the other hand it refers to the effect of the information on

the receiver.

According to Hamilton and Chervany, 1981) System Effectiveness has two general views: the goal-

centred view and the system-resource view. The goal-centred view focuses on the task objectives of

the system, or the organizational units utilizing the system, and then develops criterion measures to

assess how well the objectives are being achieved. Effectiveness is determined by comparing

performance with objectives. An example of this view of system effectiveness would be comparing

actual cost and benefits to budgeted cost and benefits. The system-resource view focuses on the

attainment of a normative state, e.g. standards for “good” practices. Effectiveness is conceptualised in

terms of resource viability rather than in terms of specific task objectives. For example, system

effectiveness in terms of human resources might be indicated by the nature of communication and

conflict between IS and user personnel, user participation in system development, or user job

satisfaction. In terms of technological resources, system effectiveness might be indicated by quality of

the system or service level. Just like Shannon and Weaver (1949), Hamilton and Chervany (1981)

conclude that the system resource model recognizes that systems fulfil other function and have other

consequences besides accomplishing of official objectives, and that these need to be considered in

assessing system effectiveness. Remarkable is that they don’t categorize that as efficiency just like

Shannon and weaver did but also categorize that as system effectiveness.


23

So approaches to the evaluation of IS could be distinguished in those concerned with efficiency and

those concerned with effectiveness. In practice, the two views should converge. In order to explain the

success, or lack of success, in meeting objectives, the semantic of systems resource need to be

investigated. In the next paragraphs I will discuss the approaches concerning the effectiveness of IS

which are considered of a higher level of evaluation than those of efficiency and are subject of this

study. Efficiency is concerned with lower-level, machine (or-quasi-machine) aspects, usually considered

outside the business or organization context, while effectiveness is more concerned with thinking about

the operation of the ISs within their context. This means examining in more detail the way they are used

by people in the course of their work and, in particular, examining the contribution of the system to the

organization, especially in financial terms.

3.4.2 A conceptual hierarchy of system objectives9

System objectives broadly define the goals of the IS and embody the hierarchy of objectives for the

organization, running the gamut from a single strategic statement which is quite conceptual, to detailed

operational goals for the individual IS development project. Typically, the requirements definitions or

design specification for the information system is an operational description of the system objectives

and constitutes a reference point for IS development and operations personnel. A conceptual hierarchy

of objectives is depicted in Table 4, page 39. One of the primary objectives of IS function is to develop

and maintain information system that will enhance the organization’s ability to accomplish its

objectives. As mentioned above other researchers also mentioned that accomplishment of organizational

objectives can be evaluated from two perspectives for a specific information system, namely:

1. The efficiency with which the IS development and operational processes utilize assigned

resources (staff, machines, materials, money) to provide the information system to the user;

2. The effectiveness of users, or the users’ organizational unit, using the IS in accomplishing

their organizational mission

For the purpose of this study the emphasis is on the effectiveness-oriented objectives of this hierarchical

framework as depicted in Table 4, page 39. Systems’ effectiveness is ideally assessed in terms of

9 Scott Hamilton and Norman L. Chervany, MIS Quarterly, September 1981


24

information system’s contribution to accomplishment of organizational objectives, i.e., its effect on

organizational performance (Level 3).10

The accomplishment of IS objectives can be assessed by performance measures. The purpose of next

paragraph is to provide an overview range of approaches for evaluating ISs’ effectiveness and to point

out the human and organizational issues attached to measurement and evaluation.

3.4.3 Relevant approaches to evaluate Information Systems

Some relevant approaches to the evaluation of the effectiveness of IS are:

1. Utilisation;

2. Cost-benefit analysis;

3. Non-financial approaches to IS effectiveness:

• Objective analysis;

• User or customer satisfaction surveys (Ives et al. 1983; Doll and Torkzadeh,1988);

• Service Level Monitoring

4. Combinatorial approaches:

• Multi-objective, multi-criteria analysis (Chanler, 1982: Vaid-Raizda 1983);

• Balanced scorecard (Kaplan and Norton, 1992);

• DeLone and McLean’s IS Success Model 1992;

• Updated DeLone and McLean’s Model 2003);

• The Seddon Model 1997

• Technology Acceptance Model (TAM) (Adams et al., 1992; Davis et al.,; Gefen and

Straub, 1997;; Amoako-Gyamah, 2004);

Sub 1: Utilisation

This measure, which can be seen to be the border between the efficiency and effectiveness zones, is

rarely used now, but it is worth mentioning for completeness. The argument here is that, if an

information system is used a lot, then it is in some way effective. A comparison here would be with

hospital beds occupancy. One could argue that months with most or all of the beds occupied are

“successful” and this is used in the hospital environment as a rough measure of effectiveness.

10 Organizational performance may include considerations of effects on the external environment. For example,

manufacturers typically include product safety (e.g., consumer injuries from automobile defects) within organizational

performance objectives, and even employ information systems to rack current owners and safety records.


25

There are a number of problems with this type of measures for information systems effectiveness:

• if utilisation is measured in terms of time online or number of keystrokes this is a poor

measure as users normally wish to minimize their effort (time online or keystrokes);

• in other context, perhaps low level data entry, operators may be paid on the basis of the

number of keystrokes and it is their financial interest to increase that number, regardless of the

benefits to the organization. In the 1970s and early 1980s, word processing operators paid in this

way used to cursorily tap the space bar for the same reason;

• certain systems may be mandatory: for example, hotel reservation clerks have no choice and

use the reservation system, no matter how bad it is;

• it ignores the value or importance of the task being performed: for example, some tasks are

performed rarely (perhaps annually or only in emergencies), but they could be crucial to the

running of the business.

Sub 2: Cost-benefit analysis

Cost benefit analysis quantifies the system’s effect on organisational performance in terms of dollar,

e.g., direct cost savings, tangible financial benefits. Cost/benefit analysis often used in capital budgeting

to assess the return on investment (R.O.I.).

Sub3: Non-financial approaches to IS effectiveness

Objective analyses or Post Installation Review

The focus of a Post Installation Review (PIR) is often on assessing whether the system meets the

required definition, i.e., “does the system do what it is designed to do?” However, the scope of the PIR

may include a post hoc review of the development and operations processes, an assessment of the

information and support provided, and analysis of the actual use process, and cost-benefit analysis of the

system effects on user performance.

User (or customer) satisfaction surveys (Ives et al., 1983; Doll and Torzadeh, 1988):

User satisfaction surveys, through questionnaires and/or interviews, focus on assessing the user’s

perceptions of the information and support provided by the IS support function. User’s satisfaction

surveys typically assess such aspects as the quality of reports, timeliness quality of service, and user

communication.


26

Service Level Monitoring

Service level monitoring focuses on assessing the information and support provided to the user based on

the terms established between IS support organization and user personnel. Assessments of the

information provided include turnaround times, response times, and error rates. Assessment of the

support provided includes the time required to respond to user problems and request for changes.

Sub 4: Combinatorial approaches

Recognizing the limitations of a single measure, the approaches outlined in this section attempt to

combine various techniques

Multi-objective, multi-criteria analysis (Chandler, 1982; Vaid-Raizada 1983)

Recognizing how difficult it is to attach financial figures to cost and benefits, the technique uses instead

the notion of utility. Managers can rank different outcomes according to their preferences and attach

weights to particular goals. This approach recognizes the different objectives of different stakeholders

(see organizational issues in paragraph 3.7) and can be used to expose conflicts. Software is available to

manipulate the weighted preferences and in theory, it can show the outcome that maximises total utility.

However, there are clear dangers in rusting the resolution of sensitive political issues to such a piece of

software; for example, should all stakeholders be given the same weight?

Balanced scorecard (Kaplan and Norton, 1992)

Managers using this approach try to recognize and measure aspects in addition to the obvious financial

ones. It attempts to set up an integrated set of performance measures that encapsulate an organization’s

strategic objectives. These measures need to be tailored to the individual organization and are drawn

from four quadrants:

1. financial – relevant financial measures;

2. customer – measures of customer service;

3. internal process – including issues such as quality and productivity;

4. learning and growth – measures of ability to improve and innovate.

Although the balance scorecard was originally marketed to evaluate entire organizations, it can be

adapted for the evaluation of information systems (Willcocks and Greaser, 2001).


27

On the positive side:

• it can be customised for individual organizations;

• it stretches beyond the financial aspects;

• it is closely related to an organization’s core objectives.

On the negative side:

• it can require a considerable amount of time and effort initially to determine the objectives and

design the measures;

• it requires skill and experience to do it effectively;

• it does not incorporate risk identification and management.

DeLone and McLean IS Success Model 1992

DeLone and McLean provide a general and comprehensive definition of IS success, that covers different

perspectives of evaluating Information Systems. According to DeLone and McLean, measurement of IS

success is critical for understanding the value and efficacy of IS management actions and IS

investments. They review the existing definitions of IS success and their corresponding measures and

classified them into six mayor categories. So, in 1992 they proposed a taxonomy and an interactive

model as the framework for conceptualising IS success. Driven by the need of a process to understand

IS and its impacts, they developed a multidimensional measuring model with interdependencies between

the different success categories (DeLone & McLean IS Success Model 1992). According to DeLone and

McLean the main dependent variables were: Use, Users satisfaction, Individual Impact and

Organizational Impact, while the main independent variables were: System Quality and Information

Quality (see Figure 4, page 28).


28

Figure 4: D & M IS Success Model 1992

Source: W. DeLone and E. McLean, 1992

The description and examples of measures for the six dimensions are:

1. System Quality; denotes system performance like data accuracy, system efficiency, response

time, etc.

2. Information Quality; refers to the quality of the IS products, such as currency, relevance

reliability, and completeness;

3. Use; refers to the frequency an information system is used, examining items like the number

of functions used, frequency of access, and amount of connect time;

4. User Satisfaction; records the satisfaction level as reported by system users, including

overall satisfaction and satisfaction of interface, etc.

5. Individual Impact; refers to measuring the impacts brought about by the information system

on individual users, such as changes in productivity, decision model, and decision making;

6. Organizational Impact; requires the evaluation of the changes by the information system to

the organization, such as decrease in operating cost, savings in labor cost, and growth in

profits.

According to the D&M IS success model, both system quality and information quality influence use and

user satisfaction which in turn shape the impact of the system on individual users and the organization.

DeLone and McLean had consulted various sources – notably the communications research of Shannon

and Weaver (1949). The information “influence” theory of Manson (1978), as well as the empirical

management information systems (MIS) from 1981 – 1987 – to postulate their comprehensive and

multidimensional model of success. In spite of the passage of time since the Shannon and Weaver frame

work in 1949 and the Manson’s extension in 1978, both appear valid today for incorporating into a

workable framework of IS success model. The reason for the existence of different measures for IS


29

success is understandable when one consider that “information” as the output of a system can be

measured at different levels – the individual/personal/personnel level, the technical level, the semantic

level, and the effectiveness level – and different stakeholders are involved at each different level.

The D&M IS success model provides a more comprehensive approach to measure IS success and

organizes previous research into a more coherent model. It helps explain the often-conflicting results

and provide a base for related studies. Although they proposed the model and the interrelationships

between the multi-dimensions, DeLone and McLean did not test the model empirically. Many

researchers, such as Seddon and Kiew (1994), Goodjue and Thompson (1995), Taylor and Todd (1995),

Teng and Calhoun (1996), Igbaria and Tan (1997), Igbaria et al. (1997) and Wixom and Watson (2001),

have studied the associations between the measures identified in the IS success model by DeLone and

McLean (1992), and this immense popularity of D&M IS success model eloquently speaks of the

importance of IS success researches.

As I have seen throughout the study each of the dimensions of the D&M Model 1992 is likely to face

measurement problems, unless great care is taken their aggregation is likely to be even more

problematic.

Updated DeLone and McLean IS Success Model 2003

In response to the progress in IS applications, DeLone and McLean revised their original model and

proposed an update version in 2003. Service Quality was added into the success model, and Individual

Impact and Organizational Impact were combined into a single variable named “Net Benefits as shown

in Figure 5, page 30. To facilitate the advancement of its applications IS need not only to provide users

information products but also meet users’ flexible information requirements. Service Quality is thus

added into the modified model to measure the service-level success since system quality focuses more

on technology-level measure. Since it is difficult to describe the multi-dimensional aspects of IS use -

mandatory or voluntary use, informed or un-informed use, effective or ineffective use, DeLone and

McLean further suggested that “Intention to Use” may be adopted as an alternative measure for IS use in

some contexts. Certain net benefits can occur as results of IS use or Intention to Use and User

Satisfaction. Net benefits are the most important success measures as they capture the balance of

positive and negative impacts of ISs on organizations. Last but not least, positive net benefits will

encourage ISs system use intention and increase user satisfaction, while negative net benefits can

decrease the intention to use and IS user satisfaction.


30

So, the updated model also consists of six interrelated dimensions of IS success: Information Quality,

System Quality and Service Quality, Intention to Use and Use, User Satisfaction and Net Benefits (see

Figure 5). They suggest that ISs can be evaluated in terms of system-, information-, and service quality;

these characteristics affect the subsequent user or intention to use and user satisfaction. As a result of

using the system, certain benefits will be achieved. The net benefits will positively or negatively

influence users’ satisfaction and the further use of information system.

The Delone and Mclean Model of Information Systems Success: A ten year Update, William H. DeLone

and Ephraim McLean. The DeLone and McClean model proposes that System Quality, Information

Quality and Service Quality singularly and jointly affect both System Use and Users Satisfaction.

Additionally, the amount of System Use can affect the degree of User Satisfaction positively or

negatively – and the degree of User Satisfaction also affects System Use. System Use and User

satisfaction are direct antecedents of Net Benefits (individually, organizational and Societal).

Figure 5: Updated D & M IS Success Model 2003

Source: Nunally, J. Psychometric Methods, 2nd

edition, McGraw-Hill, New York: NY, 1978

User Satisfaction is affected by System Quality, Information Quality, and Service Quality. User

Satisfaction may be affected by Use positively or negatively, as well as the reverse is possible. System

Use and User Satisfaction lead to the realization of Net Benefits, while the Net Benefit will then affect

System Use and User Satisfaction.

The Seddon Model 1997

A principal difference between Seddon’s and the D&M model lies in the definition and placement of

Use. Seddon (1997) argues for the removal of “information use” as a success variable in the causal

success model, claiming that use is a behavior, appropriate for inclusion in a process model but not in a


31

causal/variance model. Seddon and Kiew (1994) recommend replacing use with usefulness, stating

that use only affects satisfaction when use is voluntary. They placed use outside a reviewed model of

systems success as it was deemed more a charact eristic of user behavior than a measure of system

success (Seddon, 1997).Seddon (1997) identifies three distinct models intermingled D&M IS success

model, each reflecting a different interpretation of IS Use.

Based on a number of studies that attempted to validate the original D&M model, Seddon concluded

that there is a conceptual difficulty in D&M’s 1992 model because it combines both process/causal and

variance models. The construct “Use” in the model actually has three meanings and the meaning shifts

in different parts of the model. Seddon re-specified D&M’s model into a pure variance model (see

Figure 6). Part of the model that is relevant to the current study is shown elaborated in Figure 1, section

3 as second variance model, the Behavior mode of Use.

Figure 6: Seddons IS Success Model 1997

Source: Peter Seddon, 1997

Several researchers added new variables to the original model, combined exiting variables, or changed

the casual path in the DeLone and McLean model. Some studies identified conflicting results regarding

(not evaluated as either good or bad) Partial behavior model of IS Use

Observations, Personal Experience, and

Reports from Others

IS Succes Model

(Partial basis

for revised

expectations)


32

relationships among DeLone and McLean six original variables. For example, a positive relationship

was reported between user satisfaction and individual impact. However another study did not find a

relationship between the same variables. Figure 1, page 5 illustrates the extended DeLone and McLean

model to be empirically analysed in this research.

The Technology Acceptance Model (TAM)

One of the key measures of implementation of success is achieving the intended level of IT usage.

System usage reflects the acceptance of the technology by users (Venkatesh, 1999). The Technology

Acceptance Model (TAM) has served as basis for past researches on IS dealing with behavioural

intentions and usage of IT (Adams et al., 1992; Davis et al., 1989; Gefen and Straub, 1997; Amoako-

Gyampah and Salam, 2004) Figure 2.

As a well recognized theoretical basis for studying users acceptance, TAM (e.g. Davis, 1989) proposes

that users’ perceptions of a system’s ease of use and usefulness can influence how quickly and

efficiently users will adopt the new technologies. Thus, according to TAM, the easier a technology is to

use, and the more useful it is perceived to be, the more positive the user’s, attitude and intention towards

using the technology. Consequently, the usage of the technology increases. Recently, researchers have

explored personal and situational factors that influence users’ perceptions. One such factor is the users’

perception of his/her computer self-efficacy, i.e., proficiency at using technology (Igharia and Livari,

1995; Compeau and Higgins,1995; Venkatesg and Davis, 1996’; Venkatesh, 2000).

While users’ perceptions of computer self-efficacy have been shown to be important in their system

perceptions, knowledge workers need both computer and task proficiency to apply a workplace system

efficiently and effectively in performing their jobs. Thus, their perceptions of self-efficacy related to

both computer technology and the underlying task are likely to affect their perceptions about the system

and their intentions to use it as intended by the system developers.

3.5 Approaches to the evaluation of Medical Information Systems (MEIS)

Different frameworks and theories of information systems science, health economics and medical

informatics have been mix through into the area of medical information systems science, where they

have been lacking so far.

Previous research has applied a number of different approaches to the evaluation of medical information

systems, and information systems in general. Research has also provided taxonomies of different


33

evaluation approaches (DeLone et al. 1992 and update 2003; Fineberg et al. 1977; Grover et al. 1997;

Guyatt et al, 1986; Van der Loo et al. 1992). Taxonomies suggest that there is an effect between

different levels (see figure 7). For example, with the help of better diagnostic information physicians

might make better therapeutic decisions.

Figure 7: Levels of Clinical Efficacy related to use of diagnostic technology (Fineberg et al, 1997)

Source: Pekka Turunen and Hannu Salmela, 1998

User satisfaction indicates users’ satisfaction with information and system. Individual impact can be

seen as impact on physician’s diagnostic decision but also as other impacts, for example on time

reduction or increase. Patient outcome means change in patient’s health and organizational outcome for

example changes in personnel time, overhead cost, and interest cost.

Many researchers in information systems in general and medical information systems science in

particular have ended up concluding that the best way to evaluate information systems is to measure

organizational or patient outcome (Van der Loo et al. 1995; Hamilton et al. 1981; Ives et al. 1983). Such

assessment applies economic evaluation, especially cost-benefit analysis. Because information systems

may have organization-wide, intangible and long lasting effects and cost, economic evaluation of

information systems is considered difficult (Saarinen, 199349: see also Ives et al. 1993). The difficulty

in evaluation cost and benefits is often seen as a motivation to develop other methods to evaluate

information systems (Saarinen 1993). On the other hand, cost-benefit analysis focuses on the outcomes

or ends. Other evaluation approaches focus more on the implementation process, which produces the

outcomes.

A comprehensive evaluation of an information system requires multiple measures (DeLone et al. 1992;

Saarinen, 1993). Both improvements in information, improvements in individual decisions or actions, as


34

well as improvements in organization level can indicate IS effectiveness as success. The selection

between these measures is to a wide degree dependent on the values and objectives driving the

evaluation. The type of systems and the economic considerations of gathering the evaluation data need

to be considered as well.

In spite of the disagreement among researchers on the assumptions and evaluation factors of IS, the

following factors represent the common factors to evaluate health care information systems

performance.

a) IT integrated in IS;

b) software quality;

c) investment in training;

d) aligning corporate goals with technological investments;

e) customer services;

f) productivity;

g) cost-benefit analysis.

3.6 Measurement

It is easy to confuse measurement with something very precise, objective and “scientific”. However, as

Mason and Swanson (1981) make clear, there are many subjective and arbitrary aspects to measurement.

They argue that there are four stages to measurement, each permits considerable subjectivity:

1. identify the underlying dimension – decide what to measure e.g. functionality,

profitability;

2. define it by unit/scale – decide how to measure the dimension, e.g. there are various ways of

measuring functionality, etc.;

3. execute measurement – this refers to the methodology of carrying out the measurement and

includes: when to measure? How long? How often? and so on;

4. compare against criteria – the value of measurement must be judged as good or bad against

certain criteria and these may be set subjectively.

There are also temptations to:

• measure whatever is easier to measure;

• measure whatever gives the desired result.


35

And hence, Tapscottt’s First Law of Cost-Justifying Office Systems:

“The probability of a chooser accepting a cost-benefit analysis directly proportional to the degree to

which he/she is favourable inclined to the technology anyway.”

Tapcott (1982)

The final point is that what gets measured gets managed – managers and their staff are often judged

(evaluated) on certain measures (e.g. productivity) and these are naturally the issues they tend to focus

on in their work, neglecting issues (e.g. health and safety) on which they are not measured.

3.7 Organizational issues

IS investment appraisals normally result in the direct allocation, or re-allocation, of resources to

particular groups (or departments) within the organization. These resources include finance, jobs,

equipment, accommodation and prestige, and may have significant implications for the future both of

groups and individuals within the organization. Thus, it is often in the best interest of certain groups (the

winners) for a project to proceed and not in the best interest of other groups (the losers). Hence, together

with the uncertainties surrounding measurement and evaluation, there is potential battlefield for

organizational politics (see Willcocks and Lester, 1999)

“The rational elements are tools used by participants to gain new ground or to project ground already

won. They also serve as “facades” to mask political motives and legitimise self-interest.”

Franz and Robey (1984)

In many organizations, users departments increasingly have to use their budgets to pay for IS

developments and clearly it is normally in their interest to reduce this expenditure and save their

budgets. This provides further scope for political manoeuvres.

Other types of IS evaluation (e.g. post-implementation) may also have (slightly more) indirect

implications for the allocation of resources, including prestige and status. For example, organizations

will often carry out a post mortem to allocate blame, especially where the system development has failed


36

badly. Where the system is a success, this may also stimulate a re-allocation of resources. In each case,

there are considerable organizational and political implications.

3.8 Summary

Organizations are particularly concerned about the effectiveness of their Information Systems in terms

of evaluating them in their business context. In practice, the most common approaches are the financial

ones that stem from the notion of cost-benefit analysis, especially those concerned with investment

appraisal. However, there are some non-financial approaches as well as combinatorial approaches that

seek to get beyond mere finance. Nevertheless all approaches to IS evaluation must recognize both the

subjective nature of measurement as well as the organizational aspects that surround IS evaluation.


37

CHAPTER 4 HYPOTHESES DEVELOPMENT AND PROPOSED

MODEL

4.1 Introduction

Evaluation of ISs is an integral part of the management control process. ISs in general, and especially

ERP systems are used organization-wide, and are part of organization’s automation packages in all

aspects of the organizational/business process. Due to its multidimensionality, its quantitative (scale)

and qualitative aspects, and multiple, and often conflicting, evaluator viewpoints it is expected that

measures of ISs success require different evaluation approaches dependent on the stage of development

of the organization and the ISs itself. At the pre-implementation stage and implementation stage subject

of study and evaluation (assessment) is the IS project. In the context of project management, an IS is

deemed effective (successful) if it accomplishes its objectives - not only to the management but also to

users, developer(s), and all personnel involved in IS implementation, use and support -, and the planned

activities are completed within the allocated time and budget. Yet the scenario at post-implementation

stage is quite different from the pre-implementation and implementation stage. In post-implementation

stage ISs provides various degrees of system, information and service quality. New users experience

these quality features by actually using the IS. Users are ether satisfied or dissatisfied with quality

features of IS. As discussed in the literature review in Chapter 3 measurement of overall ISs and ERP

effectiveness in post-implementation stage is more complicated than any unit or departmental IS

success since there are more users and departments with different needs involved.

4.2 Comparison of Evaluation Approaches

Several IS evaluation approaches currently employed to assess system effectiveness can be compared by

mapping them into the conceptual hierarchy (Hamilton and Chervany, 1981) (see Table 4. While many

different approaches have been suggested for effectiveness measurement, a selection of the frequently

employed in IS organizations were discussed in Chapter 3.The Scope of each evaluation approach is

depicted in Table 4 in terms of objectives being evaluated.

Evaluation of approaches was previously categorized as being either semantic level or effectiveness

level, and two approaches can be compared using the conceptual hierarchy. Effectiveness level is

objective oriented and determines whether the system has achieved desired “outcomes” or result”

objectives and focuses on assessing the accomplishment of Level 2 and 3 effectiveness oriented


38

objectives. Both semantic and effectiveness evaluative approaches are typically used in providing

evaluative information on system effectiveness.

Evaluation of IS effectiveness has been difficult due to its multidimensionality, its quantitative and

qualitative aspects, and the multiple evaluator viewpoints. Several suggestions can be made for IS

practitioners and IS researchers for evaluating IS effectiveness. For IS practitioners it is important to

incorporate multiple viewpoints of multiple objectives and performance measures into the assessing

system benefits to user organizational performance.

For researchers, the dependent variable in many studies is of the IS success or system effectiveness. To

measure system success it is important to incorporate multiple success criteria (objectives), differing

viewpoints on each criterion, and the varying importance of each criterion. In addition, the use of a

standardized validated instrument to measure system success is encountered in order to make research

results comparable. As recommended in previous studies and also in my analyses of the different

approaches, I decided to incorporate multi-evaluator viewpoints on system effectiveness into my

evaluation approach while using a combinatorial approach in my discussion in next paragraphs in an

attempt of measuring the dependent variable “IS effectiveness” or “IS Success”.

4.3 Applying the IS Success Models in research context

As recommended by previous researcher and based on literature review discussed in Chapter 3 I

consider the logic of an updated D&M IS Success Model a good framework for my research as

hospitals are in a post-implemental stage of both their ISs in general as well as the ERP system. Since

ISs implemented in the hospitals are considered as a whole to achieve organizational objectives all three

quality dimensions, namely system quality, information quality and service quality need to be included

in the proposed model.

Selection Procedure

Despite the ever increasing number of health care information systems, publish evaluation studies are

scarce. To my knowledge, no evaluation framework has been proposed specially for patient care

information systems. In my literature review I have examined a wide range of attributes for evaluating

such systems (see Appendix 1). My review has shown that the dimensions of success for IS defined by

DeLone and McLean are applicable to inpatient care IS. It is unlikely that one single factor is decisive in

the success of IS. Therefore a multidimensional construct, as proposed by D&M, is more appropriate for


39

the evaluation of IS. Studies have shown that the D&M frame work is useful in evaluating patient care

IS and should be explored in future evaluation research, with modifications to include contingent

factors, such as user involvement during system development and implementation and organizational

culture.

Levels in Conceptual Hierarchy Measures to Measure accomplishment of objectives

Hierarchy of

objectives

Service

Level

Monitoring

User

Satisfaction

Survey

Post

Installation

Review

Cost

Benefit

Analysis

3. Resource Investment Cost-

Benefit

Analysis

2. Production capacity

1. Resource Consumption

Eff

icie

ncy-

Ori

ente

d O

bjec

tives

0. Requirements defini-

tion of IS

1. Information and

Support provided

Service

Level

Monitoring

User

Satisfaction

Survey

2. Use Process and

User performance

Utilisation

Eff

ectiv

enes

s-O

rien

ted

Obj

ectiv

es

3. Organizational

Performance

1. P

ost i

mpl

emen

tatio

n R

evie

w

2. O

bjec

tive

anal

ysis

3. C

ombi

nato

rial a

ppro

ache

s

Cost-

Benefit

Analysis

Table 4: Comparison of Evaluation Approaches in/ based on conceptual hierarchy

Source: Hamilton and Chervany, 1981


40

4.3.1 Operationalization of variables and proposed IS Success Model

In order to empirically analyse the DeLone and Mclean Model, all variables in the model must be

operationalized. Existing measures of information system success that have acceptable psychometric

quality are used, extended with some identified in this study to strengthen the variables (constructs) and

re-structured for the context, while a general and standard approach is adopted. DeLone and McLean

recommended tested and proven measures from previous research and that is what has been done in this

research basically, while some re-structuring and shaping have been performed to get better tests results

both in this research as well as in future research no matter what the context. A brief description of the

selected items for the measurement instrument, the questionnaire from tested survey instruments in

information system literature is as follows:

Based on the Updated D&M Model 2003 and Seddons’ IS Succes Model 1997, discussed in Chapter 3,

75 items divided 56 dimensions (see Appendix 1) were used to operationalize the 6 success measures.

Items from attached questionnaire (see Appendix 3) were used to operationalize System Quality,

Information Quality and Service Quality, Perceived Ease of Use, Perceived Usefulness, Intention to Use,

Benefits of Use (System Usage), User Satisfaction and Net Benefits.

As depicted in Figure 1, the resulting model comprises nine interrelated constructs in an IS success

model and Behaviour model of IS Use, namely: System Quality, Information Quality, Service Quality,

Benefits of Use and Net Benefits for the IS Success model and Perceived Ease of Use, Perceived

Usefulness and Intention to Use for the Behaviour model of Use, while Use as behaviour also was

measured for verification reason in the context.

4.3.1.1. Variables of IS Success Model

System Quality (SQ) measures the desired operational characteristics of an IS, e.g. whether or not there

are bugs in the systems, the consistency of the user interface, ease of use, response rates in interactive

systems, quality of documentation, and sometimes, quality and maintainability of the program code”

(Seddon and Kiew, 1997). Seven dimensions and seven items were used to operationalized the System

Quality variable and are shown in Part 1 of the questionnaire (see Appendix 2, part 1): timeliness,

accuracy, ease of use, consistency (response time and availability), flexibility, integration and reliability.


41

Information Quality (IQ) is "concerned with how good is IS in terms of output. Patient information for

example must be precise, understandable, complete, and available on time, to name few; issues such as

format, content, timeliness, relevance, accuracy and reliability of information generated by an

information system. Not all applications of IT involve the production of information for decision

making (e.g., a word processor does not actually produce information) so Information Quality is not a

measure that can be applied to all systems (Seddon and Kiew, 1997). Those six dimensions were used to

operationalize the Information Quality construct in six items.

Service Quality (SRQ) is “concerned with consistency in what a variable should measure; a global

judgment of attitude relating to the superiority or excellence of service. The perception of service quality

results from the comparison of expectations with performance. In the hospital context Service Quality

refers to the perceived quality of the support and the service provided by the provider of the system

and/or the internal staff responsible for IS support and maintenance. This construct was measured in five

dimensions and a five-item-scale developed in the marketing area and then adapted to the IT context. So,

five dimensions and five items were used to operationalize the Service Quality variable as shown in part

3 of the questionnaire: responsiveness (service consistency), reliability, empathy, assurance (adequacy)

and support.

System Usage (SU) examines the actual use of information systems, the extent of use of information

systems in the users' jobs, and the numbers of information system packages used in the users' jobs. It is

expected that resources such as human effort will be consumed as the system is used. IS Use might be

measured in hands-on hours spent analyzing reports, frequency of use, number of users, or simply as a

binary variable: use/none-use. This variable was operationalzed in three dimensions; intensity,

frequency and scope of current research.

As mentioned before in this study this variable was operationalized applying Seddon’s Behavior model

and results were compared with respondents answers on Intention to Uses, so there is a verification to

what instant System Usage in the IS Success Model matches with Intention to Use in de Behavior Model

and actual Use. System Usage supported by the variables “Perceived Ease of Use”, “Perceived

Usefulness”11 and “Intention to Use” from Behavior Model were operationalized in twelve dimensions


42

and fourteen items. System Use and its supporting variables in this study are shown in section 4, 5, 6 en

7 of the questionnaire.

User Satisfaction (US) examines the successful interaction between the information system itself and

its users. User Satisfaction is an important means of measuring user’s opinions about ISs. This variable

was operationalized using the following dimensions: satisfaction with the specifics, content, accuracy,

format, ease of use, timeliness, information satisfaction and overall satisfaction. The instrument consists

of eight dimensions and nine items. User Satisfaction is a subjective evaluation of the various

Consequences: Individual, Organizational, and Societal (depicted in the top-right corner of Figure 6)

evaluated on a pleasant-unpleasant continuum. Of all the measures in Figure 1 and 6 User Satisfaction is

probably the closest in meaning to the ideal Net Benefits measure (Seddon and Kiew, 1997).

Net Benefits to Individuals, Organizations, and Society are the most important success measures as they

capture the balance of positive and negative impacts of ISs on users (e.g. radiologist, technologists,

physicians and hospitals in general/ valuates the benefits of the Information System for the users and is

operationalized by “Benefits from the end-users’ view (BU) and Net Value from organizational view

(NV). As the “impacts” of IS have involved beyond the immediate user, researchers have suggested

additional IS impact measures, such as work group impacts, inter-organizational and industry impacts,

customer impacts and societal impacts. Former implicates a clearly continuum of ever-increasing

entities, from individuals to national economic accounts, which could be affected by IS activity. So the

choice in this study is done based on the systems being evaluated and their objectives. Rather than

complicate the model with more than 3 success measures for Net Benefits “impact” measures for net

benefits from organizational and societal end of view were kept in a single category called “Net

Benefits” while assuming in this study that organizational and societal benefits are equal. Although for

some studies, such finer granularity may be appropriate such further refinements will be resisted for sake

of parsimony. “Net Benefits” success measures are most important, but they cannot be analyzed and

understood without “system quality”, “information quality” and service quality”.

Benefits from end-users’ view (BU) examines effect of the information system on the users

performance. In this research the benefits from end-users’ view (BU) variable were operationalized by


43

five dimensions: improved individual productivity, decision effectiveness, learning, task performance

and problem identification in five items.

Net value from organizational view (NV) or Net Benefits from organizational (NB) examines the

influence of the information system on overall organizational performance. The instrument was to

operationalize the overall organizational impact variable. This measure was chosen because it measures

the impact of information system in areas that are highly important to all types of organizations. These

areas include reduction of enhancement of internal operations, overall productivity gains, improvement

of organization image, service effectiveness, and increased sales and profit. There are six dimensions

and six items measuring the impact of information systems in these areas.

Net Nenefits (NB) is an idealized comprehensive measure of the sum of all past and expected future

benefits, less all past and expected future cost, attributed to the use of an Information Technology

Application. Any use of resources (including, learning to use, and/or using the system is a cost. To

measure Net Benefits, one has to adopt some stockholder’s point of view about what is valuable and

what not. (Seddon and Kiew, 1997)

4.3.1.2 Variables of Behavior Model of Use

Perceived Ease of Use (PEU) is concerned with the degree to which a person believes, that using a

particular system would be free of efforts (Davis, 1989) and is operationalized in four dimensions:

processing customers request - derived in current study from Dr. Timothy Jung’s (2009): Customer

Usage; receiving customer order, accepting customer orders and customer service request, assessing

low-cost carrier e-airline system success -, user friendliness, information search and ease of use and five

items.

Perceived Usefulness (PU) concerns the belief, that using a system would enhance job performance

(Davis, 1989) and is operationalized in five dimensions: task innovation (effectively), task productivity,

(user) efficiency (data processing correctness, report preparation and distribution timeliness), customer

satisfaction and management control and five items. “Confirmed expectations” are positively related to

“user satisfaction” with IS because it implies realization of the expected benefits of the system. Lastly,


44

while initial acceptance (usage) of IS is an important first step toward realizing success, long-term

viability of an IS its eventual success depends on it continuous use “system continuance” rather than

first-time use. It is than hypothesized that system continuance will be positively associated with user

satisfaction, net benefits and confirmed expectations.

Intention to Use (IU) is concerned with the fact that a person’s intention to perform a behavior is the

best predictor of his behavior (Fashbien) and is operationalized in four dimensions and five items. The

dimensions are: behavior, target, situation in which performing and time of performing. Important for

the Intention to perform a behavior construct is the repeated-observation criterion. According to

Fishbein & Ajzen there should be a high relation between a person’s intention to perform a particular

behavior and his actual performance of that behavior. Within the framework of Fishbien & Ajzen,

intentions are viewed as the immediate antecedents of corresponding overt behaviors. The apparent

simplicity of this notion is somewhat deceptive. However it is often impossible or impractical to

measure a person’s intention immediately prior to his performance of the behavior, the measure of

intention obtain may not be representative of the person’s intention at the time of the behavior

observation. Intervening events that may lead to changes in intentions will therefore also have to be

taken into consideration. For example, a person intends to buy a car three months hence, any change in

his financial position, the price of the car, or the availability of gasoline may influence his intention and

must therefore be taken into account if accurate behavior prediction is to be achieved. Barring such

changes in intentions, an appropriate measure of intention will usually allow accurate prediction of

behavior.

Attitudes and Intentions

As Intentions have often been viewed as the “conative component of attitude” and it has usually been

assumed that this conative component is related to the attitude’s affective component attitude has been

left out of the model as recommended by Fisgbien & Ajzen. This conceptualization has led to the

assumption of a strong relation between attitudes and intentions. Fishbein & Ajzen’s stated that “a

person forms beliefs about an object; he automatically and simultaneously acquires an attitude towards

the object”.


45

4.4 Measures of IS Success Model and proposed Framework

Unfortunately, combining variance and process models is exactly what D&M attempted to do (see

Figure 5, page 30). The result is that many of the boxes and arrows in D&M’s model can, and do, have

both a variance and an event-in-a-process interpretation. The result is a level of muddle thinking that is

likely to be sub-optimal/counter-productive for future IS research.

In an effort to overcome the problem mentioned above, Seddon and Kiew presented a re-specified and

slightly extended version of the Updated D&M IS Success Model 2003. The re-specified model retains

most of the features of the Updated D&M model, but eliminates the confusion caused by the multiple

alternative meanings for the boxes and arrows. They did it by splitting Updated D&M model into two

variance models, namely the main model the IS Success Model and a sub-Model for IS Use and so

eliminating the process model interpretation of the D&M model.

By clarifying the three meanings of IS Use in the D&M model, introducing four new variables

(Expectations, Consequences, Perceived Usefulness, and Net Benefits to Society), and reassembling the

link between the variables, it has been possible to develop a re-specified and slightly extended model of

IS Use & IS Success shown in Figure 6, page 31..

The three meanings of “IS Use” in the D&M Model are:

1. Use as a variable that Proxies for the benefits from Use:

2. Use as the Dependent Variable in a Variance Model of Future IS Use.

3. IS Use as an event in a Process Leading to Individual or organizational Impact.

In other words, the reason that D&M’s model seems to say so much, is that it is actually a combination

of three models:

1. a variance model of IS Success, where the independent variables are probably System

Quality, Information Quality and Service Quality, and the dependent variables are IS Use (as

Meaning 1 proxy for Benefits from Use) and User Satisfaction;

2. a variance model of IS Use as a behavior (Meaning 2 for IS Use);

3. a process model of IS Success, where IS Use is a Meaning 3 event that necessarily precedes

outcomes such as User Satisfaction, Individual Impact, and Organizational Impact.


46

Starting at the top left of Figure 6, page 31 the behavioral variance model asset that, all other things

being equal, higher level of Expectations about the net benefits of future IS Use (henceforth

Expectations) will lead to higher levels of (Meaning 2) IS Use. The other things that would be held

equal would include factors such as subjective norm and perceived behavioral control (Taylor and Todd,

1995). Expectations might be measured by an instrument such as Davis’s (1989) Perceived Usefulness,

or in money terms (if that were possible), or by some other special-purpose instrument.

In case of mandatory use of a system by various members of an organization, it is the Expectation of

senior management (for whom use is not mandatory) that determine IS Use, not the Expectations of their

employees (who may prefer not to use it). Comparing this Expectation in the behavior model to the

D&M model leads to the implications that IS Use under Meaning 2 for IS Use seems to imply that

System Quality, Information Quality, Service Quality and User Satisfaction are part of a causal variance

model that predicts future IS Use. This is not so plausible as its seems, as the problem is that no matter

how good a system has been in the past, past benefits is not a sufficient condition for future benefits.

Rather, potential users will use the system that they hope will offer higher net benefits in the future.

Thus in Figure 6, it is more correct to show Expectations of net future benefits, and not the four

variables in the Updated D&M’s model 2003 as the casual variables that drives IS Use. Moving

clockwise in Figure 6, the consequences of IS Use are represented by the block of text labeled

“Individual, Organizational, and Societal Consequences of IS Use” (henceforth Consequences). The

Consequences are intended to be value-neutral description of outcomes attributed to IS Use, not

measures of IS Success. Consequences attributed to IS Use can include indirect outcomes. Workers and

managers may have different goals. Even if they agree on what the outcomes of system use are (the

Consequences), they may draw different conclusions about the success of the system. The arrow from IS

Use to Consequences in Figure 6 represents the hypothesis that more IS Use implies more

Consequences.12

In advantages of modeling Consequences explicitly in Figure 6 is that while it seems valid to

hypothesize that more Use implies more Consequences, it is not necessarily true that more Use implies

more benefits. For example, for non-volitional users more Use may mean more distress. Thus, Figure 6,

Consequences have been separated from Net Benefits (D&M’s Impacts) to make it clear that IS Success

measurement requires the adoption of someone’s point of view. Without knowledge of that point of

view, it is impossible to hypothesize relationships between IS Use and IS Success. Moving clockwise

12 Use-Consequences Silver et al (1995) Information Technology Interaction Model (Figure 7)


47

again, the larger rectangle at the bottom of Figure 6 contains the re-specified model of IS Success. The

idea behind this positioning of the IS Success model (the large rectangle) relative to the Use and

Consequences variables at the top half of Figure 6 is that based on observations, personal experience,

and reports from others about the Consequences of IS Use, the observer makes judgments about various

aspects of what he/she regards as system success. IS Success is thus conceptualized as a value judgment

made by an individual, from the point of view of some stakeholders. The dotted vertical arrow from

Consequences to Success indicates that there is no clear casual relationship from Consequences to

Success. Perhaps if one could measure the importance of each Consequence, one might be able to

calculate a weighted-sum-of-outcomes measure of IS Success (Ajzen and Fishbien, 1980).

Stepping inside the large rectangle labeled “IS Success Model,” we find a rather complex set of variance

model relationships between seven IS Success measures arranged in three columns. This IS Success

model is the logical equivalent of Figure 4, page 28; all six of the original D&M IS Success Model 1992

success measures appear here. System Quality, Information Quality, User Satisfaction, Individual

Impacts and Organizational Impacts are shown explicitly. Use appears as an example measure at the

bottom of the Other Benefits from Use column (column 3) as “Volitional IS Use.” In additional, two

new variables, Perceived Usefulness and Net Benefits to Society, have been added to the model.

a. Measures of the independent variables (column 1):

The two variables in column 1 of the IS Success model are System Quality, Information Quality. These

variables defined in Figure 4, page 28 are identical to D&M’s variables of the same name. In terms of

relationships between these two variables, it is hypothesized in Figure 5 that System Quality,

Information Quality and Service Quality which is added in this updated model are independent

variables. For example, if the designer gets the specifications wrong, a technically high-quality system

may produce useless information. This seems to be consistent with D&M’s model.

b. General Perceptual Measures of the Net Benefits from IS Use (column 2)

As indicated in the discussion of Figure 4, page 28 the four remaining classes of variables on the

downstream side of D&M’s model are really only classifications of measures of Benefits from IS Use. A

pie can be cut up in many ways, and in Figure 6 the primary cut is based on the distinction between two

general purpose perceptual measures of Net Benefits (that seems likely to be applicable in almost all

situations) and all other measures. The two general perceptual measures (Perceived Usefulness and User

Satisfaction) appear in column 2, and other measures (which include Meaning 1 IS Use) are grouped in

column 3.


48

A weakness with the two general measures is that they are both perceptual. Perceptual measures are

distrusted because people do not necessarily say what they believe, nor do what they say. Worse still,

perceptions can be downright wrong. Notwithstanding these difficulties, Perceived Usefulness and User

Satisfaction are potentially useful for many studies, because they are conceptually meaningful, and

relatively easy to measure. The next few paragraphs define the meanings of these two variables.

User Satisfaction

With respect to User Satisfaction, Naylor et al. (1980) define the general concept “Satisfaction” as “the

result of the individual taking outcomes that have been received and evaluate them on a pleasant-

unpleasant continuum.” Applied to on an IS context, Users Satisfaction is a subjective evaluation of the

various outcomes of IS Use on a pleasant-unpleasant continuum. In this case, the relevant outcomes are

the Consequences depicted in the top-right corner of Figure 6.

The hypothesized relationship between Perceived Usefulness, User Satisfaction, and two Quality

variables in column 1 of Figure 6 is based on the theoretical and empirical work reported by Seddon and

Kiew (1994). They argue that first, the existence of factors related to Systems Quality and Information

Quality in the most User Satisfactions instruments (e.g., Ives et al.’s (1983) User Satisfaction measures,

and Doll and Torkzadeh’s (1988) End User Computing Satisfaction is evidence to support the use of

these two factors (System Quality and Information Quality) as independent variables in a variance

model of User Satisfaction. Second, for very similar reasons, System Quality and Information Quality

should also be influential in determining Perceived Usefulness. Thirdly, User Satisfaction taps a wider

range of needs, cost, and benefits of IT application use than Perceived Usefulness,13 so Perceived

Usefulness may validly be included, along with the other two factors (System Quality and Information

Quality), in a variance model of User Satisfaction.

Relationships between the four variables in columns 1 and 2 may thus be represented by two OLS

regression models, with Perceived Usefulness and User Satisfaction, respectively, as dependent

variables. These relationship are shown in the path diagram at the left of the IS Success model in Figure

6 (arrows indicate hypothesized causality).

13 For instance a very cheap old computer may still be useful for word processing but many people would not be

satisfied with it. So satisfaction must involve the weighing up of a wider range of factors than mere usefulness.


49

Compared to the D&M model these variables are very similar. The two differences are, in the first place

that Perceived Usefulness occupies the slot filled by IS Use in D&M model. Here like Meaning 1 Use in

D&M’s model, it proxies here for Benefits from Use. Secondly, D&M’s arrow pointing up from User

Satisfaction to IS Use is modeled in Figure 6 by the feedback arrow from IS Success to Expectations.

This feedback relationship is discussed shortly.

c. Other measures of the Net Benefits of IS Use (column 3). D&M’s classification of IS Success

measures by whether they measure to individual or an organization is helpful and is retained in column 3

of the IS Success model in Figure 6. In addition, new category, Net Benefit to Society, has also been

added because there are clearly situations, e.g. of impacts of widespread/extensive use of IS, where the

unit of analysis needs to be our whole society, and not just one or more individuals, or one or more

organizations.

Net Benefit is a weighted sum of many positive and negative factors, and that a first –approximation

weighting of each factor can be estimated using an ordinary least squares (OLS) regression model. It

follows that if the variables semantically closest in meaning to Net Benefits are treated as proxies for

Net Benefits (dependent variable), the other IS Success variables can be treated as independent variables

in the OLS regression model.14

In Figure 6, the six left-pointed arrows in the IS Success model have been drawn to indicate that User

Satisfaction and Perceived Usefulness are both likely to be semantically closer to the notion of Net

Benefits than the other measures. Figure 6 shows that User Satisfaction is dependent on six variables

(System Quality, Information Quality, Perceived Usefulness, Net Benefits to Individuals, Net Benefits

to Organizations, and Net Benefits to Society. Perceived Usefulness is hypothesized to depend on the

same six variables, excluding itself. This may not be valid in all situations; it needs to be tested

empirically.

The final relationship to describe in Figure 6 is the feedback path from IS Success to Expectations in the

behavior model. All other things being equal, it is hypothesized that higher Net Benefits from past use

will lead to higher Net Benefits about net future benefits. Since User Satisfaction has been chosen in

Figure 6 as the variable closest in meaning to Net Benefits, the arrow representing this feedback path in

Figure 6 has been drawn from User Satisfaction to Expectations. However, if a more comprehensive or

14 In terms of OLS regression, avoids having to say that variable x causes variable y.


50

reliable measure of Net Benefit existed in the IS Success model, feedback arrow would be from that

more comprehensive measure to Expectations.

That completes the description of the IS Success model in Figure 6, page 31 which will be applied in

this study, after some modifications. The focus of the re-specified model is very much the same as

D&M’s. All seven categories of IS Success measure that D&M identified in their comprehensive and

valuable survey, and of the three meanings for IS Use implicit in their model, are present in the

proposed model in Figure 1. Meaning 1 for IS Use, as a proxy for Benefits from Use, appears at the

bottom of column 3 of the IS Success model as an example ‘Other” measure of net benefit. Meaning 2

for IS Use, the behavior, appears in the box labeled IS Use in the IS Use model. However, because of

the need to maintain clear definition for all variables, Meaning 3 for IS Use (the process-model

meaning) has been omitted deliberately from Figure 6.

In this study also the process side has been taken in consideration in terms of measuring current use.

This as an indication and for confirmation consideration of previous studies that intentions, which are

difficult to approve differ from realization, because as mentioned above people often act different then

they intended.

Although Perceived Usefulness has shown in different studies (Davis (1989, 1993) and Davis et al.

(1989) and others that it is an important predictor of IS future use I consider it important to test and have

this confirmed in the context of this study. If people use IT because they expect it will be useful, it

would seem eminently sensible to measure success by whether they found it is actually useful. It is

therefore argued that the IS Success model is strengthened by including Perceived Usefulness explicitly

as a key IS Success measure in Figure 6.15

In the re-specified model, the feedback loop from Perceptions back to Expectation explicitly recognizes

the importance of learning. The model in Figure 6 asserts that Expectations are continuously being

revised in the light of new experiences with the system. In the clockwise fashion, revised expectations

lead to revised levels of IS Use, which in turn lead to revised perceptions of IS Success, and ultimately,

to revised expectations. This important variable was not incorporated in the studies of Davis et al.’s

1989 TAM, Figure 2, page 12.

15 Perceived Usefulness was the IS Success measure chosen by Franz and Robey (1986)


51

Although D&M in Figure 4 did not set out to build a model that predicts IS Use, it seems likely that the

arrows in Figure 6 from Use down to User Satisfaction, and from User Satisfaction up to Use, were

intended to recognize the possibility of this sort of learning effect.

The research model for post-implementation ISs used to guide the study is shown in Figure 1, page 5 is

a multidimensional model, and the dimensions are interrelated as discussed above. ISs are first

implemented and exhibit various degrees of system, information and service quality. Users and

managers then experience these quality dimensions by using ISs for their works and decisions. Users

and managers can be either satisfied or dissatisfied with the ISs. The intention to use ISs and the three

quality dimensions influence the individual value of using ISs. Collective values of using ISs trigger

influence on organizational performance. Sequencing relative individual works from

business/organizational processes, the individual impacts also collectively affect User Satisfaction.

The entire research suggest that there can be positive benefits from automation, process redesign

activities, and increased timeliness or output quality associated with successful ISs system development,

as these effects have not been studied statistically in the specific context of ERP systems this study tries

to contribute in a while to this gap by a central scope on ERP systems among others ISs in this study.

Based on above discussion the relationships in the proposed model in Figure 1 are:

NB = ƒ1 (SQ, IQ, SRQ, BU, US) - success model

BU = ƒ2 (SQ, IQ, SRQ, IU, PEU, PU) - success and behavior model

US = ƒ3 (SQ, IQ, SRQ, PEU, PU, IU, BU) - success and behavior model

IU = ƒ4 (PEU, PU, US, SQ, IQ, SRQ, BU, US, NB) - relationships in behavior model itself

and with success model

PU = ƒ5 ( PEU ) - behavior model

Where NB is Net Benefits (Organizational and Societal), BU is Benefits of Use (Individual), SQ is

System Quality, IQ is Information Quality, SRQ is Service Quality, IU is Intention to Use, PEU is

Perceived Ease of Use, PU is Perceived Usefulness and US is User Satisfaction.

According to the theoretical basis of the DeLone and McLean, extended with that of Davis and Seddon,

the hypotheses that follow directly from the proposed research model are summarized in Table 5:


52

Code Hypothesis Model

H1a System Quality is positively related to ISs Benefits of Use IS Success model

H1b System Quality is positive related to ISs User Satisfaction IS Success model

H2a Information Quality is positively related to ISs Benefits of Use IS Success model

H2b Information Quality is positively related to ISs User Satisfaction IS Success model

H3a Service Quality is positively related to ISs Benefits of Use IS Success model

H3b Service Quality is positively related to ISs User Satisfaction IS Success model

H4a User Satisfaction ISs is positively related to users’ Intention to Use ISs Link both models

H4b-c User Satisfaction ISs is positively related to Net Benefits hospital IS Success model

H5a Benefits of UseISs is positively related to ISs User Satisfaction IS Success model

H5b-d Benefits of Use ISs is positively related to Net Benefits hospitals IS Success model

H6a Perceived Ease of Use ISs is positively related to Intention to Use ISs Behavior Model

H6b Perceived Ease of Use ISsis positively related to Perceived Usefulness ISs Behavior Model

H7 Perceived Usefulness ISs is positively related Intention to Use ISs Behavior Model

H8 Intention to Use ISs is positively related to Benefits of Use ISs Link both Models

H9 Net Benefits hospitals is positively related to Intention of Use ISs Link both Models

Table 5: Research hypotheses

Source: Prior studies; Shih-Wen Chin, 2006

These hypotheses were tested as set out in Chapter 5, Methodology (see for research model with

hypothesis Appendix 14, page 116.


53

CHAPTER 5 RESEARCH METHODOLOGY

5.1 Procedures and Sample

The objective of the study is to empirically test the inter-relationships between the variables in the

proposed model for the evaluation of the effectiveness of post-implementation ISs and identify the

factors contributing to high-quality ISs. Such “why questions” can be answered by using the case study

method (Yin, 2003). A quantitative approach is used to analyse a series of event exhibiting some

theoretical principles. The purpose is to examine in details the dynamics present in relevant

organisations and conceptually interpret the significance of various factors that influence the

effectiveness of ISs. In this regard conceptualisation of the case study methodology is followed in an

attempt to understand the concepts involved in the field of success model in ISs and define the

substantive domain of these concepts and their relationships. The literature review in Chapter 3 offers

the opportunity to formulate a general research proposition and to identify factors of potential interest in

evaluating the success model of ISs, These factors are further explored by conducting questionnaire

surveys in the three different hospitals with implemented ERP systems among other ISs in the period of

April 26 till May 12, 2009.

Selection of the three hospitals is based on the need to collect detailed data about the Overall ISs

implementation process in each hospital. The selected hospitals vary significantly in terms of

organization size, type of patients, management approach, as well as in the degree of “success” in their

Overall ISs implementation efforts.

In all three cases, an initial interview was conducted, observations and results obtained from initial

interview were reviewed, and a second meeting or e-mail correspondence with the same interviewees

was conducted for necessary elaboration and clarification.

When interviewed by the researcher for collecting archival data IS and financial managers at the three

selected organizations frequently refer to internal documents or seek help of in-house and external

experts in responding to researcher’s inquiries. The researcher also met with those experts who were

able to provide more detailed information. After taken the interview the results were processed and the

questionnaire was designed and pre-tested according to the framework mentioned in previous sections,


54

and then sent to the three hospitals registered at the Ministry of Health with more than 100 employees

and an identified automation organization.

The following sections present the data obtained from these questionnaires with regard to the driving

forces and principles for Overall ISs implemented in the three hospitals, as well as significant issues

related to the implementation process.

The study further aims to propose a success model for post-implementation ISs and empirically test the

inter-relationships between those dependent variables in the proposed model. In the survey,

questionnaires were sent to 148 users and 7 IS support staff of the three hospitals in April 2009. A total

of 99 questionnaires were returned to researcher. The response rate was 64%. While this is higher than

expected as mentioned in 1.3.3.2, the participation of the different hospitals is much skew between the

private hospital and the two public hospitals. This was caused by difference in coordination approach of

the surveys among the private and the public hospitals. Overall there were no questionnaires deleted

because of incomplete answers, so all 99 questionnaires were left for analysis. The effective response

rate stays 64%. Table 3 presents essential information about ISs implementation within the three

hospitals as provided by the coordinators at the hospitals and confirmed by the users en support staff.

Among the returned questionnaires, 69 (70% response rate) were completed by participants of DKH, 14

(14% response rate) by participants of LH and 12 (12% response rate) by participants of SZN. 4

questionnaires had no hospital name on them but could be traced by a unique reference number given

per hospital after collection of returned questionnaires for completeness check of processing. Because of

time constraints the four questionnaires were left out from specification under the respective hospitals.

Demographics

From the 99 responses 90% represents finance and administrative professions while only 10% represents

the core professions of a hospital, namely: technicians 2%, nurses 3% and medical assistants 5%.

Respondents worked primarily (95 %) in executive professions while 5% had a senior management

profession. 8% had an academic degree, while 3% held an under-graduate degree and 45% had a high

school diploma. At the gender side there were 87% women and 13% men. 59% of the respondents use

computers 1 to 5 years and only 4% has more than 10 years experience with computers. 60% of the

respondents have experience with an IS before using current ISs of which 25% has used medical

information systems before.


55

Furthermore the respondents of DKH complaint about congestion of the in ERP system due to less

licensing, while some computer are slow, more automation overall and at department level is

appreciated as well as more ICT and IS education. Finally it is interesting to report that 59% of

respondents are 31 to 50 years old while 16% are 51 to 60 years, which seems reasonable for a good IS

environment (see Appendix 6 for hospital sample, page 95).

Instrument Validity

With regard to data analysis, composite reliability coefficient (Cronbach’s alpha) has been computed in

order to assess the internal consistency of each scale. Table 6 presents the results associated with the

assessment of the internal consistency of each scale. The composite reliability coefficients of all the

scales range from 0.698 to 0.898. The scale for Use was adjusted after the pre-test from two to four

items which just like previous research have shown not to be a strong predictor of IS Success and so in

the D&M model the weakest predictor of Net benefits. This modification failed to increase reliability of

the scale so the instrument with the two-item scale was used to confirm the internal reliability of the

construct.

Nr. Scale Dimensions N of items Cronbach

α

1 System Quality (SQ) 7 7 0.766

2 Information Quality (IQ) 6 6 0.865

3 Service Quality (SRQ) 5 5 0.896

4 Perceived Ease of Use (PEU) 4 4 0.886

5 Perceived usefulness (PU) 5 5 0.898

6 Intention to Use (IU) 4 5 0.881

7 System Usage (SU) 3 2 0.698

8 User Satisfaction (US) 8 9 0.870

9 Benefit of Use (BU) 5 5 0.736

10 Net Benefits (NB) 6 6 0.874

Table 6: Results of factor analysis of constructs scale

Source: Own research


56

A survey methodology was used for data collection. The study aims to examine the effectiveness of

ISs in a hospital environment testing an extended version of the DeLone and McLean IS Success

Model 2003. Data was used for data analysis. Structural Equation Modeling (SEM) is the primary

analysis used to examine the proposed hypotheses developed in fulfilling the study objectives. The

SEM results have been interpreted relative to the implications in the health sector.

5.2 Case Evidence

5.2.1 Introduction

The 2006-2011 Policy Paper of the Ministry of Health identified two core problems in the health care

system: financing and lack of trained personnel. The focus of the Ministry’s policies for 2006-2011 is to

stop the decline of the health care sector. Measures planned to regulate and organize the system include

institutionalization of a National Health Council; strengthening of management, upgrading health

legislation, continued privatization of government hospitals, the Regional Health Service, and other

institutions, and restoration of health care facilities in the interior. The government will implement “a

compulsory national health insurance system for the total population, including mechanisms to regulate

salaries of service providers, to control prices of drugs and other inputs, and to control the cost of

intramural care.” Financial policies will focus on ending open-end financing of hospitals, budgeting,

including the limit of government expenditure to 6% to 8% of GNP, are addressed in the Policy Paper

(see Appendix 11 for indication of distribution of spending by payer). Intramural care should be limited

to less than 52% of the health care budget. The role of the hospitals should be emphasized on the

support of primary health care. Furthermore supply of clinical care will be organized and procedures

will be implemented regarding extension of high level clinical care. This will be done by support of:

• Training of radiologists and assistant radiologist;

• Finalization of constructing Mother and Child center of LH;

• Improvement of automation within the hospitals; start evaluation of automation;

• Support systems sector wide:

o Improvement of Purchasing, Communication, and Information Systems by

implementation of a National Health Information System (NHIS);

o Implementation of professional procurement;

o Professionalism in Public Relations (PR) and Communication.

• Upgrading hospitals;

• Upgrading Streekziekenhuis Albina.


57

There are four general hospitals in Paramaribo, one in the district of Nickerie and one in Albina of

which less information is available. There is one psychiatric hospital. There are 0.27 beds per 10,000

population: 387 in Academic Hospital, 368 in s’Lands Hospital, 217 in Diakonessen Hospital, 287 in St.

Vincentius Hospital (a Roman Catholic hospital), and 97 in Streekziekenhuis Nickerie (Nickerie District

Hospital). According to last census of 2004 the population of Suriname was 492,829 inhabitants with

and annual growth of 1.40% (see Appendix 12). The combined occupancy rate of the four major

hospitals in Paramaribo is 70%, a rate that increased slightly from 62% in the last decade. The average

length of hospitalization is 11 days. The Academic Hospital is the largest hospital in Suriname and was

the first mover with a department for emergency for medicine, and has an average of 33,959 admissions

yearly. Physicians are graduates from both the Medical School of the University of Suriname and

Medical Schools from abroad. The Central School of Nursing and the intramural training programs of

the Academic Hospital and the St. Vincentius Hospital are training nurses and nursing auxiliaries, but

the programs cannot keep up with the demand. The Nursing School has a new program for bachelor’s

degree Nursing.

Unfortunately the Academic Hospital and St. Vincentius Hospital were not able to participate in this

study but I hope they can benefit from the outcome.

The National Health Information System (NHIS) unit and the Bureau of Public Health at the Ministry of

Health are major end points for national health data analysis and dissemination. Most of the health data

and information reaches the NHIS unit through the reports of the Bureau of Public Health (which

remains health data and is the analytical unit of the Ministry of Health), the medical registration of

hospitals, the Regional Health Services and the Medical Mission, and professional health associations.

At the moment of the study the latest auditors report regards the Draft Management Letter of the interim

audit of 2008 dated January 30, 2009. The auditor’s opinion for fiscal year 2007 was a disclaimer of

opinion. In the Draft Management Letter the auditor reports: non existence of Administrative Policy

Manual and performing of the quarterly budget variance analyses for the first quarter 2008.

5.2.2 Case Background‘s Lands Hospital

The ‘s Lands Hospital is the oldest hospital in Suriname and has shown its right to exist. There is a

discussion going on regarding the formal status of ‘s Lands Hospital (LH). In this discussion it is

doubted if the foundation “Stichting ‘s Lands Hospital” which should have been established since


58

August 22, 1983 exist or not, and if the hospital is a department, unit or section of the Ministry of

Health. The publication of the registration in the Register of Foundations and the by laws of the hospital

are missing. The hospital has a long history. The activities started in 1688, when city hall and church

were housed in one building and has actually 368 licensed beds. In 1693-1695 plans were made to build

a hospital which then became ‘s Lands Gasthuis Paramaribo. The large number of stationed military

personnel made it necessary to build a ‘military hospital’. A start was made in 1758 and the hospital was

completed in 1760. The civilian hospital burnt down to ground in 1821. Thus the military hospital

became also a civilian hospital. The military hospital (LH) was expanded and renovated in 1849.

The s’Lands hospital has several special functions. Almost half of all babies that are born yearly in

Suriname (see Appendix 12) are delivered in this hospital. The Mother and Child Health Department

offers prenatal services and provides women with Pap tests. The hospital also performs rental dialysis.

Nearly 60% of patients admitted yearly are covered by the Ministry of Social Affairs and 24% by the

State, while this is 559 admissions from the interior yearly. On average 1,385 babies are delivered

yearly in this hospital. The hospital has a policy of linking hospital with primary level services and

maintains a general polyclinic that is open to the public until 23:00 hours daily and also during the

weekend. It has a department for community and home based care to limit the duration of stay in the

hospital.

There is a contradiction in data derived from different sources. Much is forgotten and a lot has to be

kept secret with the spirit of love. The course of the story would be better understandable if it coupled

with the history of Suriname in which the colonial relation with the Netherlands is at the corner and

developments in health care in Suriname a central focus point. Also shortage in funds and political

constrains have pressed a stamp at the history of LH both in the past and present.

At the moment of the study there was no business plan or policy paper of the hospital available while

the latest auditors report regards the fiscal year ending per December 31, 2000. The auditors’ opinion is

a disclaimer of opinion. In his report of February 23, 2005 the auditor reports three causes of the

backlog. Firstly: wrong processing of contributions of patients of the Ministry of Social Affairs by IS.

Secondly: the difficulties in presenting reconciliation between general ledger and control account

outstanding accounts receivables and liabilities and thirdly: that the hospital is qualitatively and

quantitatively under staffed.


59

5.2.3 Case Background Diakonessenhuis

The Diakonessenhuis (Deaconesses’ hospital or Nursing home) was established in 1962 by a joint

cooperation of the Protestant Churches in Suriname and is one of the two private general hospitals in

Suriname in the capital, Paramaribo. It starts in the Netherlands in 1955 when a small group of men

chaired by Dr. M.A. Melle who initiated to establish a protestant’s health care organization even

supported by regional health care as a core. To achieve this goal they also established an executive

committee in Paramaribo which afterwards rolled into a foundation in 1958, “Surinaamse Stichting

Diakonessenarbeid” (Surinamese Foundation for Deaconesses work). The objectives of the foundations

are: public service, nursing, and care for pregnant women, senior citizens, the ill and disabled, perform

social and social-medical work, establish and maintain one or more policlinics and hospitals etc.. The

hospital is annex a convent (nurses’ home) and is situated at the Ziniastraat. Starting with 30 licensed

beds in 1961 the hospital has grown to 217 licensed beds today.

Management of the hospital is formed by representatives of the Moravian Church, the Lutherian

Congregation and the Reformed Congregation. Executive management granted to three directors,

namely a Medical-Director, a Financial-Director and a Nursing Director. Furthermore there is a Medical

Management which is chaired by a Medical Doctor of the hospital. Spiritual care for patients, nursing

personnel and other personnel, which is an integral part of the services, is the responsibility of the house

preacher and his staff. The hospital is a general hospital and is open for everybody no matter what

ethnicity or religion, in case of availability. Diakonessenhuis has a good reputation and image in health

care in Suriname, and got the change to achieve its objectives. The hospital succeeds in balancing

medical and missionary goals while there was also as much as possible attention for the deaconesses. In

this regard it is remarkable to mention the cooperation between the Medical Mission of the Moravian

Church (MEDIZEBS) which covers health care for the greater part of the population in the interior, de

Medical Mission Suriname (MZS) which covers health care for the upland Amerindians and Maroons

(descendants of runaway slaves) and the “Pater Ahlbrinck Stichting Stichting” (P.A.S.) (Father

Ahlbrinck Foundation) which covers health care for the lowland Amerindians. The management of the

Diakonessenhuis is responsible for managing the federation of those three foundations which are known

as Medical Mission. This responsibility will be taken over by the Medical Mission Suriname, a national

non-governmental organization established in 2001 with history working with the Amerindians and

Maroons in the interior which is contracted by the Ministry of Health to provide health care services in

this area for 60,000 persons. This organization is meant to be the successor of the three organizations

mentioned before. The Diakonessenhuis personnel consist of capable medical doctors, nurses and other

staff and it plays an important role in the health care in Suriname. It cooperates extensively with the


60

Ministry of Health and participates in the hospital board and the preventive care of the Administration

for Public Health (BOG) (Bureau Openbare Gezonheidszorg) Malaria, National Immunization Program.

There is also a partnership with the St.Vincentius Ziekenhuis in Waspar and other activities. Also the

work of D.O.V.E. (DOOR ONDERLINGE VERBONDENHEID EEN) which covers all kinds of

services to the Diakonessenhuis is worth mentioning. D.O.V.E. deals with services at social and

community level for patients in the hospital, and consists among others of offering literature, toilet

articles etc., placement of plants, installation of nursing call or pager system, for the children’s

department etc. This group is very important for Diakonessenhuis and is of understanding to all

personnel.

In the past 30 years the Diakonessehuis has grown consistently, and has acquired a significant place in

Suriname’s health care system. Despite this the Diakonessenhuis nowadays suffers a financial problem.

The last five years the Diakonessenhuis was financed with gifts from friends and this will be continued

in the in coming years. However the solution to this problem should be found in a sound cost-benefit

structure of operational cost and revenues based on activity based costing, or health fund tariffs filled up

with government subsidy. The financial problem is discussed extensively the last few years with both

the Minister of Health and the Hospital Board, and there is nothing to be desired that the

Diakonessenhuis will solve this problem.

At the moment of the study there was no policy paper of the hospital available nor an auditors’ opinion

on the financial statements as per December 31, 2005 compiled by the auditor.

5.2.4 Case Background Streekziekenhuis Nickerie

The Streekziekenhuis Nickerie (District Hospital) is a basic hospital and was established in 1991. The

requirements for a basic hospital are minimal availability of at least one of next physicians: an internist,

a pediatrician, a gynecologist, a general surgeon, an anesthetist and an urologist. Nickerie District

Hospital has an operating room, an obstetrics department, an X-ray facility, and a medical laboratory.

The hospital is one of the tree public hospitals and has two emergency physicians and two room

physicians for support and control of in-patients. Furthermore the hospital gets support from Paramaribo

of the following physicians: a radiologist, an internist, an ophthalmologist (eye doctor), a dermatologist,

and a neurosurgeon. The physicians keep policlinics during weekends. The hospital was renovated in

the course of during 1993-1996 with a loan of 8 million US dollars from the Inter- American

Development Bank. The major problem faced by this hospital is the lack of medical specialists. The


61

hospital has 97 licensed beds of which 68 for SZF and patients with a card of the Ministry of Social

Affairs.

Although the hospital has to be operated on a break-even base it doesn’t get subsidy from the

government. The latest auditors’ report covers a backlog from December 1999 till December 31, 2004

and is dated January 31, 2007. The auditors’ opinion is a disclaimer of opinion. In his report the auditor

reports among other findings concerning the organization that the Accounting Information System did

not have the possibility to process adjustments and that nether accounting nor automation were

functioning sufficiently.

5.3 Respondent Characteristics

The sample represents three hospitals with an ERP system among other ISs. Participants in the study are

148 system users and 7 IS support personnel, experienced, well educated and on average employed with

their organizations for 12.5 years. To remedy the situation of relatively limited responses, the study has

put together the data of all respondents for analysis. It should also be noted that the three hospitals differ

from each other in timing of their IS implementation, composition and structure. Experientially based

differences in organizational positions, or the user’s role in the development of the application, may

cause differing frames of reference.

5.4 Research Instrument and Variable Measurement

5.4.1 Research Instrument

The research instrument contains a series of statements to which participants note their level of

agreement and disagreement on a 7-point Likert scale ranging from strongly agree (score = 7) to

strongly disagree (score = 1). Please consult Appendix 2 for a complete listing of statement items. Each

of the statements is used as an indicator variable that measures the latent dimension and constructs of

interest in this study.

“System Quality” (SQ) in the ISs environment measures the desired features of the ISs . Timelines,

system accuracy and flexibility are examples of qualities that are valued by users of ISs. “Information

Quality” (IQ) captures the ISs content issue. ISs document content should be personalized and complete.

“Service Quality” (SQ), the total overall support delivered by the service provider, applies regardless of

whether this support is delivered by the IS support department or not.


62

I measured participants’ perceptions of System Ease of Use (PEU) using several statements such as: “It

will be easy to get ISs to do what I want to do”, ISs will be flexible to interact with: “My interaction

with ISs will be clear an understandable”, and ISs will be easy to use. Perceptions of System Usefulness

(PU), on the other hand, are measured with statements like “Using ISs will increase my productivity”

and using ISs will improve my job performance”. I measure participants’ behavioural Intention to Use

(IU) the system as intended by developers with several statements. One statement was “I intend to use

the ISs in electronic mode, rarely printing out copies of work papers as I proceed through my tasks”.

This statement mirrors the hospital’s emphasis on using ISs system in electronic mode (i.e., not printing

out the electronic work papers and thus bypassing the system). Other statements measuring IU relate

mostly to the particular professional task performed by participants by participants, e.g. “I intend to use

ISs in planning and tailoring related programs”.

I measured System Usage (SU) using statement such as: “I use the ISs extensively (100% of my job

time)” and “I use the IS seven days a week (1=1day and 7=7 days).

I measured Benefit of Use from end-users’ view (BU) using statements common to three sample groups,

e.g., “ISs allow me to communicate with other users in an effective way”, ISs help me effectively to

manage and store knowledge that I need” and “ ISs help me to identify problems in an early stage”. The

statements items measuring task Net Value from organizational view” (NV) or “Net Benefits (NB) are

specific to the work roles performed by the participants. For example, I prepare assessed agreement with

statements such as “ISs enhance competitiveness or create strategic advantage”. “ISs establish and

maintain a good image”, and “ISs have dramatically increased our service capacity”.

5.4.2 Variable measures

Items used to operalitionalize the constructs were adopted from relevant previous researches. The

adopted items were validated, and the changes in wording were made to tailor the instrument for this

research. To determine if the 7 System Quality items, 6 Information Quality items, 5 Service Quality

items, 4 Perceived Ease of Use items, 5 Perceived Usefulness items, 5 Intention to Use items, 2 System

Usage items, 9 User Satisfaction items, 5 Benefits of Use, and 6 Net Benefits items could be reduced to

a smaller group of meaningful factors, a principal component analysis was conducted based on the

responses obtained from all participants surveyed in the study. With no item dropped, ten components

with eigenvalues greater than one emerged; the best results were obtained with varimax rotation. Results

of confirmatory factor analysis indicated that a priory assumption was substantiated with a ten-factor


63

solution, and the loading of the ten components are presented in Table 6. All the Cronbach’s alpha

coefficients of the ten constructs are 0.698 and greater. The reliability values based on Cronbach’s alpha

are all greater than the recommended minimum of 0.5 (Hair et al. 2006). Based on the extensive

examination of the psychometric properties of the scales, the study concludes that each variable

represents a reliable and valid construct.

5.5 Data Analysis and Discussion

Path analysis has been adopted to test the proposed model as well as to check the relationship among the

constructs. The hypothesized paths in the proposed model in Figure 1, and described in paragraph 4.4

were tested using SPSS 17.0., with 95% likelihood.

5.5.1 Model testing

The instrument validation in terms of content validity was continued by analyzing Pearson correlation.

Table 7, Figure 8 and Appendix 8 show the descriptive statistics and correlation matrix and path

analysis ranging from 0.204 to 0.660, which produced a significance level between 0.000 and 0.024.

Thus strengthen content validity in the instrument validation.

The structural variance models were evaluated on the basis of five goodness-of-fit measures. A widely

used fit measure is the statistical significance of the chi-square statistic which indicates whether the

model has a poor fit with the data. The drawback of the chi-square test is that significance is sensitive to

sample size and number of parameters in the model (Bentler and Bonett, 1980), and as a result, the test

may provide an inappropriate indication of poor fit. In the current analysis, the proposed IS success

structural model factors are treated as dependent variables. The descriptive statistics and correlation

matrix related to the formulated hypotheses are stated in Table 7, Figure 8 and in Appendix 8 all

relationships in the proposed structural model are stated.


64

Construct Mean Std.

Dev.

NB BU US IU PU SQ IQ SRQ PEU

1. NB 28.84 5.188 1

2. BU 23.25 4.157 0.560* 1

3. US 41.35 7.440 0.552* 0.645* 1

4. IU 23.31 5.105 0.387* 0.381* 0.204** 1

5. PU 26.75 4.382 0.404* 1

6. SQ 30.49 6.008 0.450* 0.585* 1

7. IQ 38.034 6.167 0.476* 0.572* 0.631*** 1

8. SRQ 24.84 5.205 0.220** 0.283* 0.369*** 0.332*** 1

9. PEU 19.02 3.964 0.415* 0.660* 1

Correlations are significant at *p-value < 0.01; **p-value < 0.05 (2-tailed), ***not hypothesized

Table 7: Descriptive statistics and Correlation Matrix


Appendix 8 presents the parameters and the p-values of the assumed path for the ISs success measures.

As indicated, all path coefficients depicted in Figure 8 were as hypothesized. The paths from Perceived

Ease of Use (PEU) to Perceived Usefulness (PU) and Intention to Use (IU) were significant, as was path

Perceived Usefulness (PU) to Intention to Use (IU). Perceived Ease of Use shows to be a strong

predictor of Perceived Usefulness.

The path from Intention to Use (IU) to Benefits of Use (BU) was also significant. These results were

slightly different from Shih-Wen Chien (2004). The study further indicates that Perceived Usefulness

and Perceived Ease of Use both had a significant total effect on Intention to Use.

Further, Figure 8 shows that the relationship between Service Quality and Benefit of Use, the

relationship between Service Quality and User Satisfaction, the relationship between Benefit of Use and

User Satisfaction, the relationship between User Satisfaction and Net Benefits, the relationship between

Benefits of Use and Net Benefits, and Net Benefits and Intention to Use are all significant at a 0.01

statistical significant level. The latter confirms a positive relationship between the individual impact and

process improvement or organizational impact.


65

Figure 8: Path Analysis Results


5.5.2 Quality dimensions in Proposed IS Success Model

Appendix 10 presents both the relationships among the success measures variables. It is very interesting

to notice that, all three quality dimensions, System Quality, Information Quality, and Service Quality

play important roles in influencing IS’s Benefits of Use and IS User Satisfaction. It reveals the fact that

most ISs, at post-implementation stage, are complicated systems required to provide excellent System

Quality; users are therefore in great need of the service provided by the IS support department.

.387


66

5.5.3 Net Benefits of Use in Proposed IS Success Model

The results support the hypothesized impact of User Satisfaction on Benefit of Use via feedback of

Intention to Use, as assessed by System Dependence and suggested by the DeLone and McLean (1992)

and Seddon (1997) models. The relationship between Benefit of Use and User Satisfaction in this model

is a significant one taken in consideration that User Satisfaction can support user’s voluntary use of ISs.

A close look at the causal structure of Seddon’s model supports specification of a relationship between

Perceived Usefulness and User Satisfaction. Perceived usefulness influence users Satisfaction the in

return influence expectations about future benefits, thereby affecting ISs Use. ISs Use and its associated

consequences provide feedback to Perceived Usefulness, assessed as Net Benefits accrue. However, the

study has empirical identified the factors that influence Perceived Usefulness on Intention to Use. What

prompts users is to truly appreciate the benefits of ISs come from the three quality dimensions and the

behavior aspect from System usage.

With the addition of a non-directional path between Perceived Usefulness and Benefits of Use (System

Usage) in the proposed model structural model, the effect size of User Satisfaction on System Usage

was reduced, while a strong relationship between dependence on IS and Perceived Usefulness was

observed. ISs provide necessary information for users’ job performance, but using other means to obtain

this information is laden with procedural complexities. If information is perceived as useful because it is

job relevant and not available through other means, then ISs will be used regardless of persons overall

satisfaction with ISs.

This study implicates that Benefit of Use is directly influenced by, System Quality, Information Quality,

and Service Quality. The results suggest that the effect of System Quality and Information Quality on

Perceived Usefulness via User Satisfaction and Benefits of Use is substantially greater than the effect of

Perceived Ease of Use on Perceived Usefulness. In the D&M model Perceived Usefulness is directly

influenced by beliefs concerning Perceived Ease of Use and Information Quality through their effects on

Uses Satisfaction and System Usage. Here too, the effects of Information Quality on system

dependence, the measure Benefit of Use is taken into account here, and User Satisfaction, are greater

than the effects of Perceived Ease of Use on these variables. Perceived Usefulness, as an attitudinal

measure of Benefits of Use, is influenced by User Satisfaction and System Usage. In comparison with

the study of Shih-Wen Chien the results indicate that in contexts where effective task execution

substantially depends on information delivered by the system, beliefs about Information Quality are


67

more dominant in shaping IS Success than beliefs about Perceived Ease of Use, this as an effect of

mandatory use.

5.5.4 User Satisfaction in Proposed IS Success Model

Although User Satisfaction is extensively adopted by researchers and practitioners to evaluate IS

Success, important issues pertaining to its meaning and measurement across population subgroups have

not been adequately examined. To achieve the greatest usefulness in decision-making, instruments like

end-user computing satisfaction (EUCS), which is designed to evaluate system success, should be

robust. That is, they should be able to facilitate comparisons by providing equivalent measurements

across diverse samples that represent the variety of conditions or population subgroups present in

organizations.

Both the D&M (1992) and the Seddon (1997) models specify User Satisfaction ad influenced by beliefs

about Benefits of Use, System Quality and Service Quality. These relationships are consistent with

TAM, where attitudes towards System Usage are influenced by beliefs about the system. Given the

position of Benefits of Use in the Individual Impact category of the D&M Model, User Satisfaction is

caused by Benefits of Use and Net Benefits. On the other hand, in the Seddon Model, Perceived

Usefulness impacts User Satisfaction. To the instant that Perceived Usefulness, as an attitudinal measure

of realized net benefits, is similar to the expectations of future to be realized by using ISs, Seddons’

Model confirms the specifications TAM that finds beliefs concerning Perceived Usefulness capable of

strengthening User Satisfaction and thereby influences System Usage. However, results of this empirical

study suggest that Perceive Usefulness and Perceived Ease of Use can exert indirect influence on

Benefits of Use and User Satisfaction through Intention to Use.

5.5.5 Beliefs in Proposed IS Success Model

D&M 1992 and Seddon (1997) models focus both on beliefs concerning System Quality that embedded

Ease of Use as considered in TAM. Moreover, the two models take Information Quality into

consideration, as a belief that not is explicitly considered in TAM. Perceived Usefulness is an attitudinal

measure of Net Benefits. Seddon’s model elaborates the causal structure of TAM by separating beliefs

concerning expectations of Net Benefits associated with System Usage and general perceptual measures

of Net Benefits associated System Usage. Perceived Usefulness is an attitudinal measure of realized net


68

benefits and is influenced by beliefs concerning Information Quality and System Quality. In this study

Perceived Ease of Use and System Quality are separated since Perceived Ease Use tends to concern

more with individual perceptions while System Quality is usually an inherent feature of the system

itself. By influencing attitudes concerning realized net benefits, both System and Information Quality

influence expectations of net benefits in a different way than Intention to Use that equates to belief

about Perceived Usefulness, Perceived Ease of Use as embedded in TAM. The D&M Model does not

explicitly take either Perceived Usefulness or Perceived Ease of Use in consideration.

5.5.6 Net Benefits in Proposed IS Success Model

Performance management or integral management, for example the balanced scorecard allows

organizations to see the positive and negative impact of ICT and so ISs activities as factors crucial for

the organization as a whole. The value of performance management rises if it is used to coordinate a

wide range of ISs management processes, such as individual and team goal-setting, performance

appraisal and rewards for IS support personnel, resources allocation, and feedback-based learning.

Management from both IS member and projects are likely to benefit from a systematic framework based

on goals and measures that are agreed upon in advance.

5.5.7 Summary

Using the path methodology for confirmation in this study it appears that Benefits of Use (r = 0.560, p =

0.000) and User Satisfaction (r=0.552, p=0.000) influence Net Benefits significantly and directly while

both Benefits of Use (r = 0.916) and Information Quality (r = 0.583 had a strong total effect on Net

Benefit (see Appendix 13 for direct, indirect and total effects)]

Finally we can conclude that System Quality and Information Quality seem to be strong predictors of

User Satisfaction, while Benefits of Use is a strong predictor of User Satisfaction and on their turn both

Benefits of Use and User Satisfaction are strong predictors of Net Benefits and thus of organizational

performance. The above mentioned relationships confirms the intercalations between the constructs and

indicates in what instant Net Benefit is influenced by the different measures, while Net Benefits in turn

is a good predictor of Intention to Use and keeps the cycle going on.


69

CHAPTER 6 CONCLUSIONS AND RECOMMENDATIONS

The primary purpose of this study was is to understand the relationship between the effectiveness or

success of Information Systems and organizational performance in the hospital environment in

Surname. Secondly, to examine if this relationship leads to productivity improvement and thirdly

addressing the three main research questions and finally addressing the central question of the thesis. To

address the central question in this thesis in a systematic way I found an extended version of the

Updated DeLone and McLean IS Success Model 2003 incorporating the behavior aspect of use from

Davis’s Technology Acceptance Model (TAM) elaborated by Seddon in his IS Success Model 1997 the

best tool to help me addressing it. As a result of this fifteen hypotheses were formulated and the test

results presented in this thesis confirm the relationship and in what instant the factors involved influence

the effectiveness of IS on organizational performance in hospitals in Suriname.

6.1 Conclusions

As discussed in Chapter 5 the instant to what the different factors influence the effectiveness of

Information Systems in hospitals Suriname is positive and multidimensional. The fundamental factors

influencing effectiveness of Information Systems on organizational performance in hospitals in

Suriname are six interesting factors: System Quality, Information Quality, Service Quality, User

Satisfaction, Benefits of Use and Net Benefits for the IS Success Model, while for the behavior aspect

of Use which influence Benefits of Use those factors are: Perceived Ease of Use, Perceived Usefulness

and Intention to Use.

The participants in this study were 99 users and IS support staff of three general hospitals in Suriname

with at least 100 personnel and an identified automation organization. Regression analysis, OLS and

SEM was conducted to test the proposed hypothesis.

After testing the hypothesis the strongest or main factors influencing the effectiveness on organization

are as summaries in Chapter 5: User Satisfaction and the Benefits of Use are the strongest and main

factors predicting Net Benefits directly and Information Quality indirectly and so those three are the

main factors influencing the effectiveness or success of Information Systems on organizational

performance in hospitals in Suriname.

Notwithstanding the fact that Benefits of Use is offering a high contribution to User Satisfaction the

contribution from User Satisfaction to Intention to Use is low. This is primarily caused by Service


70

Quality which is offering a low contribution to User Satisfaction so user cannot benefit from his/her

Intention to Use due to the weakness in Service Quality.

On the positive side, respondents see the benefits of computers for their own education as well as source

of information. Access to up-to-date information provided in the context of the display of information

may be the “carrot” that designers can offer users who otherwise may be reluctant to use computer

systems.

On the negative side respondents results show that the use of information system by the core professions

in the three hospitals is underdeveloped and there is a significant difference in approach and culture

between private and public hospitals.

6.2 Recommendations

To improve the ability to evaluate Information Systems’ effectiveness the next recommendations are

offered:

1. define and/or derive appropriate system objectives and measures of accomplishment:

Since ultimate effects of an Information System on organizational performance may not be

direct and immediate, and since the value of information is only realized in its use, a focus on

definition of level 2 objectives of the hierarchy of System Objectives and measures is

recommended, namely the behavior Process of IS Use, Benefits of Use and Users performance;

that’s why System Usage was stressed double in the study. For example, organizational

objectives and performance measures might be expressed in sales revenues, customer

satisfaction, morale, and return on investment and profit contributions. System objectives in

terms of the behavior Process of IS Use and User organisational performance reflect IS effect on

these organizational processes, including changes in decision makers, changes in the decision

making process, and changes in user organizational performance. Surrogate measures of the

utility IS in supporting organizational processes are recommended for assessing IS effectiveness,

including User Satisfaction, Information Satisfaction and System Utilization.


71

2. develop a general view for evaluation IS:

a. develop a goal centred view;

Determine task objectives of the system, or the organizational units utilizing the

system, and then develop criterion measures to assess how well the objectives are being

achieved. Effectiveness is determined by comparing performance to objectives. An example

of the goal-centred view of systems effectiveness would be comparing actual cost and

benefits to budgeted costs and benefits;

b. develop a system-resource view;

System effectiveness is determined by attaining a normative state, e.g., standards for “good”

practices. Effectiveness is conceptualised in terms of resource viability rather than in terms

of specific task objectives. For example, system effectiveness in terms of human resources

might be indicated by the nature communication and conflict between IS and user personal,

user participation in system development, or user job satisfaction. In terms of technological

resources, system effectiveness might be indicated by the quality of the system or service

level.

In assessing system effectiveness, the evaluation approach would partially depend on which of

these two views is considered. In practice, the two views should converge. In order to explain the

success or the lack of success resources need to be investigated.

3. enlarge the range of performance being evaluated:

The intangibles qualitative effects of IS on organisational processes, Level 2 objectives of

conceptual hierarchy and performance, Level 3 of conceptual hierarchy are often more

significant for assessing system effectiveness. Since objectives and measures are typically

efficiency-oriented and easily quantified, a need exist to enlarge the range of performance being

evaluated to include Level 2 effectiveness oriented objectives. Build appropriate capacity for

useful effectiveness data analysis, data management and presentation at all levels and in different

applications and organizational processes

4. recognize the dynamic of the IS implementation process:

The dynamic nature of the IS implementation process suggests that evolutionary changes in

objectives will occur because of learning by users and IS support personnel and changes in the

environment. Thus implementation of an IS is viewed as a planned organization change which

will modify users’ work system to improve its functioning. This view explicitly emphasizes the

importance of considering effects of technical change on the user organizational processes and


72

the dynamic nature of IS implementation. IS effectiveness is explicitly conceptualised in terms

of achievement of objectives and the “institutionalisation of change”, e.g., ongoing user training,

use of IS services, and IS support.

5. incorporate multiple dimensions and multiple viewpoints into evaluations of IS:

Account for differing evaluator viewpoints. The establishment of realistic mutually agreed upon

definitions of appropriate objectives and measures of accomplishment at the outset of the system

development should be prescribed. Differing viewpoints need to be considered not only in

initially establishing these objectives and measures, but also in assessing system effectiveness.

6. improve user support:

Establish Service Quality evaluation system. IS support personnel should be trained regularly in

all aspect, e.g. professional skills in IT and support, social skills and service in relation with

integral management to assure higher data quality, timeliness and promotes communications an

resolution problems. Continually improve user education; improve user understanding and

awareness through education, improved data accuracy, improved inventory control and

production scheduling, improved coordination between functional groups, and improved

customer satisfaction.

7. link fragmented Information Systems hospitals with interfaces to the ERP system:

The defragmented Information System within the hospitals should be linked with the ERP

system. After achieving this stage link integrated Information System of individual hospitals with

Information System of other health care institutions, organizations, programs, units and the

Ministry of Health without disrupting the decentralized databases, by achieving shared forms and

operational definitions, developed through central and decentralized data warehouses (including

metadata), Thus frequent communication between entities mentioned above, and easy access to

overall health care data and related information throughout the health care sector will be

established.

8. regular evaluations:

Establish regular evaluation of Information Systems within the hospitals and within the

health care industry to improve and maintain the quality of both Information System and

Evaluation System (IS Success Model). Appropriate statistics are needed.

9. use simple routine questionnaire:

Simple routine questionnaire and formal observational assessment of demands and needs are

very useful to determine inequities in services problems to access and concerns that can be

addressed and assist in planning of services an distribution of resources. Also, these


73

processes can be monitor whether best practices are met.

10. carefully design of security features:

Carefully design of security features and clear communication with core professions;

Concerns of core professions about computers because of loss of personal and professional

privacy should be addressed by both careful designing of security features and clear

communication with clinicians about how captured clinical information is going to be used.

6.3 Limitations

Just like in most studies on net benefits of IS success, the analysis in this study is limited by both

data and empirical specifications concerns. The data covers only the three hospitals involved; the

other hospitals in Suriname with successfully implement ISs and ERP systems may have been

overlooked. Another misestimation also might be the extent of adoption when some hospitals

combine other ISs with ERP from Microsoft Business Solution Navision and reverse and ERP from

Microsoft Business Solution Navision with those of other ERP packages. Moreover, the study focus

in three larger and general hospitals with more than 100 personnel and an identified automation

environment may render the results inapplicable to smaller hospitals. Nevertheless, this samples

does enable us to measure the success or effectiveness of IS on organizational performance,

achieving thereby a significant advantage.

6.4 Implications for Future Research

Taking in consideration that Suriname is a developing country and Health Care suffers a lot under

the social, economic and political circumstances I suffered some delay in searching for information

in general and at the public hospitals particularly. So applying concepts and models will be relative.

As this will not influence the overall results of the study I will consider it as a real evaluation and

thus a fundamental stage in evaluating the ISs in the hospital environment in Suriname. The

availability and accessibility of co-ordinators and respondents of the public hospitals was not easy

for successfully gathering of the required information.

Future studies should repeat this study for public and private hospitals separately as well as whether

any difference exists between the different subgroups of IS users, e.g. executives, mid-level

managers, end users and IS support personnel in terms of their ISs and ERP success model so as to

provide more comprehensive evaluation approach.. Differences in objectives and their importance

might also be studied as a measure of managerial effectiveness. Further studies are needed to


74

distinguish between evaluating different applications and evaluating how extensively and ERP

system is adopted for certain organizationally relevant functions. Careful consideration should be

given to the preparation time of the study, time and circumstances in the period of both preparation

and execution of the study as well as the size and nature of the sample; e.g. low-level administrator

may not be able to distinguish between problem solving and decision rationalization, but this

distinction may be more apparently engaged in scientific or analytical works.

Another issue for further study is a more in depth analysis on the factors reviewed by this study.

Proper weights should be assigned to relevant factors based on the special features of individual

industries (such a size and type of the industry) as they can help evaluate IS success model in a more

accurate and precise manner.

To attract current users, IS support personnel, physicians, nurses, and other core professions using

the ISs accessing up-to-date information provided in this context of the display of clinical

information may be one “carrot” that designers can offer them who otherwise may be reluctant to

use new ISs.


75

BIBLIOGRAPHY

1. Ahmad Mashhour and Zakira Zaatreh, A framework for evaluating the effectiveness of

Information Systems at Jordan banks; An Empirical Study, Journal of Internet banking and

Commerce, April 2008.

2. Algoe, Maltie, Suriname Health System Profile 2006 for MPH 1105, Coordinator NHIS –MOH,

2006.

3. Anderson, Sweeney, Williams, Feema and Shoesmith, Statistics for Business

Hair, Joseph F, Jr, Black, William C., Babin, Barry J., Rolf, Anderson, Rollph E., and Tatham,

Ronald L, Multivariate Data Analyses, 6th Edition, Pearson, Prentice Hall, 2006.

4. Arens, Alvin, A, Elder, Randal J., Beasley, Mark S. Beasley, Auditing and Assurance Services, An

Integrated Approach, 12th Edition, Prentence Hall, Upper Saddle River, New Jersey 07458, 2008.

5. Crape, Byron, PhD., Health Information Systems Assessment of Suriname, Assessment of the

Suriname Health Information Systems for now and the future, The Ministry of Health Suriname

and the Pan American Health Organization, 2007.

6. Creswell, John W., Research Design, Quantitative, and Mixed Methods Approaches, 3rd Edition,

University of Nebraska-Lincoln, 2009.

7. DeLone, William H. and McLean, Ephaim R., Information Success. The Quest for the Dependent

Variable. Information Systems Research 1992.

8. DeLone, William H. and McLean, Ephaim R., The DeLone and McLean Model of Information

Systems Success: A Ten-Year Update, Journal of Management Information Systems, Volume 19

No. 40: 9-30, Spring 2003

9. De Ware Tijd, Ziekenhuisdirecteuren naar Nederland, Local Newspaper, November 11, 2008

10. De Ware Tijd (2008). Waterberg bij afsluiting 2008: “Medisch equipment en


76

Opleidingsmogelijkheden op stapel”, Local Newspaper, December 31, 2008

11. Grover, Varun and Jeong, Seung Ryul and Sergers, Albert H., Information Systems

effectiveness: The construct space and patterns of application. Information & Management,

1996.

12. Hamilton, Scott and Chervany, Norman L., Evaluating Information System effectiveness

– Part I: Comparing Evaluation Approaches, MIS Quarterly, 1981.

13. Hamilton, Scott and Chervany, Norman L., Evaluating Information System effectiveness

– Part II: Comparing Evaluator Viewpoints, 1981.

14. Ives, Black. Olson, Margerethe H. and Baroudi,Jack, 1983, The Measurement of User

Information Satisfaction, Communications of the ACM, 1983.

15. Kaplan R. S. and Norton, The Balanced Scorecard, Measures that Drive performance, Harvard

Business Review, Volume 70, No. 1: 71-79,January-February 1992.

16. Laudon, Kenneth C. and Laudon, Jane P., Management Information Systems, 6th edition,

Organizational and Technology in the networked enterprise, Prenhall 2000.

17. Marshall B. Romney and Paul J. Steinbart, Accounting Information Systems, 10th edition,

Prenhall, 2006.

18. Meijeden van der, M.J., Tange H.J., Troost, J., and Hasman, A.,Determinants of Success of

Inpatient Clinical Information Systems: A Literature Review, The Journal of American Medical

Information Association, May-June 2003.

19. Seddon, Peter B., A Re-specification and Extension of the DeLone and McLean Model of IS

Success, Department of Information Systems, The University of Melborne, Parkville, Victoria

3052, Australia, Information System Research vol. 8, No. 3, September 1997.

20. Pekka Turunen and Hannu Salmela, The cost-benefit approach to medical information systems

evaluation, Turku School of Economics and Business Administration, Institute of Information


77

Systems, Turku Centre for Computer Science, TUCS Technical Report No 195, September 1998

ISBN 952-12-0252-1 and ISSN 1239-1891, 1998.

21. Rackaityte Renata, AIS effectiveness of criteria and indicator, setting up system, Luthuanian

University of Agricultue, 2003.

22. Shih-Wen Chien, An extension of D&M Model, ERP Implementation Environment –High Tech

manufacturing Firms in Taiwan, 2004.

23. Van der Loo, R.P., Van Gennip, E. M. S. J. and Baker, A, R., Evaluation of automated

information systems in health care; an approach to classifying evaluative studies. Computer

Methods and Programs in Biomedicine, 1995.

24. www.szn.sr May 29, 2009 19:55 hours, Last update: May 22, 2009.

25. Yin, Robert K, Case Study Research, Design and methods, Sage Publications, Inc, 3rd Edition,

Applied Social Research Methods Series, Volume 5, 2003.

26. Gitman, Lawrence J., Principles of Managerial Finance, 11th Edition, Pearson Education, ISBN

0321311507, 2006

http://www.szn.sr/


78

APPENDIX 1: Description of data set

Description of Data set Var. Variable description Part # Variable Question code in quest. type in survey System Quality (SQ) DeLone & McLean (1992) 1 SQ1 Timeliness 1 Metric 1SQ1 SQ2 Accuracy 1 Metric 1SQ2 SQ3 Ease of use (convenience of access) 1 Metric 1SQ3 SQ4 Consistency (availability/error recovery) 1 Metric 1SQ4 SQ5 Flexibility (response/turnaround time) 1 Metric 1SQ5 SQ6 Integration 1 Metric 1SQ6 SQ7 Reliability 1 Metric 1SQ7 Information Quality (IQ) Rai et al (2002) 2 IQ1 Format 2 Metric 2IQ1 IQ2 Content 2 Metric 2IQ2 IQ3 Timeliness 2 Metric 2IQ3 IQ4 Relevance 2 Metric 2IQ4 IQ5 Accuracy (precision) 2 Metric 2IQ5 IQ6 Reliability (completeness) 2 Metric 2IQ6 Service Quality (SRQ) DeLone & McLean (2003) 3 SRQ1 Responsiveness 3 Metric 3SRQ1 SRQ2 Reliability 3 Metric 3SRQ2 SRQ3 Empathy 3 Metric 3SRQ3 SRQ4 Assurance 3 Metric 3SRQ4 SRQ5 Support 3 Metric 3SRQ5 Perceived ease of use (PEU) Davis (1989) 4 PEU1 Processing customers request (Actual research, 2009) 4 Metric 4PEU1 PEU2 User friendliness 4 Metric 4PEU2 PEU3 Information search 4 Metric 4PEU3 PEU4 Ease of use 4 Metric 4PEU4

Perceived Usefulness (PU) Davis (1989) = BI = IU =Use (TAM) 5

PU1 Task innovation (effectivety) 5 Metric 5PU1 PU2 Task productivity 5 Metric 5PU2 PU3 (User) Efficiency 5 Metric 5PU3 PU4 Customer satisfaction 5 Metric 5PU4 PU5 Management control 5 Metric 5PU5

Intention to Use (IU) = Behavior Intention (BI) Bedard et al. (2003) 6

IU1 Behavior (Fishbein & Ajzen) 6 Metric 6IU1 IU2 Target (Fisbein & Ajzen) 6 Metric 6IU2 IU3 Situation in which performing (Fisbein & Ajzen) 6 Metric 6IU3 IU4 Time of performing (Fisbein & Ajzen) 6 Metric 6IU4 IU4 Time of performing (Fisbein & Ajzen) 6 Metric 6IU5


79

System Usage (SU) 7 SU1 Intensity (hours spent, number of users) 7 Metric 7SU1 SU2 Frequency (or simple binary variable: use/none use) 7 Metric 7SU2 SU3 Scope (overall) 7 Metric 7SU3 User Satisfaction (US) DeLone & McLean (1992) 8 US1 Satisfaction with specifics 8 Metric 8US1 US2 Content (Doll et al, 1988) 8 Metric 8US2 US3 Accuracy (Doll et al, 1988) 8 Metric 8US3 US4 Format (Doll et al, 1988) 8 Metric 8US4 US5 Ease of Use (Doll et al, 1988) 8 Metric 8US5 US6 Timeliness (Doll et al, 1988) 8 Metric 8US6 US7 Information satisfaction 8 Metric 8US7 US8 Overall Satisfaction 8 Metric 8US8 US8 Overall Satisfaction 8 Metric 8US9

Benefits of Use from end-users' view (BU) Martinsons, et al. (1999) = Benefits of Use (BU) 9

BU1 Improve individual productivity 9 Metric 9BU1 BU2 Decision effectiveness 9 Metric 9BU2 BU3 Learning 9 Metric 9BU3 BU4 Task performance 9 Metric 9BU4 BU5 Problem identification 9 Metric 9BU5

Net value from organizational view (NV) Martinsons, et al., (1999) = Net Benefits (NB) 10

NB1 Operating cost reduction 10 Metric 10NB1 NB2 Overall productivity gains 10 Metric 10NB2 NB3 Improvement of organizational image 10 Metric 10NB3 NB4 Service effectiveness 10 Metric 10NB4 NB5 Increased sales/revues/profit 10 Metric 10NB5 NB6 Increased work volume 10 Metric 10NB6 Demographic variables 11

DM1 Hospital name 11 Non metric 11DM1

DM2 Department/Unit 11 Non metric 11DM2

DM3 Job description 11 Non metric 11DM3

DM4 Years in current position 11 Metric 11DM4 DM5 Years in current hospital 11 Metric 11DM5

DM6 Highest level of education 11 Non metric 11DM6

DM7 Position in Organizational Hierarchy 11 Non metric 11DM7

DM8 Age 11 Metric 11DM8

DM9 Gender 11 Non metric 11DM9

DM10 Length of use 11 Metric 11DM10 DM11 Experience with computers 11 Metric 11DM11 DM12 Experience with Health Care Systems 11 Metric 11DM12 Prepared by: Delano Gefferie


80

APPENDIX 2: Questionnaire; English version

Paramaribo, April 15, 2009 To: All users and IT-staff of hospitals in Suriname Thesis project: Evaluating the effectiveness of Information Systems in a hospital environment Dear users/IT-staff, Herewith I send you a questionnaire related to my thesis project at Maastricht School of Management in cooperation with the Lim A Po Institute for Social studies. You are requested to fill out this questionnaire regarding my research concerning the effectiveness of Information Systems in hospitals in Suriname. The purpose of this study is to learn more about what factors are influencing organizational performance in a hospital environment and what impact they have on the performance of the hospitals in Suriname. This research will provide the management of the hospitals in Suriname with information, needed to improve the effectiveness of the Information Systems they have in place in particular and hospital management in general. You are eligible to participate, because you are registered as user or developer of information Systems in your hospital. In this research s'Lands Hospital, Diakonessenhuis and Streekziekenhuis Nickerie are participating. You are requested to fill out the attached questionnaire. The survey will last approximately 10-15 minutes and there is no need asking for your name during this survey. Questionnaires completed should be sent within 14 days to the coordinator of your hospital. For s'lands Hospital Mrs. Scholsberg-Lieveld, Secretary of CEO , Diakonessenhuis, Mrs. A. Fitz Jim, Manager Planning and Control and for Streekziekhuis Nickerie Mr. Lalay, Financial Manager. You can obtain further information about this study or voice concerns or complaints about the study by e-mailing or calling the researcher, Delano Gefferie, at [email protected] or mobile

8819239 Thanks in advanced. Kind regards, Delano Gefferie


81

Evaluating effectiveness of Information Systems on performance in a Hospital Environment THE QUESTIONNAIRE (for all users of Information Systems and IS support staff) Please provide the following information? All information you provide will be confidential and will not be identified with you. Part 1: System Quality (SQ) This following sections of the survey conveys your own personal feeling concerning the use of the Information Systems you use on your department or unit of the hospital. You will be asked "To what extent are the following statements accurate". Please use the scale for this answers:

Please follow these instructions: Check each scale in the position that describes your evaluation

1 of the factor being described? Check each scale, do not omit any?

2 Check only one position for each scale?

3 Check in the space on scale from 1 to 7 with X 1 = Very Strongly Disagree 2 = Strongly Disagree 3 = Disagree 4 = Neutral 5 = Agree 6 = Strongly Agree 7 = Very Strongly Agree

4 Work rapidly. Rely on first impression!

5 The qualifications strongly disagree and strongly agree at the beginning and the end of the scales are stated to support your orientation!

6 This questionnaire consists of eleven (11) parts. Verify after completion if all parts have been

filled in! To what extent do you agree with the following statements? 1 Information Systems provide up-to-date information. 1 2 3 4 5 6 7 disagree agree 2 Information Systems are accurate. 1 2 3 4 5 6 7 disagree agree 3 The Systems are easy to learn and to use. 1 2 3 4 5 6 7 disagree agree 4 Systems are available when I need it. 1 2 3 4 5 6 7 disagree agree 5 Systems are flexible. 1 2 3 4 5 6 7 disagree agree 6 Information Systems are integrated. 1 2 3 4 5 6 7 disagree agree 7 The systems are not subject to frequent problems and crashes. 1 2 3 4 5 6 7 disagree agree Part 2: Information Quality (IQ) 1 Information is provided in a format which is useful for me. 1 2 3 4 5 6 7 disagree agree


82

2 The Information Systems provide output that is exactly what you need. 1 2 3 4 5 6 7 disagree agree 3 I get the information I need in time. 1 2 3 4 5 6 7 disagree agree 4 I have more relevant, useful and significant information. 1 2 3 4 5 6 7 disagree agree 5 The systems provide accurate and clear information. 1 2 3 4 5 6 7 disagree agree 6 The information is not subject to misinterpretation and discussions. 1 2 3 4 5 6 7 disagree agree Part 3: Service Quality (SRQ) 1 Support/IT-staff give prompt service to users. 1 2 3 4 5 6 7 disagree agree 2 IT staff performed the task in and at the time they had promised 1 2 3 4 5 6 7 disagree agree 3 Relation with IS/IT-staff is good. 1 2 3 4 5 6 7 disagree agree 4 IT/IS staff have the knowledge to do their job well 1 2 3 4 5 6 7 disagree agree 5 IT department has up to date hardware and software. 1 2 3 4 5 6 7 disagree agree Evaluation of overall Information Systems Part 4: Perceived Ease of Use (PEU) 1 It will be easy to get Information Systems to do what I want to do. 1 2 3 4 5 6 7 disagree agree 2 Information Systems will be flexible to interact with. 1 2 3 4 5 6 7 disagree agree 3 My interaction with Information Systems will be clear and understandable. 1 2 3 4 5 6 7 disagree agree 4 Information Systems will be easy to use. 1 2 3 4 5 6 7 disagree agree Part 5: Perceived Usefulness (PU) 1 Using Information Systems will enhance my effectiveness on the job 1 2 3 4 5 6 7 disagree agree 3 Using Information Systems increase my productivity. 1 2 3 4 5 6 7 disagree agree 2 Using Information Systems will make my job easier 1 2 3 4 5 6 7 disagree agree 4 Using Information Systems will enable me to accomplish task faster 1 2 3 4 5 6 7 disagree agree 5 Using Information Systems improve my job performance. 1 2 3 4 5 6 7 disagree agree


83

Part 6: Intention to Use (IU) 1 I Intend to use Information Systems in performing analytical procedures. 1 2 3 4 5 6 7 disagree agree 2 I Intend to use Information Systems in planning and tailoring related programs. 1 2 3 4 5 6 7 disagree agree 3 I intend to use Information Systems in electronic mode, rarely printing out copies 1 2 3 4 5 6 7 of work papers as I proceed through my tasks. disagree agree 4 I intend to use Information Systems to help me make decisions 1 2 3 4 5 6 7 disagree agree 5 I intend to increase use Information Systems 1 2 3 4 5 6 7 disagree agree Part 7: Use (SU) 1 I use the Information Systems extensively (100% of my job time) 1 2 3 4 5 6 7 disagree agree 2 I use the Information Systems seven (7) days a week (1 = 1 day and 7 is 7 days) 1 2 3 4 5 6 7 disagree agree 3 I am not forced by management to use the Information Systems at my disposal 1 2 3 4 5 6 7 I use the Information Systems because it is totally mandatory disagree agree 4 I use all Information Systems within the organization 1 2 3 4 5 6 7 disagree agree Evaluation of overall Information Systems Part 8: User satisfaction (US) 1 Information Systems provide sufficient information for effective decision making. 1 2 3 4 5 6 7 (objectives and alternatives) disagree agree 2 Information Systems provides reports that seem to be just about exactly what 1 2 3 4 5 6 7 I need. disagree agree 3 Accuracy of output information is high. 1 2 3 4 5 6 7 disagree agree 4 Information Systems reports are in a format which allow me to perform my duties 1 2 3 4 5 6 7 effectively. disagree agree 5 Users understand the systems well. 1 2 3 4 5 6 7 disagree agree 6 I get the information I need to perform my job effectively in time. 1 2 3 4 5 6 7 disagree agree 7 Output information is relevant (to intended function). 1 2 3 4 5 6 7 disagree agree 8 User’s feeling of participation is sufficient. 1 2 3 4 5 6 7 disagree agree 9 Overall satisfaction of the Information Systems is high. 1 2 3 4 5 6 7 disagree agree Evaluation of overall Information Systems


84

Part 9: Benefit of Use from end-users' view (BU) 1 Information Systems allow me to communicate with other users in an effective way. 1 2 3 4 5 6 7 disagree agree 2 Information Systems provide me with sufficient support in all stages of the 1 2 3 4 5 6 7 decision making process. disagree agree 3 Information Systems help me effectively manage and store knowledge that I need. 1 2 3 4 5 6 7 disagree agree 4 The Information Systems provide the information required to perform my duties. 1 2 3 4 5 6 7 disagree agree 5 Information Systems help me to identify problems in an early stage. 1 2 3 4 5 6 7 disagree agree Part 10: Net Benefits from organizational view (NB): 1 Information Systems enhance competitiveness or create strategic advantage. 1 2 3 4 5 6 7 disagree agree 2 Information systems enable the organization to respond more quickly to change. 1 2 3 4 5 6 7 disagree agree 3 Information Systems have dramatically increased organizational productivity. 1 2 3 4 5 6 7 disagree agree 4 Information System projects provide organizational success. 1 2 3 4 5 6 7 disagree agree 5 Information Systems establish and maintain a good image. 1 2 3 4 5 6 7 disagree agree 6 Information Systems have dramatically increased our service capacity. 1 2 3 4 5 6 7 disagree agree Evaluation of overall Information Systems

Part 11: Demographic Information

1 Hospital name:

2 Department/Unit: Years Years

3 Job description: (check one) in position in hospital Technician Nurse Pharmacist Physician Physician assistant Other (specify):


85

4 Highest level of education: (check one) Physician Nurse

High school

Associate degree Some college Bachelor's degree Master's degree Post master's degree Other (specify):

5 Position in Organizational Hierarchy/Position Classification: Operational level Lower Management Middle management Upper Middle Management Senior management

6 Age:

7 Gender: Male Female

8 Length of time (in months) you have used Health Care System/Accounting Information System (choose one):

9 Have you used other computer systems before? Yes No 10 If your answer was Yes to question 7, was it a Health Care System?

Yes No 11 Use of IS:

Frequency of use times weekly times monthly Times spent hours weekly hours monthly Evaluation of overall Information Systems

Your opinion is important. Please add any comment here:

Thank you for participating in this study!!! Evaluating of overall Information Systems



86

APPENDIX 3: Questionnaire; Dutch version Paramaribo, 27 april 2009 Aan: Gebruikers en IT-personeel Ziekenhuizen in Suriname Afstudeerproject: Beoordeling/Evaluatie van de Effectiviteit van Informatie Systemen (als geheel) in ziekenhuizen Geachte gebruikers en IT-personeel, met het Lim A Po Institute for Social studies een questionaire toekomen met het verzoek deze in te vullen i.v.m. mijn onderzoek naar de effectiviteit van geautomatiseerde Informatie Systemen in ziekenhuizen in Suriname. Het doel van het onderzoek is na te gaan in welke mate de effectiviteit van geautomatiseerde Informatie Systemen van invloed is op de performance van de ziekenhuizen in Suriname en de mate, waarin de de verschillende factoren, die hierbij van invloed zijn een rol spelen. Dit onderzoek zal het management van de ziekenhuizen in Suriname de nodige informatie voor verho- ging van de effectiviteit van de geautomatiseerde Informatie Systemen verschaffen en zo en bijdrage leveren aan het verhogen van de effectiviteit van de ziekenhuizen in het algemeen en de Informatie Systemen in het bijzonder. De Informatie Systemen, die in dit onderzoek als een geheel worden geëva- lueerd en waarop u zich derhalve bij invulling van de questionaire voordurend op moet richten zijn: 1. Het Navision Systeem met de reeds geoperationaliseerde modules; 2. Alle Informatie Systemen, waarvan u buiten het Navision Systeem gebruik maakt. De Informatie Systemen genoemd onder punt 1 en 2 zullen bij de invulling van de questionaire dus steeds gezamenlijk worden gewaardeerd. U bent vanwege u betrokkenheid als gebruiker of als ontwikkelaar van een of meerdere Informatie Systemen binnen uw ziekenhuis gekozen om deel te nemen aan het onderzoek, waarin het Diakonessenhuis, het s'Lands Hospitaal en het Streekziekenhuis Nickerie participeren. U wordt gevraagd bijgaande questionaire in te vullen. Het invullen hiervan zal ongeveer 15-20 minuten duren en het vermelden van uw naam is niet vereist. De ingevulde formulieren worden binnen 5 dagen terug verwacht bij de coördinator van uw ziekenhuis. Voor het s'Lands Hospitaal is dat, mevrouw Scholsberg-Lieveld, Directie Secretaresse. Voor het Diakonessenhuis, mevrouw A. Fitz Jim, Manager Planning and Control en voor het Streek- ziekenhuis Nickerie, de heer O. Lalay, Financial Manager. Indien u vragen mocht hebben ben ik gaarne bereid die te beantwoorden. Mijn contact informatie is als volgt: Telefoon: 08819239 E-mail: [email protected] Bij voorbaat dank voor uw medewerking! Met vriendelijke groet, Delano Gefferie


87

(Vragenlijst (voor gebruikers van Informatie Systemen en IS ondersteunend personeel) Nr…………….. Gelieve de volgende infomatie te verstrekken! Alle verstrekte informatie zal vertrouwelijk worden behandeld en niet met u in verband worden gebracht. In de volgende delen van de vragenlijst kunt u uw eigen mening geven over de door u in het ziekenhuis en op uw afdeling ingebruikzijnde Informatie Systemen als geheel. Uw wordt gevraagd "In welke mate de gestelde beweringen juist zijn". Gelieve bij de beantwoording uitsluitend gebruik te maken van de opgegeven schaal: Volg deze instructies: Geef bij elke bewering uw waardering in de positie, die uw mening het beste weergeeft: 1.. Gebruik elke schaal, sla geen enkele over? 2.. Vul slechts een beoordeling/waardering/positie per schaal in? 3.. Geef uw beoordeling op de schaal van 1 tot en met 7 aan met een X, onder het cijfer, dat u kiest:

1 = Ten Stelligste Oneens 2 = Zeer Oneens 3 = Oneens 4 = Neutraal 5 = Eens 6 = Zeer Eens 7 = Ten Stelligste Eens

4.. Reageer snel en vertrouw op uw eerste indruk over de factor, die u dan beoordeeld! 5.. De aanduidingen oneens en eens bij het begin en eind van de schalen zijn slechts bedoeld om

u te ondersteunen in uw oriëntatie! 6.. Deze questionaire bestaat uit elf (11) delen. Controleer na de invulling als u alle delen heeft invulgevuld!

In welke mate bent u het eens met de volgende beweringen? Deel 1: Systeem Kwaliteit 1 De Informatie Systemen leveren tijdige informatie af. 1 2 3 4 5 6 7

oneens eens 2 De Informatie Systemen zijn nauwkeurig/accuraat. 1 2 3 4 5 6 7

oneens eens 3 De Informatie systemen zijn makkelijk aan te leren en te gebruiken. 1 2 3 4 5 6 7

oneens eens 4 De Informatie Systemen zijn beschikbaar wanneer ik ze nodig heb. 1 2 3 4 5 6 7

oneens eens 5 De Informatie Systemen zijn flexibel in gebruik. 1 2 3 4 5 6 7

oneens eens 6 De Informatie Systemen zijn gekoppeld (geïntegreerd). 1 2 3 4 5 6 7

oneens eens 7 De Systemen zijn niet onderhevig aan (vrij van) regelmatige problemen en noodsituaties 1 2 3 4 5 6 7

oneens eens Deel 2: Informatie Kwaliteit 1 Informatie wordt in een voor mij bruikbare presentatie vorm/samenstelling/format verstrekt. 1 2 3 4 5 6 7

oneens eens 2 De Informatie Systemen leveren de output/uitvoering/resultaten op die ik nodig heb. 1 2 3 4 5 6 7

oneens eens


88

3 Ik krijg de benodigde Informatie op tijd. 1 2 3 4 5 6 7

oneens eens 4 Ik krijg ter zake dienende/relevantere, bruikbare en belangrijke Informatie. 1 2 3 4 5 6 7

oneens eens 5 De Informatie Systemen leveren nauwkeurige/accurate/zuivere en duidelijke informatie af. 1 2 3 4 5 6 7

oneens eens 6 De informatie is niet vatbaar voor verkeerde interpretatie en meningsverschillen. 1 2 3 4 5 6 7

oneens eens Evaluatiie van Informatie Systemen als geheel Deel 3: Service Kwaliteit 1 Ondersteund en IT-personeel verleend onmiddellijk ondersteuning aan gebruikers. 1 2 3 4 5 6 7

oneens eens 2 Ondersteunend en IT-personeel voeren hun taken naar behoren en zoals afgespro- 1 2 3 4 5 6 7

ken uit. oneens eens 3 De relatie met het ondersteunend en IT-personeel is goed. 1 2 3 4 5 6 7

oneens eens 4 Ondersteunend/IT-personeel beschikt over de kennis om hun werk goed te kunnen 1 2 3 4 5 6 7

doen. oneens eens 5 De IT-afdeling heeft bij de tijdse/moderne apparatuur/hardware en programmatuur/ 1 2 3 4 5 6 7

software. oneens eens Deel 4: Zichtbaar (aannemelijk) gemak bij gebruik 1 Het zal makkelijk zijn d.m.v. de Informatie Systemen te doen wat ik wil. 1 2 3 4 5 6 7

oneens eens 2 De Informatie Systemen zullen flexibel zijn in het gebruik. 1 2 3 4 5 6 7

oneens eens 3 De omgang met de Information System zal duidelijk en verstaanbaar zijn. 1 2 3 4 5 6 7

oneens eens 4 De Informatie Systemen zijn makkelijk/simpel om mee te werken. 1 2 3 4 5 6 7

oneens eens Deel 5: Verwacht gebruik (aannemelijk nut) in de toekomst 1 Het gebruik van de Informatie Systemen zal mijn effectiviteit op het werk verhogen. 1 2 3 4 5 6 7

oneens eens 3 Het gebruik van de Informatie Systemen zal mijn arbeidsproductiviteit verhogen. 1 2 3 4 5 6 7

oneens eens 2 Het gebruik van de Informatie Systemen zal mijn werk makkelijker maken. 1 2 3 4 5 6 7

oneens eens

4 Het gebruik van de Informatie Systemen zal mij in staat stellen mijn werk sneller af te krijgen. 1 2 3 4 5 6 7

oneens eens 5 Het gebruik van de Informatie Systemen zal mijn taakuitvoering verbeteren. 1 2 3 4 5 6 7

oneens eens Deel 6: Beoogd gebruik 1 Ik ben van plan de Informatie Systemen te gebruiken bij analytische werkzaamheden. 1 2 3 4 5 6 7


89

oneens eens 2 Ik ben van plan de Informatie Systemen te gebruiken in planning en aanpassing van samen- 1 2 3 4 5 6 7

werkende/gekoppelde programma's. oneens eens

3 Ik ben van plan de Informatie Systemen in de electronische mode te gebruiken en weinig uit te 1 2 3 4 5 6 7

te printen bij mijn taakuitvoering. oneens eens 4 Ik ben van plan de Informatie Systemen te gebruiken bij het nemen van beslissingen. 1 2 3 4 5 6 7

oneens eens 5 Ik ben van plan het gebruik van de Informatie Systemen te verhogen 1 2 3 4 5 6 7

oneens eens Evaluatiie van Informatie Systemen als geheel Deel 7: Systeem gebruik 1 Ik gebruik de Informatie Systemen intensief (100% van mijn werktijd). 1 2 3 4 5 6 7

oneens eens 2 Ik gebruik de Infomatie Systemen zeven (7) dagen in de week (1 = 1 dag en 7 is 7 dagen). 1 2 3 4 5 6 7

oneens eens 3 Ik ben niet verplicht door de leiding om de mij ter beschikking gestelde Informatie Systemen 1 2 3 4 5 6 7

te gebruiken. oneens eens 4 Ik gebruik alle Informatie Sytemen binnen de organisatie. 1 2 3 4 5 6 7

oneens eens Deel 8: Gebruikerstevredenheid 1 De Informatie Systemen leveren toereikende/voldoende informatie voor effectieve 1 2 3 4 5 6 7

besluitvorming (doelstellingen en alternatieven). oneens eens

2 De Informatie Systemen leveren exact de rapportages, die exact nodig heb af. 1 2 3 4 5 6 7

oneens eens 3 Accuratesse/zuiverheid van de output/uitvoer informatie is hoog. 1 2 3 4 5 6 7

oneens eens 4 De Informatie Systemen leveren de verslagen in de format/samenstelling, die mij in staat 1 2 3 4 5 6 7

stellen mijn taken effectief te doen. oneens eens 5 De gebruikers kunnen goed omgaan met de Informatie Systemen. 1 2 3 4 5 6 7

oneens eens 6 Ik krijg de informatie, die ik nodig heb om mijn werk effectief te doen op tijd. 1 2 3 4 5 6 7

oneens eens

7 Output/uitvoer informatie is relevant voor mijn functie. 1 2 3 4 5 6 7 oneens eens

8 Het gebruikers' gevoel van betrokkenheid/er bij behoren is voldoende. 1 2 3 4 5 6 7 oneens eens 9 De algehele tevredenheid/satisfactie over de Informatie Systemen is hoog. 1 2 3 4 5 6 7

oneens eens Deel 9: Voordelen gezien vanuit de belevingswereld van de gebruiker 1 De Informatie Systemen maken effectieve/doelgerichte communicatie met andere 1 2 3 4 5 6 7

gebruikers mogelijk. oneens eens


90

4 De Informatie Systemen geven me voldoende ondersteuning in alle fasen van het 1 2 3 4 5 6 7

besluitvormingsproces. oneens eens 3 De Informatie Systemen ondersten me effectief bij het managen/leiding geven en 1 2 3 4 5 6 7

het opslaan van de nodige kennis. oneens eens 4 De Informatie Systemen leveren de voor mijn werk vereiste informatie. 1 2 3 4 5 6 7

oneens eens 5 De Informatie Systemen ondersteunen me om problemen in een vroeg stadium te 1 2 3 4 5 6 7

identificeren. oneens eens Deel 10: Uiteindelijke waarde creatie vooruit de organisatie bezien 1 De Informatie Systemen verhogen de concurrentiekracht en/of creëren strategische 1 2 3 4 5 6 7

voordelen. oneens eens 2 De Informatie Systemen stellen de organisatie in staat om snel te reageren op ver- 1 2 3 4 5 6 7

anderingen. oneens eens 3 De Informatie Systemen hebben de productiviteit in het algemeen sterk verhoogd. 1 2 3 4 5 6 7

oneens eens 4 De implementatie van Informatie Systemen maakt de organisatie succesvol. 1 2 3 4 5 6 7

oneens eens 5 De Informatie Systemen vestigen en zorgen voor een blijvend goede reputatie 1 2 3 4 5 6 7

voor de organisatie. oneens eens 6 De Informatie Systemen hebben de klantgerichtheid sterk verhoogd/verbeterd. 1 2 3 4 5 6 7

oneens eens Evaluatie van Informatie Systemen als geheel

Evaluatie effectiviteit geautomatiseerde Informatie Systemen (als geheel) in ziekenhuizen in Suriname

Nr. …

QUESTIONAIRE (voor alle gebruikers van Informatie Systemen en IT personeel)

Gelieve de volgende infomartie in te vullen! Alle informatie zal confidentieel worden behandeld.

Deel 11: Algemene informatie:

1 Naam ziekenhuis:

2 Afdeling:

Jaren jaren

3 Functie (kies één) in functie in ziekenhuis

Technicus

Verpleegkundige

Apotheker


91

Medicus

Medisch assistent

Anders (vul in):

4 Hoogst genoten opleiding Medicus Verpleegkundige

Middelbare school

Diploma

Universiteit

Bachelor's graad

Master's graad

Post master's graad

Anders (vul in):

5 Plaats in de ziekhuishuis Hierarchy

Uitvoerend niveau

Lager management

Middle management

Senior management

6 Leeftijd:

7 Geslacht: Mannelijk Vrouwelijk

8 Periode (in maanden of jaren) gedurende welke u reeds gebruik maakt van de huidige geautomatiseerde Informatie Systemen:

Maanden Jaren

9 Heeft u eerder gebruik gemaakt van andere computer systemen?

Ja Nee

10 Als het antwoord op vraag 9 Ja is, was of waren het Medisch Informatie Syste(e)m(en)?

Ja Nee

11 Gebruik van de geautomatiseerde Informatie Systemen in het ziekenhuis (als geheel):

Frequentie van gebruik: keren per week keren per maand

Bestede tijd: uren per week Uren per maand


92

Uw mening is belangrijk. Gelieve uw opmerkingen hier te vermelden:

Bedankt voor uw deelname aan dit onderzoek!!!

Evaluatie van Informatie Systemen als een geheel



93

APPENDIX 4: Results pre-test

Reliability analysis

n=8

Nr. Scale Dimensions N of items Cronbach

α

1 System Quality (SQ) 7 7 0.869

2 Information Quality (IQ) 6 6 0.860

3 Service Quality (SRQ) 5 5 0.833

4 Perceived Ease of Use (PEU) 4 4 0.873

5 Perceived usefulness (PU) 5 5 0.995

6 Intention to Use (IU) 4 5 0.935

7 System Usage (SU) 3 2 0.581

8 User Satisfaction (US) 8 9 0.963

9 Benefit of Use from end-users view (BU) 5 5 0.940

10 Benefits of Use from organizational View (BU) 6 6 0.966

Confirmatory factor should be > 0.5 and ideal 0.7: Hair et al (2005)



94

APPENDIX 5: Indication of human resources in health sector in Suriname

Source: Ministry of Health; Suriname Health Profile 2006 for MPH 1105


95

APPENDIX 6: Hospital sample

Demographics HOSPITAL

Frequency Percent Valid Percent Cumulative

Percent DKZ 69 69.7 72.6 72.6LH 14 14.1 14.7 87.4SZN 12 12.1 12.6 100.0

Valid

Total 95 96.0 100.0 Missing System 4 4.0 Total 99 100.0

Department

Frequency Percent Valid missing 13 13.1 adm. 1 1.0 afd. inkoop 1 1.0 automatisering 1 1.0 bedden huis 2 2.0 boekhouden 2 2.0 boekhouding/financiële

zaken 1 1.0

centrale inkoop 1 1.0 chirurgie 2 2.0 cm 2 2.0 crp 1 1.0 debiteuren 1 1.0 debiteuren administratie 1 1.0 facturatie 1 1.0 fin. bewakings unit 1 1.0 finan. adm 12 12.1 finan.adm 1 1.0 gym. 1 1.0 gyn 1 1.0 help desk2 1 1.0 inkoop 1 1.0 int.mn. 1 1.0 it 1 1.0 jc 1 1.0 kinder afd. 1 1.0 lab 2 2.0 lab adm. 1 1.0 loon adm. 3 3.0


96

med. adm. 1 1.0 med. arch 1 1.0 med. reg. 13 13.1 med.adm. 1 1.0 med.administratie 1 1.0 medische adm. 1 1.0 mr 1 1.0 opname 2 2.0 opname adm. 1 1.0 p&o 1 1.0 paramedische dienst 1 1.0 pers. zaken 2 2.0 pl & ltnl 1 1.0 planning en control 1 1.0 planning/ controle 2 2.0 poliklinieken 1 1.0 poort adm. 1 1.0 ro 1 1.0 rontgen 1 1.0 salaris administratie 1 1.0 systeembeheer 1 1.0 vj 1 1.0 vk- bk 2 2.0 zorgmanagement 2 2.0 Total 99 100.0

Profession


Percent Technician 2 2.0 20.0 20.0 nurse 3 3.0 30.0 50.0 medical assistant 5 5.1 50.0 100.0

Valid



97

prof_other

Frequency Percent Valid 24 24.2 adm 1 1.0 adm. 4 4.0 adm. kracht 4 4.0 adm. medew. 11 11.1 adm. medewerker 2 2.0 adm.kracht 9 9.1 administratie 1 1.0 afd. inkoop(adm) hfd

inkoop 1 1.0

afd. secr. 4 4.0 afd.secr. 1 1.0 afdelings secr 2 2.0 applicatie beheerder 1 1.0 boekhouder 1 1.0 chef loon adm. 1 1.0 crediteuren adm. 1 1.0 finad 2 2.0 hrm medewerker 1 1.0 junior adm.medew. 2 2.0 junior medew. med. reg. 1 1.0 leiding gevende 1 1.0 manager 1 1.0 manager pl & ltnl 1 1.0 med secretaresse 2 2.0 med. adm. 2 2.0 med.secretaresse 2 2.0 medew. med. reg 1 1.0 medewerkster opname 1 1.0 medische adm. 1 1.0 personeelsfunctionaris 1 1.0 secretaresse 2 2.0 sen. adm. medew. 1 1.0 senior medewerker sa 1 1.0 sr medew.pz 1 1.0 sub hfd. 1 1.0 systeembeheer 1 1.0 teamleader klinische fact 1 1.0 teamleider/en adm 1 1.0 verpl. secr. 1 1.0 zorgmanager 2 2.0 Total 99 100.0

years_function


98


Percent 1-5 years 23 23.2 39.0 39.06-10 years 8 8.1 13.6 52.511-15 years 13 13.1 22.0 74.616-20 years 3 3.0 5.1 79.721-25 years 4 4.0 6.8 86.426-30 years 5 5.1 8.5 94.9> 30 3 3.0 5.1 100.0

Valid


years_hospital


Percent 1-5 years 18 18.2 27.7 27.76-10 years 6 6.1 9.2 36.911-15 years 23 23.2 35.4 72.316-20 years 7 7.1 10.8 83.126-30 years 6 6.1 9.2 92.3> 30 5 5.1 7.7 100.0

Valid


Education


Percent high school 45 45.5 73.8 73.8 diploma 5 5.1 8.2 82.0 university 7 7.1 11.5 93.4 bachelors degree 3 3.0 4.9 98.4 masters degree 1 1.0 1.6 100.0

Valid



99

degree_other


Percent 76 76.8 76.8 76.8 alg. econ.(net afgerond) 1 1.0 1.0 77.8 docent verplk i.c 1 1.0 1.0 78.8 drogist 2 2.0 2.0 80.8 imeao 1 1.0 1.0 81.8 inkoop manangment 1 1.0 1.0 82.8 iol 2 2.0 2.0 84.8 iol/hbo 1 1.0 1.0 85.9 lbgo 1 1.0 1.0 86.9 master in health services mgt. 1 1.0 1.0 87.9

mcsa 1 1.0 1.0 88.9 mulo 4 4.0 4.0 92.9 mulo. med. reg. 1 1.0 1.0 93.9 pd/mba 1 1.0 1.0 94.9 thans bezig met masters 1 1.0 1.0 96.0 ulo 2 2.0 2.0 98.0 ulo/mietov 1 1.0 1.0 99.0 ulo/mulo 1 1.0 1.0 100.0

Valid

Total 99 100.0 100.0 q11_5_a


Percent executive level 57 57.6 78.1 78.1 lower management 6 6.1 8.2 86.3 middle management 5 5.1 6.8 93.2 senior management 5 5.1 6.8 100.0

Valid


AGE


Percent 20-30 15 15.2 16.7 16.731-40 35 35.4 38.9 55.641-50 24 24.2 26.7 82.251-60 16 16.2 17.8 100.0

Valid



100

gender


Percent man 12 12.1 13.2 13.2women 79 79.8 86.8 100.0

Valid


number of months in use of advanced information system


Percent 88 88.9 88.9 88.9 5 2 2.0 2.0 90.90 1 1.0 1.0 91.910 1 1.0 1.0 92.9108 1 1.0 1.0 93.912 1 1.0 1.0 94.918 1 1.0 1.0 96.024 1 1.0 1.0 97.06 2 2.0 2.0 99.09 1 1.0 1.0 100.0

Valid

Total 99 100.0 100.0 Number of years using advanced IS


Percent 1-5 years 58 58.6 81.7 81.76-10 years 9 9.1 12.7 94.411-15 years 4 4.0 5.6 100.0

Valid


use week


Percent 1-10 hours 57 57.6 96.6 96.6 31-40 hours 1 1.0 1.7 98.3 more than 40 hours 1 1.0 1.7 100.0

Valid


Source: Own research and hospitals in study


101

Open Question Comments 64 missing 1. English: With the view from my position I think that the information system works well but point of focus/importance

is the information processed by all users Dutch : Bekeken van uit mijn werk vind ik dat het informatie systeem goed werk levert maar het gaat om de informatie die ingevoerd wordt door alle gebruikers

2. English: It takes too long to get the final invoices out of the system after control activities and postings Dutch : Bij controle werkzaamheden te langdradig en steeds doorboeken totdat uiteindelijke fact. uit de bus komt

3. English: Please start every part with a short explanation of the topic to avoid difference of interpretation Dutch : Bij de enquête gaarne korte uitleg bij de verscheidene kopstukken/delen i.v.m. interepretatie verschillen

4. English: Management of the hospital should learn to listen to employees/subordinates. I am still not satisfied with the performance of the system/how the system works and how problems are solved/handled. Too often trust is on Dk-nr. They forget often that the system is a “chain” and not fragmented “parts/eyelets”. Dutch : De beleidsmakers v/h huis zouden moeten leren luisteren naar hun medewerkers. ik ben nog steeds niet tevreden hoe het systeem werkt en problemen worden opgelost. Men vertrouwd te vaak alleen de te gebruiken dk- nr. men vergeet te vaak dat het systeem een ketting is en geen losse "oogjes"

5. English: The implementation of Navision was a step in the right direction; especially for the administrative part there Is more “transperancy” now Dutch : De implementatie van navision was een stap in de goede richting; vooral voor het administratief gedeelte is er meer "openheid" in zaken gekomen

6. English: Still there is no 100% automation. dropping “Grote Beer” for “ Account View” is a change but no improvement. Efficiency and effectiveness leave much to be desired still. Dutch : Er is nog steeds geen sprake van 100% automatisering. De overstap van progamma "grote beer" naar accountview " is een verandering, maar geen verbetering. effectiviteit en efficiency laten nog veel te wensen over

7. English: Navision is used daily on regular base but if someone else logged in I cannot use it anymore. Dutch : er wordt dagelijks gebruik gemaakt van Navision een aantal keren p/dag als er een ander gebruiker in nav zit kan ik daarna geen gebruik maken van Navision

8. English: Some questions are formulated negatively what forced me to go back to the scale of every statement again and again. The questionnaire is compile at a certain level that not all levels will understand Dutch : Er zijn vragen die negatief geformuleerd zijn, wat maakt dat je iedere keer weer naar de antwoord mogelijk- lijkheden moet kijken. De vragenlijst is op een bepaald niveau opgesteld, dat niet alle "lagen" zal begrijpen.

9. English: Automated IS are used in this hospital and we and get the information we need. Dutch : Geautomatiseerde info systemen wordt wel gebruikt in dit ziekenhuis en we kunnen wel informatie krijgen

10. English: It is impossible to imagine life today without information systems in actual setting in the hospital environment. Of course it always could be better and more effectively. Dutch : Gebleken is dat informatie systemen in zijn huidige vorm een niet weg te denken proces is binnen het ziekenhuis wezen. Uiteraard kan het altijd nog beter en effectiever

11. English: Please use other font (difficult to read, font size too small. Dutch : Gelieve andere letter type te gebruiken (moeilijk te lezen, te kleine lettertype

12. English: Implementation of an Information System is a must but the system should be accessible at any time during working hours. Some of the PCs are slow which irritate users but influence user-friendliness in a positive way/sense (shorter waiting time at admission is shorter) and providing information is faster Dutch : Het hebben van een IS is noodzakelijk, maar het systeem moet op elk tijdstip v/d dag (zolang we in dienst zijn) toegankelijk zijn. Sommige PCs zijn traag, wat irritatie met zich mee brengt. dat het de klant vriendelijkheid bevodert is goed(kortere wachttijden bij opname) en leveren bepaalde inf. snel uit.

13. English: It’s always a pleasure/good/nice to have proper/sufficient ISs to do your job adequately/well. Dutch : Het is altijd fijn om een goede informatie systeem te hebben om je werk naar behoren te doen.

14. English: the system works/performs well. I use the system for recording/input. I am not part of management. Dutch : het systeem functioneert goed. ik gebruik het voor het invoer werk. ik maak geen beleid uit

15. English: The system could be qualified as reasonable and good. The system offers a lot of opportunities. For questionable changes. It would be useful/convenient/handy if the other departments were automated too, so some manual procedures/tasks/activities could be skipped/deleted. The system or the computers are still to slow to process the required information/requests/orders. Dutch : Het systeem is redelijk tot goed te noemen.hte systeem biedt teveel mogelijkheden voor abusievelijke veranderingen. Het zou handig zijn indien ook de overige afdelingen geautomatiseerd waren, waardoor bepaalde handmatige werkzaamheden konen vervallen. Het systeem of de computers zijn nog traag in de verwerking van de gegevens en/ of opdrachten

16. English: The system is not always user-friendly if it comes/is regarding my/ to my specific task of payroll administrator. For sure it need some modifications to make my job easier. Dutch : Het systeem is t.a.v. mijn specifieke werkzaamheden van honorarium niet al te vriendelijk. Het behoeft zeker nog enkele aanpassingen om mijn werk te kunnen vergemakkelijken


102

17. English: The system could be reliable and up to date and perform well if all users follows the right procedures when using the system. That’s why refreshment training is needed from time to time/regularly. Dutch : Het systeem kan betrouwbaar en up to date zijn en ook goed werken wanneer alle gebruikers de juiste Handelingen plegen tijdens het gebruiken van het systeem. Daarom is van tijd tot tijd een refreshment Nodig

18. English: It was not so clear to me which ISs were meant Dutch : Het was voor mij niet zo duidelijk over welke geautomatiseerde systemen het ging

19. English: The IS performs pretty good/well Dutch : Het werkt redelijk goed

20. English: The more extensive/ the more comprehensive the better the information. Dutch : Hoe uitgebreider hoe beter en meer info.

21. English: I use the system daily Dutch : ik ben elkedag gebruiker v/d geaut.info syst(en)

22. English: Unfortunately I am not a user of one of the ISs, therefore I cannot give an opinion about the ISs. My perception is that based on the results/output of the ISs which I receive from users to fulfill my duties. Dutch : Ik ben helaas geen gebruiker van een der informatie systemen en kan derhalve geen oordeel hieromtrent vellen mijn perceptie is veelal gebaseerd op de resultaten/ output van de informatie systemen die ik van gebruikers tbv mijn werkzaamheden ontvang

23. English: I consider ISs in hospitals as very efficient, in particular for information and PR purposes Dutch : Ik vind de geautomatiseerde info. systemen in zieken huizen zeer efficiënt, vooral als het gaat om Informatie en PR

24. English: In our hospital Navision is used daily. Access/licensing is not enough. That keeps things waiting/ “It delays the process somewhat. Dutch : ln ons ziekenhuis wordt er dagelijks gewerkt in Navision. Er is weinig licentie. Dat houdt wel een beetje op.

25. English: IS is a usual tool. It could be good if this tool could be teach from/starting at primary/elementary school Dutch : Informatica is een alledaags gebruikte hulpmiddel het zou ook goed zijn als dit hulpmiddel ook vanaf de basis school gegeven zouden worden

26. English: Information system is not so good when comparing to former system. Patient information can not be displayed so you have to make phone calls to ask for explanation/clarification. Dutch : Informatie systeem vind ik niet zo goed in vergelijking met het vorige systeem. informatie van patiënten zijn ook niet echt te zien waardoor je toch telefonisch contact moet leggen met andere afdelingen

27. English: Font is to small; scale of 1 -7 is not logical; questions are varying/different; definition of IS is not clear. Dutch : Letter gr te klein categorisering 1-7 onlogisch, vraagstelling veranderend,definitiering info syst niet helder

28. English: More benefits could be generated if everyone use the system in the right way and record data accurate Dutch : Men kan meer profijt ontvangen van het systeem, wanneer iedereen correct en secuur invoert

29. English : Users of Navision should be informed regularly about new developments concerning Navison. Dutch : Men moet het personeel die met navision werken steeds meer informeren over nieuwe ontwikkelingen in Navision

30. English: Mister to be honest, your assessment is unrealistic. Dutch : Meneer als ik eerlijk mag zijn vind ik uw waardering onrealistisch.

31. English: Not everyone has the same authorization to access the information system. Dutch : Niet iedereen heeft de zelfde rechten om toegang te krijgen in het informatie systeem.

32. English: Sometime there is no access to the system because there are more colleagues logged in. they keep you waiting too long before you get access to the system Dutch : Soms krijg je geen toegang tot het systeem want je wordt gegrendeld door je mede collega's. men laat je lang wachten voordat je licentie krijgt

33. English: I disagree with the description of the different subjects/points/variables but I agree with the other parts of the questionnaire. Dutch : Uw punten beschrijving vind ik juist niet op z'n plaats, maar betreffende het overige ben ik wel mee eens.

34. English: I consider it a good development in the communication but the system is not always online/ready, slow and difficult to find/search for particular parts Dutch : Vind het een goede ontwikkeling in de communicatie het systeem werkt echter niet altijd; langzaam, moeilijk om bepaalde onderdelen te vinden

35. English: Former computer system was easier in use/to work with. Actual computer system is down more often so work delays; in stead of the statement “neutral” the statement “other opinion” could be used/stated. Dutch : Het vorig computersysteem was veel makkelijker; het huidig computer systeem valt vaker uit waardoor 't werk niet vlot verloopt; ipv de bewering "neutraal" zou de bewering "andere mening" kunnen worden geplaatst

Source: own research and hospitals in study


103

APPENDIX 7: Hypothesized relationships and correlation

H0 1a : System Quality is not related to IS Benefits of Use

HA 1a : System Quality is positively related to IS Benefits of Use

Results show a significant and medium16 correlation between System Quality and Benefit of Use, r =

.450; p = .000. Therefore Hypothesis 2 (H0 1a) can be rejected.

H0 1b : System Quality is not related to User Satisfaction

HA 1b : System Quality is positively related to IS User Satisfaction Results show a significant and large correlation between System Quality and User Satisfaction, r = .585;

p = .000. Therefore Hypothesis 2 (H0 1b) can be rejected.

H0 2a : Information Quality is not related to IS Benefit of use

HA 2a : Information Quality is positively related to IS Benefits of Use Results show a significant and medium correlation between Information Quality and IS Benefit of Use, r

= .476; p = .000. Therefore Hypothesis 2 (H0 2a) can be rejected.

H0 2b : Information Quality is not related to IS User satisfaction

HA 2b : Information Quality is positively related to IS user satisfaction Results show a significant and large correlation between Information Quality and User Satisfaction, r =

.572; p = .000. Therefore Hypothesis 2 (H0 2b) can be rejected.

H0 3a : Service Quality is not related to IS Benefit of Use

HA 3a : Service Quality is positively related to IS Benefit of Use Results show a significant and small correlation between Service Quality and IS Benefit of Use, r =


H0 3b : Service Quality is not related to IS user satisfaction

HA 3b : Service Quality is positively related to IS user satisfaction Results show a significant and small correlation between Service Quality and IS user satisfaction, r =

.283; p = .002. Therefore Hypothesis 2 (H0 3b) can be rejected. 16 Correlation strengths: small: r .10 - .29; medium r .30 - .49; large r .50 – 1.00 (Pallant, 2007)


104

H0 4a : User Satisfaction is not related to users’ Intention to Use

HA 4a : User Satisfaction is positively related to users’ Intention to Use

Results show a significant and small correlation between User satisfaction and user’ intention to use, r =


H0 4b : User Satisfaction is not related to Net Benefits

HA 4b-c: User Satisfaction is positively related to Net Benefits

Results show a significant and large correlation between User satisfaction and Net Benefits, r = .552; p

= .000. Therefore Hypothesis 2 (H0 4b) can be rejected.

H0 5a : Benefit of Use is not related to User Satisfaction

HA 5a : Benefit of Use is positively related to User Satisfaction

Results show a significant and large correlation between benefits of use and user satisfaction, r = .645; p

= .000. Therefore Hypothesis 2 (H0 5a) can be rejected.

H0 5b : Benefit of Use is not related to Net Benefits

HA 5b-d: Benefit of Use is positively related to Net Benefits

Results show a significant and large correlation between Benefits of use and Net Benefits, r = .560; p =

.000. Therefore Hypothesis 2 (H0 5b) can be rejected.

H0 6a : Perceived Ease of Use is not related to Intention to Use IS HA 6a : Perceived Ease of Use is positively related to Intention to Use IS Results show a significant and large correlation between perceived ease of use and intention to use, r =


H0 6b : Perceived Ease of Use is not related to Perceived Usefulness. HA 6b : Perceived Ease of Use is positively related to Perceived Usefulness. Results show a significant and medium correlation between perceived ease of use and perceived

usefulness, r = .415; p = .000. Therefore Hypothesis 2 (H0 6b) can be rejected.

H0 7 : Perceived Usefulness is not related Intention to Use IS

HA 7 : Perceived Usefulness is positively related Intention to Use IS

Results show a significant and medium correlation between perceived usefulness and intention to use, r

= .404; p = .000. Therefore Hypothesis 2 (H0 7) can be rejected.


105

H0 8 : Intention to Use is not related to Benefits of Use

HA 8 : Intention to Use is positively related to Benefits of Use



H0 9 : Net Benefits of hospitals is not related to Intention of Use

HA 9 : Net Benefits of hospitals is positively related to Intention of Use





106

APPENDIX 8: Overview Alternative Hypothesis and testing results

Code Alternative Hypothesis Test Result p-value

H1a System Quality is positively

related to IS Benefits of Use Pearson Accept r = .450

p = .000

H1b System Quality is positively

related to IS User Satisfaction Pearson Accept r = .585

p = .000

H2a Information Quality is positively


p =.000

H2b Information Quality is positively


p = .000

H3a Service Quality is positively


p =.016

H3b Service Quality is positively


p =.002

H4a User Satisfaction is positively

related to users’ Intention to Use Pearson Accept r = .204

p = .024

H4b-c User Satisfaction is positively

related to Net Benefits hospitals Pearson Accept r = .552

p = .000

H5a Benefit of Use is positively

related to User Satisfaction

Pearson Accept r = .645

p = .000

H5b-d Benefit of Use is positively

related to Net Benefits hospitals


p = .000

H6a Perceived Ease of Use is

positively related to Intention to

Use IS


p = .000


107

H6b Perceived Ease of Use is

positively related to Perceived

Usefulness.


p = .000

H7 Perceived Usefulness is

positively related Intention to

Use IS


p = .000

H8 Intention to Use is positively

related to Benefits of Use Pearson Accept r = .381

p = .000

H9 Net Benefits of hospitals is

positively related to Intention of

Use


p = .000

Correlation strengths: small: r .10 - .29; medium r .30 - .49; large r .50 – 1.00 (Pallant, 2007)



108

APPENDIX 9: Descriptive statistics

System Quality

Information Quality

Service Quality

Perceived Ease of Use

Perceived Usefulness

Valid 98 98 98 96 97N Missing 1 1 1 3 2

Mean 4.3622 4.4466 4.9286 4.7595 5.3567Median 4.4286 4.3333 4.8000 4.7500 5.2000Mode 5.00 5.00 4.00 5.00 5.00Std. Deviation .83897 1.01454 1.03446 .98318 .87427Minimum 2.43 1.83 2.60 3.00 3.00Maximum 7.00 7.00 7.00 7.00 7.00

Statistics

Intention to

Use User

Satisfaction Benefit of Use Net Benefits Use USE_high N Valid 94 98 95 94 97 97 Missing 5 1 4 5 2 2Mean 4.6793 4.5557 4.6663 4.8333 4.1048 4.5309Median 4.9000 4.5903 4.6000 4.8333 4.2500 4.5000Mode 5.00 5.00 5.00 5.00 4.00 5.00Std. Deviation 1.03400 .82484 .83112 .87171 .89852 1.40834Minimum 1.40 2.33 2.40 2.00 1.75 1.00Maximum 7.00 7.00 7.00 7.00 6.75 7.00



109

APPENDIX 10: Overall results final tests Structural Equation Model

Correlations 1

System Quality

Information

Quality Service Quality Benefit of Use

Pearson Correlation 1 .631** .369** .450**

Sig. (1-tailed) .000 .000 .000

System Quality

N 98 98 98 95

Pearson Correlation .631** 1 .332** .476**

Sig. (1-tailed) .000 .000 .000

Information Quality

N 98 98 98 95

Pearson Correlation .369** .332** 1 .220*

Sig. (1-tailed) .000 .000 .016

Service Quality

N 98 98 98 95

Pearson Correlation .450** .476** .220* 1

Sig. (1-tailed) .000 .000 .016 Benefit of Use

N 95 95 95 95

**. Correlation is significant at the 0.01 level (1-tailed).

*. Correlation is significant at the 0.05 level (1-tailed).

Correlations 2 System

Quality Information

Quality Service Quality

User Satisfaction

Pearson Correlation

1 .631** .369** .585**

Sig. (1-tailed) .000 .000 .000

System Quality

N 98 98 98 98

Pearson Correlation

.631** 1 .332** .572**

Sig. (1-tailed) .000 .000 .000

Information Quality

N 98 98 98 98

Pearson Correlation

.369** .332** 1 .283**

Sig. (1-tailed) .000 .000 .002

Service Quality

N 98 98 98 98


110

Pearson Correlation

.585** .572** .283** 1

Sig. (1-tailed) .000 .000 .002

User Satisfaction

N 98 98 98 98


Correlations 3

User Satisfaction Intention to Use Net Benefits

Pearson Correlation 1 .204* .552**

Sig. (1-tailed) .024 .000

User Satisfaction

N 98 94 94

Pearson Correlation .204* 1 .387**

Sig. (1-tailed) .024 .000

Intention to Use

N 94 94 91

Pearson Correlation .552** .387** 1

Sig. (1-tailed) .000 .000 Net Benefits

N 94 91 94



Correlations 4

Use User Satisfaction Net Benefits

Pearson Correlation 1 .243** .178*

Sig. (1-tailed) .008 .044

Use

N 97 97 93

Pearson Correlation .243** 1 .552**

Sig. (1-tailed) .008 .000

User Satisfaction

N 97 98 94

Pearson Correlation .178* .552** 1


N 93 94 94




111

Correlations 5

USE_high User Satisfaction Net Benefits


Sig. (1-tailed) .001 .028

USE_high

N 97 97 93


Sig. (1-tailed) .001 .000

User Satisfaction

N 97 98 94



N 93 94 94



Correlations 6

Perceived Ease

of Use

Perceived

Usefulness Intention to Use

Pearson Correlation 1 .660** .415**

Sig. (1-tailed) .000 .000

Perceived Ease of Use

N 96 96 93


Sig. (1-tailed) .000 .000

Perceived Usefulness

N 96 97 94

Pearson Correlation .415** .404** 1

Sig. (1-tailed) .000 .000 Intention to Use

N 93 94 94



112

Correlations 7

Intention to Use Benefit of Use User Satisfaction


Sig. (1-tailed) .000 .024

Intention to Use

N 94 92 94


Sig. (1-tailed) .000 .000

Benefit of Use

N 92 95 95


Sig. (1-tailed) .024 .000 User Satisfaction

N 94 95 98




Correlations 8

Benefit of Use Net Benefits

Pearson Correlation 1 .560**

Sig. (1-tailed) .000

Benefit of Use

N 95 94

Pearson Correlation .560** 1

Sig. (1-tailed) .000 Net Benefits

N 94 94



113

APPENDIX 11: Indication of distribution of spending by payer

Source: Ministry of Health; Suriname Health System Profile 2006 for MPH 1105


114

APPENDIX 12: Indication of Health Status and Outcomes Vital Statistics

Source: Ministry of Health; Suriname Health System Profile2006 for MPH 1105


115

APPENDIX 13: Direct, indirect and total effects of constructs

Direct Effects Indirect Effects Total Effects

Construct BU US NB BU US NB BU US NB

SQ 0.450 0.585 - - 0.290 0.575 0.450 0.875 0.575

IQ 0.476 0.572 - - 0.307 0.583 0.476 0.897 0.583

SRQ 0.220 0.283 - - 0.142 0.279 0.220 0.425 0.297

BU - 0.645 0.560 - - 0.356 - 0.645 0.916

US - - 0.552 - - - - - 0.552



116

APPENDIX 14: Research Model with hypotheses

H3b

H1b

H1a

H2aH2b

H3a

H9

4. Other Measures of Net Benefits of IS Use

2. General Perceptual Measure of Net Benefits of IS Systems

Benefit of Use

Benefit of Use

UserSatisfaction

UserSatisfaction

Net BenefitsNet Benefits

H6b

H6a H7


Perceived Ease of UsePerceived

Ease of UsePerceivedUsefulnessPerceivedUsefulness

Intention to Use

Intention to Use

1. Measures of ThreeQuality Dimensions

System QualitySystem Quality

InformationQuality

InformationQuality

ServiceQualityServiceQuality

H5a

H4aH8

H5b-d

H4b-c

H3b

H1b

H1a

H2aH2b

H3a

H9

4. Other Measures of Net Benefits of IS Use

2. General Perceptual Measure of Net Benefits of IS Systems

Benefit of Use

Benefit of Use

UserSatisfaction

UserSatisfaction

Net BenefitsNet Benefits

H6b

H6a H7




Intention to Use

Intention to Use




Intention to Use

Intention to Use

1. Measures of ThreeQuality Dimensions

System QualitySystem Quality

InformationQuality

InformationQuality

ServiceQualityServiceQuality

H5a

H4aH8

H5b-d

H4b-c

Source: Previous studies and own research

Documents

_organizational_performance_in_hospitals_in_Suriname._A_test_of_an_extended_version_of_DeLone_and_McLean_s_IS_Success_Model_2003_by_DeLano_Gefferie_MBAIV