www.homeless.org/culture: Analyzing the Relationship between

Organizational Culture and HMIS Use among Homeless Service Providers

December 2009 By Courtney Cronley, PhD

UNIVERSITY OF TENNESSEE COLLEGE OF SOCIAL WORK OFFICE OF RESEARCH AND PUBLIC SERVICE

2

Acknowledgements

This study was funded by a Doctoral Dissertation Research Grant from the

United States Department of Housing and Urban Development’s Office of University

Partnerships. The author would like to thank the following groups for their participation

in and cooperation with this research: the East Tennessee Coalition to End Chronic

Homelessness; the Michigan Coalition Against Homelessness; Holland/Ottawa County

Continuum of Care (CoC); Out-Wayne Counties CoC; and Pontiac/Oakland County

CoC.

Contact

The author can be reached at ccronle1@utk.edu to answer any questions

regarding the content of this report. A full description of the methodology and results is

available through the dissertation abstracts in the Hodges Library at the University of

Tennessee, www.lib.utk.edu.

3

Table of Contents

Executive Summary…………………………………………………………….……………….4

Research Report

Introduction…………………………………………………………………………........5

Background…………………………………………………………………..…………..5

Research Questions…………………………………………...………………………..7

Design……………………………………………………………………………………. 7

Results…………………………………………………………………………………… 9

HMIS Use………………………………………………………………………... 9

Organizational Culture…………………………………………………………12

HMIS Use and Organizational Culture……………………………………… 15

Limitations………………………...…………………………………………………………….17

Summary and Recommendations.…………………………………………………………. 17

References……………………………………………………………………………………...20

4

Executive Summary

The United States Department of Housing and Urban Development (HUD) has

required federally-funded homeless service providers to participate in their homeless

management information systems (HMIS) since 1999. As of now, though, no one has

examined how and to what extent these technologies are being used. Theory and

research suggest that the technology dissemination is contingent upon the

organizational culture in which new resources are being used. This study represents the

first empirical analysis of HMIS use and explores the cross-level relationship between

staff members’ HMIS use and organizational culture.

Staff members at 26 homeless service providers completed the Organizational

Social Context (OSC) survey. Individual scores were aggregated to determine the

organizational culture of each organization. Data on HMIS use, measured as the

number of times that an individual attempted to log on to the system, were collected

from 142 individuals.

Results suggest that organizational proficiency is related to HMIS use and is

moderated by gender. The rate of log on attempts for male staff members increases in

organizations with higher levels of proficiency. Moreover, organizational culture results

revealed that the sample reported significantly higher levels of organizational

proficiency, rigidity, and resistance, compared to a national sample of children’s mental

health providers. The study concludes with the recommendation that policy makers view

HMIS implementation as an ongoing, cyclical process of interactions among the

organizational social context, the software, and the researchers developing the

technology.

5

Introduction

This study represents the first attempt to measure empirically the use of

homeless management information systems (HMIS). It stems from the Department of

Housing and Urban Development’s (HUD) 1999 mandate requiring all federally funded

homeless service providers to participate in an HMIS (HUD, 2008). To date only one

study has assessed HMIS implementation (Gutierrez & Friedman, 2005). This study

was not empirical, however. It was based on the authors’ observations and self-reports

from staff members at the organizations. Considering the funding that HUD has

allocated for this project and the expected implications of its use, it is critical that we

conduct systematic and objective assessments of the extent to which HMIS are being

used by organizations.

Background

Research demonstrates that use of new technologies, such as information

management systems and electronic referral systems, in human services can

significantly improve service provision and client outcomes (McCoy & Vila, 2002).

Studies show, though, that new technologies frequently are under or mis-used in

organizations (Carrillo, 2005, 2007; O’Looney, 2004; McCoy & Vila; Herie & Martin,

2002). Reasons for this include poorly designed technology, limited technical

competence, and leadership that do not support change (Carrilio, Packard, & Clapp,

2003).

This study considered how frequently homeless service providers were using

their HMIS and what factors were related to use. It relied on three theories to explain

6

technology use in organizations: diffusion of innovations; sociotechnical theory; and

organizational effectiveness. Table 1 describes each of the theories.

Table 1. Theory base

Theory Question

Diffusion of Innovations (Rogers, 2003) How do new ideas spread among people?

Sociotechnical theory of organizational effectiveness (Trist & Bamforth, 1954)

What is the relationship between technology and the social context in which it is used?

Organizational culture theory (Schein, 1992)

Do shared values, beliefs, and expectations develop in the organization’s social context and guide employee behavior?

Both the theories of diffusion of innovation and sociotechnical organizational

effectiveness consider how new ideas are adopted and what makes their adoption

successful. Diffusion of innovations argues that new ideas, such as the use of cell

phones, spread among people through social networks. One person tells another, who

tells another, etc. Sociotechnical theory argues that the use of new technology,

specifically in organizations, relies on a goodness of fit between the social context and

the technology. Leadership and work practices must accommodate a new technology

for it to be used, regardless of its effectiveness.

Organizational culture theory provides a conceptual bridge between diffusion of

innovations and sociotechnical theory. People in organizations behave according to

established values and expectations as well as shared history and leadership. New

ideas move into organizations to the degree that organizational leaders are aware of

and introduce the innovations. Staff members in organizations whose cultures support

these innovations, those that value proficiency and are willing to take risks with new

7

ideas, are most likely to use new technologies. Table 2 identifies the three main

constructs of organizational culture that were measured in this study: rigidity,

proficiency, and resistance.

Table 2. Organizational culture constructs

Construct Definition

Rigidity The degree to which organizations observe set policies and procedures for work processes.

Proficiency The degree to which organizations value staff competency and strive to provide the best possible services to clients.

Resistance The degree to which organizations invite change.

Research Questions

The study sought to answer the following two questions:

1) Does organizational culture influences staff members’ HMIS use?

2) Do individual characteristics (e.g. gender) interact with organizational culture

to influence staff members’ HMIS use?

Design

Four Continua of Care (CoC) in two states, Michigan and Tennessee,

participated in the study. Figure 1 shows the geographic distribution of the data

collection. A total of 26 homeless service providers and 142 staff members were

sampled. Data were collected in two-waves, 1) January – May, 2008, and 2) January –

May, 2009. Two variables were measured: organizational culture and HMIS use.

Organizational culture was assessed as an organizational-level variable, meaning that

each organization had a single score. HMIS use was assessed as an individual-level

variable, meaning that each individual had a different score. Figure 2 shows the

8

Figure 1. Distribution of data collection sites. The blue dots represent participating organizations.

structure of the sample. The analysis involved examining the relationship between

organizational culture scores and staff members’ HMIS scores. Data were collected in

two waves, 1) January – May, 2008, and 2) January – May, 2009. Two variables were

measured: organizational culture and HMIS use. Organizational culture was assessed

as an organizational-level variable, meaning that each organization had a single score.

HMIS use was assessed as an individual-level variable, meaning that each individual

had a different score. Figure 2 shows the structure of the sample. The analysis involved

examining the relationship between organizational culture scores and staff members’

HMIS scores.

The purpose was to determine if an organization’s culture score influenced how

frequently a staff member used an HMIS. Would staff members in organizations that

scored high on organizational proficiency be more likely to use the HMIS than staff

members in organizations scoring low or average? The study also assessed the

9

Nested Sample

CoC 1 CoC 2

CoC 3 CoC 4

8 Organizations4 Organizations

5 Organizations 7 Organizations

34 HMIS Users

44 HMIS Users 51 HMIS Users

13 HMIS Users

Figure 2. The sample consisted of 142 HMIS users, nested within 26 organizations, nested within four CoC. Each organization had an organizational culture score, and each staff member within each organization had an HMIS use score.

variables separately. It examined organizational culture across the sample to see if it

varied substantially among providers. In addition, the study considered how frequently

staff members were attempting to use the HMIS and if this frequency was similar across

organizations.

Results

HMIS – Variability in Use

The median number of times that staff members attempted to log on to the HMIS

was 47 times per year. Results suggest, however, that there is a large amount of variety

in how the organizations surveyed are using their HMIS. The number of times that staff

members attempted to log on to an HMIS ranged from 2 to 719 in one year. Looking at

use by organizations, one can group them into three main categories: 1) regular,

10

proportionate use; 2) irregular use; and 3) irregular, disproportionate use. Select

organizations reflecting these profiles are shown in Figures 3, 4, and 5. The user ID, on

the right, vertical axis, denotes individual staff members, in the organization. The

months are identified on the horizontal axis, beginning with 2 for February. The count,

on the left, vertical axis, denotes the number of times each month the staff member

attempted to log on to an HMIS.

Figure 3. Regular, proportionate use: A large number of staff members licensed to use the HMIS and many of them attempting to log on to the HMIS at least once every month.

11

Figure 4. Irregular use: A small number of staff members licensed to use the HMIS and attempting to log on infrequently and inconsistently.

Figure 5. Irregular, disproportionate use: A single staff member licensed to use the HMIS and attempting to log on inconsistently.

12

There was also variability among the CoC in organizational use of HMIS.

Aggregate organizational use of an HMIS ranged from a median of 33 in CoC 2 to a

median of 337 in CoC 3. Figure 6 shows the total number of times that staff members

attempted to log on to an HMIS in each CoC.

Figure 6. Pie graph showing the total number of times that staff members attempted to log on to the HMIS, by CoC. Results show great disparity in attempts from a low of 817 times in CoC 1 to a high of 6,106 times in CoC 4.

Organizational Culture – Variability across Organizations

Organizational culture results are reported as T-scores derived from a national

sample of children’s mental health providers (n = 100). A T-score of 50 means that an

organization’s culture shows average levels of proficiency, rigidity, and resistance

compared to the national sample. One standard deviation is 10 points higher or lower

than 50. Figure 7 shows that the homeless service providers in this study are markedly

different from the national sample. These organizations report average levels of rigidity

and resistance that are more than a full standard deviation above the mean (M = 60.39

13

60.39

58.11

64.11

20

30

40

50

60

70

80

Rigidity Proficiency Resistance

T S

co

re

Figure 7. A graph comparing the sample of homeless service providers (in blue) to a national sample of children’s mental health providers. It shows that the sample is markedly different from the national sample.

and 64.11, respectively) and a proficiency level that is nearly one standard deviation

above the mean (M = 58.11).

Among the homeless service providers, however, there is a great deal of

variability, particularly in proficiency scores, which ranged from a T-score of 36.30 to

71.07. Figure 8 show the two extreme organizational profiles found in the study. The

model organization shows a high proficiency score and low resistance score. The least

constructive organization shows the inverse relationship with a low proficiency score

and high resistance score.

14

50.6061.10

55.10

20

30

40

50

60

70

80

Rigidity Proficiency Resistance

T S

co

re

73.11

48.13

64.27

20

30

40

50

60

70

80

Rigidity Proficiency Resistance

T S

co

re

Figure 8. Examples of extreme organizational culture profiles. The organization on the left shows a model organization for disseminating innovations. The level of resistance is low, and the proficiency is high. The organization on the right shows a challenging site for innovation. Levels or rigidity and resistance are high while expectations of worker competency, proficiency, are low. Results also showed differences in organizational culture profiles among CoC.

Figure 9 graphs the average T-scores of all organizations in each of the four CoC.

There was an 8.33 point difference between CoC 3, in which organizations reported the

highest average rigidity score (M = 63.86) and CoC 2, where organizations showed the

20

30

40

50

60

70

80

Rigidity Proficiency Resistance

T S

co

re

Figure 9. The figure above compares average organizational culture profiles for all four CoC graphed against the normative sample of children’s mental health providers.

15

lowest average rigidity score (M = 55.53). Similarly, there was an 8.54 point difference

in average resistance scores between the highest in CoC 3 (M = 67.88) and the lowest

in CoC 2 (M = 59.34).

HMIS Use and Organizational Culture

Figures 10, 11, and 12 compare select organizations’ HMIS use with their

organizational culture profiles. The blue line indicates the select organization. The dark

green line represents the sample average.

60.56

59.2353.89

20

30

40

50

60

70

80

Rigidity Proficiency Resistance

T S

co

re

Figure 10. This organization show regular, proportionate use of the HMIS. Compared to the sample average (green line), the organization reports lower levels of rigidity and resistance (blue line) and a higher level of proficiency.

65.72

57.6953.38

20

30

40

50

60

70

80

Rigidity Proficiency Resistance

T S

co

re

Figure 11. The organization above shows irregular use of the HMIS. One staff member never uses the HMIS and other staff members do not access the system for entire months at a time. The organizational profile shows resistance that is almost two standard deviations above the mean.

16

64.11

55.54

69.90

20

30

40

50

60

70

80

Rigidity Proficiency Resistance

T S

co

re

Figure 12. A single staff member uses the HMIS, suggesting disproportionate use, and the use is sporadic and irregular, swinging between a low of less than 10 times a month to a high of nearly 40 times. The corresponding organizational culture profile shows a level of resistance that is nearly two standard deviations above the mean. Results also showed that one component of organizational culture, proficiency,

affected staff members’ use of an HMIS, although this effect was moderated by gender.

Specifically, in organizations with high levels of proficiency, men were more likely to use

an HMIS. Figure 13 shows this interaction between gender and proficiency.

48.13 52.91 57.69 62.47 67.253.25

6.53

9.82

13.10

16.38

Proficiency

Lo

g o

n A

ttem

pts

Female (0)

Male (1)

Figure 13. Men’s use of an HMIS increases as organizational proficiency increases, while women’s use is unaffected. Interestingly, the increase for men doesn’t occur until organizations show markedly higher levels of proficiency compared to the average organization.

17

Limitations

This study shared many of the limitations common to studies of organizations.

Every effort was made, however, to minimize the effects of limitations to the accuracy of

the results and interpretation. Primary limitations include:

Sample – the small, purposive sample limits the results to the organizations and

communities included in the study.

Measurement – the study relied on log on attempts to measure HMIS use. This

measurement may not measure the variety or the substantive ways in which staff

members interact with an HMIS.

Data collection – the study did not collect individual-level data about staff

members’ familiarity with technology. Factors such as education, job title, and

technical training may affect an individual’s use of an HMIS, above and beyond

the organizational culture.

Summary and Recommendations

Results from this study can be summarized in Figure 14. Dissemination of the

HMIS into homeless service providers has been an ongoing, cyclical process moving

from initial efforts by HUD to disseminate the technology to service providers adopting

it, to front-line practitioners who are expected to use the system in daily activities. HUD

designed HMIS to act as real-time data bases that case managers use as they serve

clients. As organizations adopt the technology and staff members use it, this research

shows that not all organizations are using the data base in the real-time. Possible

explanations for this finding may be that organizations are maintaining dual record

keeping systems with paper assessments that they transfer to an HMIS. Alternatively,

18

Figure 14. The diagram shows the dynamic interplay among different factors that contribute to dissemination of innovations into organizations.

they may be allocating HMIS work to a single staff member who receives data from

other staff members and enters everything en masse.

Policy makers and researchers have a responsibility to assess the

implementation and consider ways in which the technology or the methods of

dissemination should be altered to maximize successful use of HMIS as tools of

practice for service providers.

The results of the study point to two major policy area recommendations:

1) Organizational culture influences how organizations are able to adopt and

implement an HMIS.

a. Implement organizational culture change interventions aimed at reducing

resistance and rigidity while emphasizing proficiency.

b. Provide long-term funding that supports technical assistance for the

organizational change.

19

c. Act patiently. Organizational change and full implementation of technology

are long-term, non-linear processes that face numerous setbacks and

challenges.

2) Organizations are using the HMIS differently and often in ways contrary to the

original design.

a. Provide organizational specific training that supports the unique service

environments of each provider.

b. Monitor implementation through more implementation evaluations to

ensure fidelity to the original design.

20

References

Carrilio, T. (2005). Management information systems: Why are they underutilized in the social services? Administration in Social Work, 29(2), 452-462.

Carrilio, T. (2007). Using client information systems in practice settings: Factors affecting social workers’ use of information systems. Journal of Technology in Human Services, 25(4), 41-62.

Carrilio, T. E., Packard, T., & Clapp, J. D. (2003). Nothing in – nothing out: Barriers to the use of performance data in social service programs. Administration in Social Work, 27(4), 61 – 75. Gutierrez, O. & Friedman, D. H. (2005). Managing project expectations in human services information systems implementations: The case of homeless management information systems. International Journal of Project Management, 23. Herie, M. & Martin, G. W. (2002). Knowledge diffusion in social work: A new approach

to bridging the gap. Social Work, 47(1), 85-95. McCoy, H. V. & Vila, C. K. (2002). Tech knowledge: Introducing computers for coordinated care. Health and Social Work, 27(1), 71-74. O’Looney, J. (2005). Social work and the new semantic information revolution.

Administration in Social Work, 29(4), 5-34. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press. Schein, E. (1992). Organizational culture and leadership. San Francisco: Jossey-Bass. Trist, E. L. & Bamforth, K. W. (1951). Some social and psychological consequences of the longwall method of coal-getting. Human Relations, 4, 3-38. The United States Department of Housing and Urban Development. (2007). The Annual

Homeless Assessment Report to Congress. Washington D.C.: Author.

The University of Tennessee College of Social Work 224 Henson Hall Knoxville, TN 37966-3333 865-974-3176 www.csw.utk.edu