Embed Size (px)
Citation preview
June 2003 George Mason University 1
Needs Assessment
Farrokh Alemi, Ph.D.
June 2003 George Mason University 2
Key Reason for Failure of IT Investment “I can’t get the report I need?” “What is this for?” “This is not what I do.”
Why are so manyinformation systems
unused?
June 2003 George Mason University 3
Some Recent Student Postings
“... if I could determine WHY the information is needed, I could better assist in determining WHAT information would be needed …”
“ … we could pinpoint what people want, need, and desire …”
“…why do you need this is how are you going to use it comes into play…”
“…you must be familiar with the changing trends of that industry…”
June 2003 George Mason University 4
Two Common Complaints
Not the right information Not at the right time
Systems that do notmeet our needs
June 2003 George Mason University 5
Why is it so hard to design systems that meet our needs? A well-defined set of requirements does not
exist The organizational leaders are changing The analysts are unable to elicit
requirements Future is not now
– The business process is changing– The IT industry is changing
June 2003 George Mason University 6
What you need versus what you want Systems are designed for organizations not
individuals Systems should help individuals see beyond
their own limitations Systems should anticipate the future
June 2003 George Mason University 7
How should we assess information needs?
June 2003 George Mason University 8
Previous Approaches
Analyze organizational tasks and see how information is used in these tasks
Ask the decision-maker about their needs Derive requirements from the existing
systems Look at strategic goals and concerns Do input-process-output analysis
June 2003 George Mason University 9
Problems with Existing Approaches Focus on past but not future needs Focus on a single issue, task, or decision Focus on information not decisions Focus on the decision-makers’ personal
goals, which may be inconsistent with the organizational goals
Focus on users’ behavior without seeking decision maker’s insight
June 2003 George Mason University 10
Our Proposed Strategy
Step Objective Performed by1 Identify issues Panel of experts and
organizational leaders2 List information needs of
top issuesPanel of experts and organizational leaders
3 Create a taxonomy Systems analysis team4 Prioritize information items Panel of organizational leaders
5 Develop resource allocation plan for data collection and analysis strategies
Systems analysis team
June 2003 George Mason University 11
Key Advantages Focus on future reduces possibility of identifying
information no longer needed “Decision driven" methodology minimizes the
potential for collecting data that may not be used By relying on a group rather than an individual,
we minimize the design of the system around idiosyncratic characteristics of a few people
By adding external experts to the membership of the group, the methodology emphasizes what organization ought to need
June 2003 George Mason University 12
Example: 3 out of 6 decisions identified1. How to reimburse the growing number of
paraprofessionals who treat mental health clients? 2. What educational programs or policies should the
state adopt to upgrade knowledge of mental illness among primary care physicians who deliver many services to rural, mentally ill clients?
3. Many chronically mentally ill patients live in nursing homes that cannot provide adequate care. What policies should the state adopt to improve care for these patients?
June 2003 George Mason University 13
Step two: 2 out of 69 items identified as needed
Item number
Item description
1 The size and character of the patient population affected by the issue
5 How expensive is it to provide services that fully meet clients’ needs?
June 2003 George Mason University 14
Step 3: Rate the information in context of specific decisions For the issue "xxx," assign
a score between 1 and 3 to each information item reflecting the need for the information. A score of 1 means the information has low priority, a score of 2 means that the information is nice to have but not essential and a score of 3 means that the item is essential for addressing the issue.
Each commissioner rated the relevance of all 69-information items for their decisions
June 2003 George Mason University 15
Set Data Collection Priorities
Essential items are important to all issues
Rapid collection items are important to most issues; need no action but plans to collect the information if the need arises
Periodic collection items include items important on some issues
Low priority items are ignored
June 2003 George Mason University 16
Data Collection Plans and Analysis of Ratings
Small range of ratings across issues
Wide range of ratings across issues
High average rating of importance
Essential information. Collect now.
Rapid data collection set. Plan data collection now.
Low average rating of importance
Low priority items Periodic data set. Collect when needed
June 2003 George Mason University 17
Examples of Essential & Rapid Data collection items
Item number Item description Item number
Description of the item
5 How expensive is it to provide services that fully meet clients’ needs?
1 The size and character of the patient population
13 The number, type, and distribution of services provided
42 The organization of the existing system
45 The monitoring and evaluation methods used with the system
47 Options to the existing system
53 The costs of implementing alternatives to the existing system
61 Legislative and regulatory changes needed to alter the system
Essential information Rapid Data Collection Set
June 2003 George Mason University 18
Examples of periodic and ignored information sets
Item number Description of the information item
Item number
Description of the information item
7 Who gets what priority for different types of care
4 Episodes of illness: frequency and responsiveness to treatment
26 How different providers serve the undeserved areas
9 The role of the family in caring for client
28 Constraints specifically limiting substitutability of providers
46 What knowledge is needed to correct the problem
29 The existing provider certification and education systems
58 How existing laws and regulations duplicate each other
31 Training that might be needed by families and the community
64 The position, attitude, and agenda of affected interest groups
35 Possible funding sources and methods of obtaining funds
56 Legislative/regulatory constraints on the existing/optional systems
57 How legislation defines responsibility for the client population
66 How and whether to mobilize power groups
67 How the community is affected by the care delivery system
Periodic Information Set Ignored Information Set
June 2003 George Mason University 19
Take Home Lesson
Needs assessment is important It matters how needs are assessed It is important to distinguish needs from
wish lists– It is important to look to the future– It important to assess in context of specific
decisions– It is important to make data collection plans
June 2003 George Mason University 20
Needs Assessment Exercise
1. List 3 decisions you will make next week
2. List information needed in each decision
3. Make master list
4. Rate the relevance of each item for each decision
5. Analyze ratings across decisions Range of ratings Average of ratings
June 2003 George Mason University 21
Minute Evaluations
Always complete the minute evaluation at end of each lecture
