School Counselors Doing Action Research Jay Carey and Carey Dimmitt Center for School Counseling Outcome Research UMass Amherst CT Guidance Leaders March

School Counselors School Counselors Doing Action Doing Action

Research Research

Jay Carey and Carey DimmittJay Carey and Carey Dimmitt

Center for School Counseling Outcome Center for School Counseling Outcome ResearchResearch

UMass Amherst UMass Amherst

CT Guidance LeadersCT Guidance Leaders

March 12, 2004March 12, 2004

What is Action Research?What is Action Research?

►Action research is a valuable and powerful Action research is a valuable and powerful way to use data to impact educational way to use data to impact educational practice. practice.

►Systematically collecting information Systematically collecting information about what is happening, evaluating the about what is happening, evaluating the effectiveness of interventions, and effectiveness of interventions, and disseminating the results helps school disseminating the results helps school counselors become more intentional and counselors become more intentional and thus more successful in helping all children thus more successful in helping all children learn. learn.

Action Research ProcessAction Research Process

►Action research allows school Action research allows school counselors to evaluate programs and counselors to evaluate programs and interventions in their specific contexts. interventions in their specific contexts.

►Practices can change immediately and Practices can change immediately and in an ongoing manner as the data is in an ongoing manner as the data is collected and analyzed.collected and analyzed.

Action Research ProcessAction Research Process

1. Identify a problem1. Identify a problem

2. Review what is known about the problem2. Review what is known about the problem

3. Develop research hypotheses or questions 3. Develop research hypotheses or questions about the problem and plan the research about the problem and plan the research process accordinglyprocess accordingly

4. Gather the data4. Gather the data

5. Analyze the data5. Analyze the data

6. Interpret results; disseminate and use findings6. Interpret results; disseminate and use findings

7. Evaluate the research process7. Evaluate the research process

Conducting Action Research: Conducting Action Research: Identify the ProblemIdentify the Problem

1. Identify the problem and determine 1. Identify the problem and determine the purpose of the research. Ask the purpose of the research. Ask yourself:yourself: What is the problem I want to solve?What is the problem I want to solve? What is the question I want to answer?What is the question I want to answer? What do I wish was different?What do I wish was different?


►Define the problem/question in clear, Define the problem/question in clear, specific language. The question may specific language. The question may be general or specific.be general or specific. ““Does the new Math Lab increase math grades or Does the new Math Lab increase math grades or

math test scores?”math test scores?” ““Which guidance curricula are having the Which guidance curricula are having the

greatest positive impact on student behavior?”greatest positive impact on student behavior?” ““Does a study skills intervention impact 7Does a study skills intervention impact 7thth grade grade

failure rates?” failure rates?” ““Does the Career Exploration Guidance Does the Career Exploration Guidance

Curriculum unit improve student knowledge of Curriculum unit improve student knowledge of career choices?”career choices?”


►Purpose of the study: Once the research Purpose of the study: Once the research question is identified, consider what you question is identified, consider what you will do with the solutions and/or answers will do with the solutions and/or answers to your question.to your question. What educational practices would you like to What educational practices would you like to

impact?impact? What student outcomes are you hoping to What student outcomes are you hoping to

influence?influence? What decisions will the study help you What decisions will the study help you

make?make?

Conducting Action Research: Conducting Action Research: What is Already Known?What is Already Known?

2. Review what is known about the 2. Review what is known about the problem.problem. Has anyone in the school asked this Has anyone in the school asked this

question before?question before? Who might have information?Who might have information? What is the relevant research in What is the relevant research in

professional journals?professional journals? What does an internet search find on this What does an internet search find on this

topic?topic?

Conducting Action Research: Conducting Action Research: What is Already Known?What is Already Known?

►Evaluating findings from research Evaluating findings from research literature and internet searches:literature and internet searches: What is the source? How reliable is it?What is the source? How reliable is it? What are the strengths/weaknesses of the What are the strengths/weaknesses of the

research design, sampling, effect size, research design, sampling, effect size, measures used, treatment fidelity, measures used, treatment fidelity, researcher bias, instrument reliability and researcher bias, instrument reliability and validity?validity?

Conducting Action Research: Conducting Action Research: Develop HypothesesDevelop Hypotheses

3. Develop research hypotheses or 3. Develop research hypotheses or questions about the problems and plan questions about the problems and plan the research process accordingly. the research process accordingly. Ask yourself what you think the answer(s) Ask yourself what you think the answer(s)

to your question(s) will be.to your question(s) will be. Identifying your hypotheses helps you Identifying your hypotheses helps you

identify your biases, which may impact identify your biases, which may impact your process.your process.

What is the opposite of your hypothesis What is the opposite of your hypothesis (the “null hypothesis”)? What would the (the “null hypothesis”)? What would the data look like if you were wrong?data look like if you were wrong?

Conducting Action Research: Conducting Action Research: Gather DataGather Data

4. Gather the data.4. Gather the data. What information do you need in order to What information do you need in order to

answer your question or solve your answer your question or solve your problem? problem?

Use multiple sources of data, or multiple Use multiple sources of data, or multiple outcome measures, wherever possible.outcome measures, wherever possible.

Decide whether you need to consider Decide whether you need to consider student achievement data, psychosocial student achievement data, psychosocial data, career data, school data, process data, career data, school data, process data, perception data, and/or results data.data, perception data, and/or results data.


►Where is the data?Where is the data? Does it already exist?Does it already exist?

►School records, Teacher recordsSchool records, Teacher records

Will you generate your own data?Will you generate your own data?►SurveysSurveys►InterviewsInterviews►ObservationsObservations


►How accurate is the data you’ve How accurate is the data you’ve chosen to use? What’s missing? chosen to use? What’s missing?

►Multiple data sources help you more Multiple data sources help you more accurately get at the complexity of a accurately get at the complexity of a situation, whereas one measure or situation, whereas one measure or data source will give you a snapshot data source will give you a snapshot view.view.


►Select and/or develop the instruments Select and/or develop the instruments you will use to gather the data. you will use to gather the data. Possibilities include:Possibilities include: SurveysSurveys Tests (of achievement, aptitude, attitude, Tests (of achievement, aptitude, attitude,

etc.)etc.) Behavioral checklists or observationsBehavioral checklists or observations Performance assessmentsPerformance assessments interviewsinterviews


►Reliability and validity need to be Reliability and validity need to be considered when selecting considered when selecting instruments. Does it measure what instruments. Does it measure what it’s supposed to? Have others used it’s supposed to? Have others used the instrument and found it to be the instrument and found it to be reliable and valid?reliable and valid?

► Just because an instrument exists Just because an instrument exists doesn’t mean it’s well done!doesn’t mean it’s well done!

Conducting Action Research:Conducting Action Research:Gather DataGather Data

► Identify and follow ethical and legal Identify and follow ethical and legal standards:standards: No participant should be exposed to physical No participant should be exposed to physical

or psychological harm.or psychological harm. Permission to use confidential data must be Permission to use confidential data must be

obtained.obtained. Participation in a study is always voluntary.Participation in a study is always voluntary. Participants may withdraw from the study at Participants may withdraw from the study at

any time.any time. Participants’ privacy rights must be Participants’ privacy rights must be

respected.respected.


► Identify the group/sample to be studied:Identify the group/sample to be studied: Ideally either the entire sample is involved in Ideally either the entire sample is involved in

the study (the class, grade, or school) or the the study (the class, grade, or school) or the group studied is a random sample of the group studied is a random sample of the population.population.

Stratified sampling uses a smaller sample Stratified sampling uses a smaller sample which has the same proportions as the larger which has the same proportions as the larger sample.sample.

Systematic random sampling is when every x Systematic random sampling is when every x number of students is chosen from the whole number of students is chosen from the whole population (every 3population (every 3rdrd, or every 4, or every 4thth for for example).example).


►Often there are context constraints Often there are context constraints which limit the sample size. which limit the sample size.

►One way to strengthen the findings is One way to strengthen the findings is to compare the group who received to compare the group who received the intervention with a comparable the intervention with a comparable group who didn’t receive the group who didn’t receive the intervention.intervention.


►Once data sources and measures are Once data sources and measures are identified, ethical standards are identified, ethical standards are considered, and the sample is considered, and the sample is identified, data can be gathered!identified, data can be gathered!

Conducting Action Research: Conducting Action Research: Analyze the DataAnalyze the Data

5. Analyze the data:5. Analyze the data: Before data can be analyzed it may need Before data can be analyzed it may need

to be edited, coded and organized for to be edited, coded and organized for analysis.analysis.

Data is often entered into a software Data is often entered into a software program such as Excel or SPSS.program such as Excel or SPSS.


►Descriptive statistics describe the data Descriptive statistics describe the data and can provide information about and can provide information about how a group has changed over time:how a group has changed over time: Measures of central tendencyMeasures of central tendency

►Mean, median, modeMean, median, mode

Measures of variabilityMeasures of variability►Variance, standard variation, rangeVariance, standard variation, range

Measures of relative standingMeasures of relative standing►Percentile rankPercentile rank


► Inferential statistics provide additional Inferential statistics provide additional information:information: Looking carefully at how a group changes Looking carefully at how a group changes

over timeover time►Use t-tests or Chi-SquareUse t-tests or Chi-Square

Looking at differences between control Looking at differences between control and intervention groupsand intervention groups►Use t-tests or Chi-SquareUse t-tests or Chi-Square

Looking at differences among more than 2 Looking at differences among more than 2 groupsgroups►Use Analysis of Variance (ANOVA)Use Analysis of Variance (ANOVA)


►Finding parent, colleague, or university Finding parent, colleague, or university partner(s) who already know how to partner(s) who already know how to input and analyze data can make this input and analyze data can make this part of the research process easier.part of the research process easier.

Conducting Action Research:Conducting Action Research:Interpret Research Results Interpret Research Results

6. Interpret research results, 6. Interpret research results, disseminate, and use findings to disseminate, and use findings to inform practice. Develop an action inform practice. Develop an action plan.plan. What do the results of your analysis What do the results of your analysis

mean?mean? What did you find out?What did you find out? Were your hypotheses correct?Were your hypotheses correct?

Conducting Action Research: Conducting Action Research: Interpret Research ResultsInterpret Research Results

►How can the results be communicated How can the results be communicated so they most effectively impact so they most effectively impact behavior or understanding? behavior or understanding? Decide what’s relevant to each audience.Decide what’s relevant to each audience. Present all relevant results even if they Present all relevant results even if they

don’t support your hypotheses.don’t support your hypotheses. Relate your findings to the purpose of the Relate your findings to the purpose of the

study, your hypotheses and previous study, your hypotheses and previous research.research.


►One of the distinguishing features of One of the distinguishing features of Action Research is that its primary goal Action Research is that its primary goal is to take action based on the research is to take action based on the research findings. Some options include:findings. Some options include: Make recommendations that will resolve Make recommendations that will resolve

the problem.the problem. Make plans and decisions about Make plans and decisions about

interventions based on the findings.interventions based on the findings. Make program plans based on the findings.Make program plans based on the findings. Develop action plans based on the findings.Develop action plans based on the findings.


►Action research seeks to use research-Action research seeks to use research-based knowledge and findings to based knowledge and findings to actively change behavior. The ultimate actively change behavior. The ultimate question is: what will you do that is question is: what will you do that is similar and what will you do that is similar and what will you do that is different based on what you learned in different based on what you learned in your research?your research?

Conducting Action Research:Conducting Action Research:Evaluate the Research ProcessEvaluate the Research Process

7. Evaluate the Research Process with 7. Evaluate the Research Process with all involved:all involved: What would we do differently next time?What would we do differently next time? Does the instrument need revising?Does the instrument need revising? Did the research generate the information Did the research generate the information

we wanted?we wanted? What new questions did the research What new questions did the research

generate?generate?


►Mills (2000) and Kemmis and Wilkinson Mills (2000) and Kemmis and Wilkinson (1998) suggest the following questions:(1998) suggest the following questions: Does the project clearly address a problem Does the project clearly address a problem

or issue in practice that needs to be solved?or issue in practice that needs to be solved? Did the researcher collect sufficient data to Did the researcher collect sufficient data to

help address the problem?help address the problem? Did the plan of action build logically from the Did the plan of action build logically from the

data?data? Did the action research actually lead to a Did the action research actually lead to a

change or did a solution to a problem make change or did a solution to a problem make a difference?a difference?


►Asking participants what they learned Asking participants what they learned from their involvement empowers from their involvement empowers them to think of themselves as active them to think of themselves as active learners in an environment where learners in an environment where evaluation is something to be used evaluation is something to be used supportively for growth rather than supportively for growth rather than something to be feared.something to be feared.

Documents

School Counselors Doing Action Research Jay Carey and Carey Dimmitt Center for School Counseling Outcome Research UMass Amherst CT Guidance Leaders March