FINAL Needs Assessment Report

8/22/2019 FINAL Needs Assessment Report

1/33

Year ThreeNational 4-H Science E-Academy

Pre-Program Needs Assessment ReportSeptember 30, 2012

Mary E. Arnold, Ph.D.

Project Evaluator

Oregon State University

With assistance from

Courtney Archibeque, MpH

Graduate Research Assistant

Oregon State University

National 4-H ScienceLeadership Academy


2/33

Table of Contents

Evaluators Statement .. iiAcknowledgements .. iii

Executive Summary .. iv

Overview .. 1

Respondent Demographics .. 3

Personal Interest in 4-H Science .. 3

Knowledge and Use of 4-H Science Tools ... 5

Increasing the Use of 4-H Science Tools ... 7

Content of the E-Academy ........ 9

E-Academy Format ..... 16

Time Commitment ..... 18

Pre and Post Academy Assignments ..... 18

Format of E-Academy Sessions ..... 18

Technological Capabilities ..... 20

Summary and Recommendations ... 21

Appendix ONE: Needs Assessment Respondent Demographics ..... 23


3/33

ii

Evaluators Statement

This document serves as the final evaluation report for the National 4-H Science Academy: Year

Three E-Academy Needs Assessmentsponsored by National 4-H Council with funding from the

Noyce Foundation. The academies were held in five locations across four regions of the country

between January and April, 2012. All academy participants were invited to participate in theprogram evaluation.

All data for the evaluation were entered by participants directly into an on-line data collection

system. Access to the system was provided by the evaluator to the participants for data entry,

but only the evaluator and her research assistants had access to the actual dataset. The

integrity and accuracy of the raw data rests with the individual participants. The integrity and

accuracy of the analysis and interpretation rests solely with me as the project evaluator. To this

end, I certify that the analysis and results presented in this document are complete and

accurate insofar as the data entered by the participants were as well. Any questions or

concerns about this report should be addressed to me.

Mary E. Arnold, Ph.D.

Project Evaluator, Oregon State University

September 30, 2012

[email protected]


4/33

iii

Acknowledgements

This needs assessment could not have taken place without the dedicated help and support of many

individuals.

First and foremost, I would like to thank National 4-H Council and the Noyce Foundation for theopportunity to conduct the Year Three Virtual Academy evaluation, which includes this formal needs

assessment. It continues to be exciting to witness the movement of the 4-H Science professional

development program from the national to the regional, and now to the local level, and to document

the important work that is taking place as a result of the Academy.

I would like to express my thanks and gratitude to Edward Bender, Janet Golden, Jo Turner, Beth

Bernstihl, and Maila Oliveria at National 4-H Council for their support and help with various aspects of

the needs assessment process.

Thank you, also, to the members of the Virtual Academy Leadership Team (VAULT) for your thoughtful

contributions to developing the needs assessment instrument.

Debbie Nistler, President of NA4-HA was very gracious in allowing us to send e-mail invitations through

their e-mail contacts to reach as many front line 4-H educators as possible. A special thank you to

Debbie and the NAE4-HA staff for making this happen and for sending out the invitations and reminders.

Similarly, I would like to extend my thanks and gratitude to all of the 4-H Science Liaisons across the

country who forwarded invitations to participate in the survey to their local 4-H contacts.

A very special thank you to my co-authors and graduate research assistant Ms. Courtney Archibeque.

Your focused assistance with the data analysis and report preparation could not be replaced. Thank you

especially for your cheerful willingness to concentrate your work time on the report so we could meet

the expected deadline.

I would like to thank each and every 4-H educator who contributed data. The sincerity with which you

approached the evaluation was evident in the data and information you provided. Your responses will

guide the work of the VALT as they plan the E-Academy based on the interests and needs you shared

through this process.

Finally, thank you to the Noyce Foundation for the generous support of the National 4-H Science

Leadership Academy. The funding provided by the foundation made this important program possible. As

a result, 4-H programs across the country are more prepared to develop and sustain programs for youth

in science, technology, engineering, and math.


5/33

iv

Executive Summary

Respondent Demographics

We received 504 responses from 48 states, with the big 4-H science states showing the greatest

participation (e.g. Illinois, Kentucky, Minnesota, and Pennsylvania). The demographic data indicate that

the intended audience (i.e. frontline county agents and educators) was reached through the needs

assessment process.

Additional demographics show that

81% of respondents were agents or educators at the county level 92% were female 88% were NOT in a science specific role 9.7% attended the National Science Academy in 2010 32 % attended a regional academy in 2012

Personal interest in 4-H Science

Over 80% indicated a high or very high personal interest in professional developmentfor 4-H Science

Over 82% indicated a commitment to learn and develop skills for 4-H Scienceprogramming

Almost 80% indicated positive support in their work environment for implementing 4-HScience programs

Use of 4-H Science Professional Development Resources

Most respondents have NOT HEARD of the national tools available to support 4-HScience.

The exception is for specific training guides, such as Junk Drawer Robotics, and ThePower of Wind; 76.8% had HEARD about these resources

Even less have USED these resources, with only 56.5% reporting using specific trainingguides

70% of respondents reported that knowing more about the resources would increasethe likelihood that they would use them

Proposed Content of the E-Academy

The topics respondents were most interested in learning about are**:

Tools and resources available to support 4-H Science programs (Doing Science) Teaching others to inquiry-based science experiences (Doing Science)


6/33

v

How to engage volunteers in 4-H Science as coaches or mentors (VolunteerDevelopment)

How to help traditional volunteers facilitate 4-H Science programs (VolunteerDevelopment)

How to effectively train volunteers in 4-H Science AND PYD principles (VolunteerDevelopment)

______________________________________________

** 75% or more are moderately to very interested in this topic

The topics respondents were next most interested in learning about are***:

Facilitating inquiry-based science experiences (Doing Science) Scientific engineering practices framework for K-12 Science (Doing Science) Facilitating science programs to limited resource and/or non-traditional audiences

(Doing Science)

How to recruit science-rich volunteers (Volunteer Development) How to include inquiry in all science programs (Curriculum) An overview of current 4-H Science curriculum resources (Curriculum) Strategies for engaging teens as teachers in 4-H Science programs(Curriculum)______________________________________________

*** 60% or more are moderately to very interested in this topic

E-Academy Format

Respondents were asked to indicate their preference for the format of the E-academy. This is

the only place where the needs assessment data differ in important ways when therespondents are separated out by group. For this analysis we separated out those who are

county educators from all other respondents, and in doing so, found an important difference.

Interactive e-learning sessions consisting of presentations with interactive componentsreceived the highest overall average rating (4.0/5.0)

Facilitated interactive learning: consisting of presentations viewed as a group at a localsite with a facilitator trained to conduct breakout sessions received the greatest number

of high ratings

BUT- if you consider only the responses from county-level educators the FacilitatedInteractive Learning is the most preferred (4.8/5.0 mean rating)


7/33

vi

Time Commitment for an E-Academy Work

Respondents were asked to indicate the amount of time they are willing to commit to participating in

the E-Academy:

69.8% for ONE part day e-academy (up to four hours total) 61.5% for ONE full-day academy (5+ hours) 59.7% for an academy series over 3-5 months with part days of up to 4 hours 14.9% for three or more full consecutive days 26.8% for Three or more part consecutive days

Delivery of E-Academy Sessions

Respondents were asked their preference for the delivery of the E-Academy session. Only two

types of delivery received moderate positive ratings:

71.2% webinar presentations (both live and recorded) 64.9% video presentations

Technological Capacity

It is important to note that while 70% indicated they have attended webinarssuccessfully, this also means that almost 30% have not

In addition, only 58% indicated they have adequate technological support locally in theevent of problems with participation that are caused by local equipment

And one out of four people (25%) report they do not have an appropriate setting, suchas a quiet room for participating in the e-academy

Summary and Recommendations

The results of the needs assessment are, overall, promising for the success of the E-Academy. In

particular, the respondents seem interested in the proposed content of the E-Academy, and

indicate they are willing to commit time to attending the academy. There are a few important

things to note that should be taken into consideration by the E-Academy planners, however.

1. Respondents indicated the greatest interest in content related to science programimplementation (Doing Science) and Volunteer Development. Particular aspects of

Curriculum also were rated moderately high. There are several items that stand out

clearly as interest areas for this audience and should be kept in mind when the content

of the academy is planned.

2. The proposed content for the Evaluation and Fund Development areas received low tomoderate ratings. The planners should plan content in these areas with the audience in


8/33

vii

mind and not plan a program that is not a match with the skills and needs of county-

based educators.

3. County educators indicated a very strong preference for the format of the E-Academy tobe a facilitated session. While creating this type of session is beyond the purview of the

E-Academy per se, some attention should be paid to providing information to states andcounties about how to plan for and host the E-Academy at the local level, including a

recommendation that learning be done in a facilitated session if possible.

4. Care should be taken to be sure the technology used for the delivery of the E-Academyis tested for potential problems that may occur when the educator participates in the

academy. The results of the needs assessment revealed just a fair to moderate

technological capacity of the educators. Especially noteworthy is that almost of

respondents reported having no on-site technical assistance to help them. Advanced

training, clear instructions for participation, and technical support will all be key to the

success of the E-Academy.


9/33

1

Year Three: National 4-H Science E-Academy

Pre-Program Needs Assessment

Overview

The aim of year three of the National 4-H Science Academy is to provide a professionaldevelopment program for county-based 4-H educators. The program design for year three

follows a natural progression from the first academy, held in December, 2010 that reached

teams of four from LGUs, to last years regional academies that reached a broader audience, to

the E-Academy that is intended to reach as many front-line educators as possible. This

progression marks the overall development, scope, and reach of the National 4-H Science

Academy program.

A development team entitled the Virtual Academy Leadership Team (VALT) was formed in the

early summer of 2012, under the leadership of Edward Bender. Team members are:

Edward Bender, National 4-H Council Janet Golden, National 4-H Council Jim Kahler, NIFA Suzanne Le Menestrel, NIFA Doug Swanson, NIFA Misty Blue-Terry, North Carolina A & T University (1890s) Trudy Dunham, University of Minnesota (North Central) Debbie Fajans, University of Vermont (Northeast Heather Kent, University of Florida (South) Steven Worker, University of California, Davis (West) Mary Arnold, Oregon State University (Evaluator)

A face to face meeting of the VALT was held in July at the National 4-H Conference. At the

meeting the team was charged with the development and delivery of the E-Academy, and

began the initial planning phase. Part of the meeting was devoted to a review of the

evaluations of the Year One and Two academies, which informed some of the initial planning.

One of the end products of the July VALT meeting was the development of the instrument for

the pre-program needs assessment. Following the meeting the evaluator drafted the initial

instrument based on discussions and planning at the meeting, and worked via e-mail with VALT

members to refine and finalize the needs assessment instrument. The needs assessment

focused on the following distinct areas:

Respondent Demographics Personal interest in 4-H Science


10/33

2

Current use of 4-H Science Professional Development Resources Proposed Content of the E-Academy

o 4-H Science Program Development and Deliveryo Volunteer Developmento Curriculum and Program Developmento Evaluationo Resource Development

Structure and Participation in the E-Academyo The Focus of the E-Academyo Time Commitment and E-Academy Worko Types of E-Academy Sessionso Technological Capacity

Respondents to the needs assessment were recruited through LGU Science Liaisons and

through direct e-mails from NAE4-HA to its members. E-mail invitations were sent at least two

times through each method, with some liaisons following up more than one time with areminder.

Data were collected on-line through a commercial survey company, and no problems were

reported with the data collection process. Data were collected between August 1 and August

20, 2012. Responses were downloaded from the on-line collector into an Excel file and finally

into SPSS for analysis.


11/33

3

Respondent Demographics

In all, we received 504 responses from 48 states, with some of the bigger 4-H science states

showing the greatest participation (e.g. Illinois, Kentucky, Minnesota, Pennsylvania). Additional

demographics show that

81% of respondents were agents or educators at the county level 92% were female 88% were NOT in a science specific role 9.7% attended the National Science Academy in 2010 32 % attended a regional academy in 2012

The demographic data indicate that the intended audience (i.e. frontline county agents and

educators) was reached through the needs assessment process. A detailed analysis of the

respondent demographics can be found in Appendix One.

It is important to note that while the respondent demographics did match those of the

intended audience, the N of 504 is small relative to the estimated population size of over 3,000.

The demographic results, however, do indicate that the responses came from a fair

representation of educators across the country, with one exception: Because of the relatively

small N, and the percentage of the respondents who indicate a moderate to strong interest in

science, it may be safe to assume that these results represent those who are most interested in

promoting 4-H Science, and not the overall body of 4-H agents/educators in general.

Nonetheless, the E-Academy is being developed as a train-the-trainer model, thus the

development of the E-Academy should be geared toward those on the local level who show aninterest in 4-H Science, as they will most likely become the trainers of others at the local level.

To this end, the responses provided in the needs assessment appear to be valuable and

meaningful.

Personal Interest in 4-H Science

Over 80% indicated a high or very high personal interest in professional developmentfor 4-H Science

Over 82% indicated a commitment to learn and develop skills for 4-H Scienceprogramming

Almost 80% indicated positive support in their work environment for implementing 4-HScience programs

Table 1.0 represents respondents personal interest in 4-H Science. Participants rated desire

to learn and develop skills for developing 4-H Science programs as the area in which there is


12/33

4

the most personal interest. The least interest is in level of positive support in your work

environment for implementing 4-H science programs. Figure 1.0 shows the percentage of

respondents who rated their interest in each item high or very high.

Table 1.0 Personal Interest in 4-H Science

Very Low Low High

Very

High

High and

Very

High Missing

Desire to learn and develop skills for

developing 4-H Science programs3

0.6%

48

9.5%

234

46.4%

193

38.3%

427

84.7%

26

5.2%

Commitment to learn and develop skills for

developing 4-H Science programs0

62

12.3%

229

45.4%

187

37.1%

416

82.5%

26

5.2%

Ability to implement what you learn in order

to enhance 4-H Science programming

2

0.4%

45

8.9%

279

55.4%

149

29.6%

428

85.0%

29

5.8%

Ability to use what you learn in order to

enhance 4-H Science programming0

30

6.0%

277

55.0%

167

33.1%

394

88.1%

30

6.0%

Level of positive support in your work

environment for implementing 4-H Science

programs

9

1.8%

68

13.5%

266

52.8%

134

26.6%

400

79.4%

27

5.4%

Figure 1.0

46.4% 45.4%

55.4% 55.0% 52.8%

38.3% 37.1% 29.6%33.1%

26.6%

0%

20%

40%

60%

80%

100%

Desire to learn

and develop skills

for developing 4-

H Science

programs

Commitment to

learn and develop

skills for

developing 4-H

Science programs

Ability to

implement what

you learn in order

to enhance 4-H

Science

programming

Ability to use

what you learn in

order to enhance

4-H Science

programming

Level of positive

support in your

work

environment for

implementing 4-H

Science programs

Very High

High


13/33

5

Knowledge and Use of 4-H Science Tools

Most respondents have NOT HEARD of the national tools available to support 4-HScience.

The exception is for specific training guides, such as Junk Drawer Robotics, and ThePower of Wind; 76.8% had HEARD about these resources

Even less have USED these resources, with only 56.5% reporting using specific trainingguides

Table 2.0 presents the awareness of participants of various tools used for 4-H ScienceTable 2.1 shows those who have used them. The most heard of tool was the Specific

Project Training Guides (Junk Drawer Robotics, Power of Wind) and the least heard of

tool was the 4-H Science Smart Competency Training Guide. Respectively, these two

tools were also the most and the least used by respondents.

Table 2.0 Knowledge of 4-H Science Tools

Heard Of: Yes No Missing

4-H Science Smart Competency Training Guide 71

14.1%

391

77.6%

42

8.3%

Using inquiry-based learning to support 4-H science on-line course 156

31.0%

306

60.7%

42

8.3%

4-H Science checklist 177

35.1%

283

56.2%

44

8.7%

4-H Science 101 Training Guide 147

29.2%

314

62.3%

43

8.5%

4-H Science Competencies 237

47.0%

223

44.2%

44

8.7%Inquiry-based learning online learning modules 174

34.5%

286

56.7%

44

8.7%

Fund Development Toolkit 165

32.7%

295

58.5%

44

8.7%

4-H Science Logic Model 260

51.6%

202

40.1%

42

8.3%

Promising Practices 135

26.8%

324

64.3%

45

8.9%

Specific Project Training Guides (Junk Drawer Robotics, Power of Wind) 387

76.8%

77

15.3%

40

7.9%

Recruiting and Developing Volunteers 287

56.9%

173

34.3%

44

8.7%

Archived 4-H Science Academy Webinars 144

28.6%

316

62.7%

44

8.7%

Curriculum Rubrics 143

28.4%

312

61.9%

49

9.7%

Youth Engagement, Attitudes and Knowledge (YEAK) Survey 152

30.2%

306

60.7%

41

9.1%


14/33

6

Table 2.1 Use of 4-H Science Tools

Used: Yes No Missing

4-H Science Smart Competency Training Guide 26

5.2%

429

85.1%

49

9.7%

Using Inquiry-Based Learning to Support 4-H Science on-line Course 60

11.9%

397

78.8%

47

9.3%

4-H Science Checklist 122

24.2%

335

66.5%

47

9.3%

4-H Science 101 Training Guide 89

17.7%

369

73.2%

46

9.1%

4-H Science Competencies 140

27.8%

318

63.1%

46

9.1%

Inquiry-Based Learning Online Learning Modules 65

12.9%

393

78.0%

46

9.1%

Fund Development Toolkit 37

7.3%

418

82.9%

49

9.7%

4-H Science Logic Model 139

27.6%

318

63.1%

47

9.3%

Promising Practices 68

13.5%

387

76.8%

49

9.7%

Specific Project Training Guides (Junk Drawer Robotics, Power of Wind) 285

56.5%

175

34.7%

44

8.7%

Recruiting and Developing Volunteers 153

30.4%

303

60.1%

48

9.5%

Archived 4-H Science Academy Webinars 51

10.1%

405

80.4%

48

9.5%

Curriculum Rubrics 56

11.1%

394

78.2%

54

10.7%

Youth Engagement, Attitudes and Knowledge (YEAK) Survey 72

14.3%

379

75.2%

53

10.5%


15/33

7

Increasing Use of 4-H Science Tools

Table 3.0 displays ways participants would increase their use of the 4-H Science Tools. Most participants

(70.6%) reported that knowing more about them would increase their usage the most. Figure 3.0

presents the results in a graphic format.

Table 3.0

Increasing Usage of 4-H Science Tools

What would increase your likelihood of using tools?

Frequency

% Missing

Knowing more about them356

70.6%

148

29.4%

Easier access to the tools 212

42.1%

292

57.9%

Training on how to use the tools 285

56.5%

219

43.5%A need for them based on my clientele 173

34.3%

331

65.7%

Other Responses Include the Following:

Thirty (30) respondents mentioned that some aspect of time, i.e. more time in the day, moretime to learn about tools, more time to use the tools, and more time to implement them, is

needed in order for them to better use the tools.

Nine (9) respondents stated that they would use the tools more if they knew where to findthem and how to access them. Some complaints about accessing the tools include: the URLs

seem to change too frequently, searching/browsing is not effective in locating the tools, State

Liaisons not sharing any of the resources with County Liaisons, enrolling in webinars is

difficult, finding archived webinars is difficult, finding the tools is difficult when not all in one

spot.

Four (4) respondents mentioned needing more resources and funds for developingeducational kits, travel to trainings or for materials.

Three (3) respondents mentioned having more staff especially administrative staff to helpwith day to day tasks so more time can be spent in learning about and using the tools.

Two (2) respondents would like the focus of team meetings to be on the tools and trainingresources and to have clearer communication between state and county staff who have

already learned about using these tools.

One (1) respondent each reported that they would like: to see how others use/implementthese tools and resources, venues to use them (partnership opportunities, project days,

community events, etc.), and that more support is needed from supervisors to use science

programming.


16/33

8

70.6%

42.1%

56.5%

34.3%

0%

20%

40%

60%

80%

100%

Knowing more about

them

Easier access to the tools Training on how to use

the tools

A need for them based on

my clientele

Figure 3.0

Increasing Usage of 4-H Science Tools


17/33

9

85.1%

78.4% 78.4%72.2%

66.9%

0%

20%

40%

60%

80%

100%

Doing 4-H Science Volunteer

Development

Curriculum and

Program

Development

Evaluation Resource

Development

Content of E-Academy

Respondents were asked about to rate their interest in the proposed content of the E-

Academy, which is arranged around five content areas: 1) Doing 4-H Science; 2) Volunteer

Development; 3) Curriculum and Program Development; 4) Evaluation; and 5) Resource

Development. The first question asked if respondents would attend an e-learning session oneach topic. The percentages of those who replied Yes are presented in graphic form in Figure

4.0

Figure 4.0

Percentage of Respondents indicating they would attend an E-Learning Session on this Topic

When asked about specific topics of interest in each of the areas, the primary interest is in the

areas of Doing Science and Volunteer Development. A second tier of interest is in curriculum.

The topics respondents were most interested in learning about are**:

Tools and resources available to support 4-H Science programs (Doing Science) Teaching others to inquiry-based science experiences (Doing Science) How to engage volunteers in 4-H Science as coaches or mentors (Volunteer

Development)

How to help traditional volunteers facilitate 4-H Science programs (VolunteerDevelopment)


18/33

10

How to effectively train volunteers in 4-H Science AND PYD principles (VolunteerDevelopment)

______________________________________________

** 75% or more are moderately to very interested in this topic

The topics respondents were next most interested in learning about are***:

Facilitating inquiry-based science experiences (Doing Science) Scientific engineering practices framework for K-12 Science (Doing Science) Facilitating science programs to limited resource and/or non-traditional audiences

(Doing Science)

How to recruit science-rich volunteers (Volunteer Development) How to include inquiry in all science programs (Curriculum) An overview of current 4-H Science curriculum resources (Curriculum) Strategies for engaging teens as teachers in 4-H Science programs(Curriculum)______________________________________________

*** 60% or more are moderately to very interested in this topic

Detailed results for each proposed topic are presented in Tables 4.0 - 4.4. Items that

respondents were 1) least interested in; 2) most interested in; and 3) had the highest combined

moderately and very interested are highlighted in each table. Also included are summaries of

the open-ended responses.

In most cases the same item was the one that received both the most very interested ratingsand the most combined moderately and very interested ratings, there were two

exceptions: 1) Learning about strategies for engaging teens as teachers in 4-H Science

programming emerged as the item with the most interest in the areas of curriculum and

program development; and 2) Building partnerships for 4-H Science rose to the top in the fund

development area, which is the only item that over 60% of respondents indicated interest.


19/33

11

Table 4.0

Interest in Doing 4-H Science Topics

Not

interested

Somewhat

interested

Moderately

interested

Very

interested

Moderately

and Very

Interested MissingTools and resources available to

support 4-H Science programs5

1.0%

50

9.9%

142

28.2%

263

52.2%

405

80.4%44

8.7%

Why inquiry is the basis for 4-H

Science

44

8.7%

141

28.0%

161

31.9%

114

22.6%

275

54.5%

44

8.7%

Connection between science

inquiry and experiential learning

32

6.3%

114

22.6%

166

32.9%

148

29.4%

314

62.3%

44

8.7%

Scientific engineering practices

framework for K-12 science

17

3.4%

92

18.3%

171

33.9%

178

35.3%

349

69.2%

46

9.1%

Facilitating inquiry-based science

experiences

15

3.0%

77

15.3%

183

36.3%

185

36.7%

368

73.0%

44

8.7%

Facilitating science programs tolimited resource or non-traditional

audiences

13

2.6%

85

16.9%

155

30.8%

206

40.9%

36171.7%

45

8.9%

Teaching others to facilitate

inquiry-based science experiences

21

4.2%

68

13.5%

157

31.2%

212

42.1%

369

73.3%

45

9.1%

Asking high quality science

questions

19

3.8%

99

19.6%

179

35.5%

162

32.1%

341

67.6%

45

8.9%

Developing science argumentation

skills

45

8.9%

142

28.2%

149

29.6%

118

23.4%

267

53.0%

50

9.9%

Other:

Six (6) respondents mentioned that they have extensive training or knowledge in science and scienceeducation so more in-depth learning and moving past the basics is needed for them.

Three (3) respondents stated that they would like to see some sessions designed with volunteers inmind with specific science activities that can be taught to and used by volunteers.

Three (3) respondents mentioned that having more specific science resources such as curricula andactivities that can be used would be helpful.

One (1) respondent mentioned that having a thorough academy on the tools, practical applications andwhat has been successful would be helpful.

One (1) respondent mentioned that they would be interested in teaching most of these sessions. Other responses included: developing science tool kits, evaluation of science content, how animal

science can fit into the new 4-H Science/STEM push, informal vs. formal science environment, and

making the case for 4-H science.


20/33

12

Table 4.1

Interest in Volunteer Development Topics

Not

interested

Somewhat

interested

Moderately

interested

Very

interested

Moderately

and Very

Interested MissingHow to recruit science-rich

volunteers

10

2.0%

80

15.9%

142

28.2%

223

44.2%

365

72.4%

49

9.7%

How to engage volunteers in 4-H

Science as coaches or mentors

11

2.2%

65

12.9%

150

29.8%

229

45.4%

379

75.2%

49

9.7%

How to help traditional volunteers

facilitate 4-H Science programs

8

1.6%

64

12.7%

131

26.0%

252

50.0%

383

76.0%

49

9.7%

Reframing the volunteer role to

match the needs of 4-H Science

programs

23

4.6%

93

18.5%

172

34.1%

167

33.1%

339

67.2%

49

9.7%

Implementing the new research-

based national 4-H campaign to

effectively identify, recruit and

retain 4-H volunteers

20

4.0%

89

17.7%

160

31.7%

182

36.1%

342

67.8%

53

10.5%

Strategies for recruiting corporate

volunteers

24

4.8%

95

18.8%

163

32.3%

172

34.1%

335

66.4%

50

9.9%

How to effectively train volunteers

in 4-H Science and PYD

7

1.4%

73

14.5%

149

29.6%

226

44.8%

375

74.4%

49

9.7%

Working with limited resource and

multi-cultural audiences in 4-H

science programs

19

3.8%

94

18.7%

142

28.2%

200

39.7%

342

67.9%

49

9.7%

Other:

Most respondents mentioned that they have had adequate training in volunteer development andrecruitment but need help with more specific issues such as finding volunteers in rural areas, specificsabout recruiting, training and supporting volunteers, real world help at the local level not theory,

One (1) respondent mentioned they need help with retention of volunteers and one other respondentreported that volunteers and parents complain that 4-H is becoming more like school.


21/33

13

Table 4.2

Interest in Curriculum and Program Development Topics

Not

interested

Somewhat

interested

Moderately

interested

Very

interested

Moderately

and Very

Interested Missing

Using the 4-H science checklist

to determine science-ready

programs

23

4.6%

120

23.8%

162

32.1%

136

27.0%

298

59.1%

63

12.5%

Using the national 4-H

Curriculum rubrics to determine

high quality curriculum

33

6.5%

128

25.4%

157

31.2%

124

24.6%

281

55.8%

62

12.3%

Revising current curriculum to

include 4-H Science programs

31

6.2%

111

22.0%

141

28.0%

159

31.5%

300

59.5%

62

12.3%

How to include inquiry in all

science programs

24

4.8%

87

17.3%

165

32.7%

166

32.9%

331

65.6%

62

12.3%

Adapting or revising acurriculum to include science

inquiry

30

6.0%

91

18.1%

164

32.5%

152

30.2%

316

62.7%

67

13.3%

An overview of current 4-H

Science curriculum resources

18

3.6%

77

15.3%

146

29.0%

201

39.9%

347

68.9%

62

12.3%

How to participate in the NYSD 57

11.3%

115

22.8%

134

26.6%

135

26.8%

269

53.4%

63

12.5%

Strategies for creating youth-

adult partnerships in 4-H science

programs

16

3.2%

116

23.0%

164

32.5%

143

28.4%

307

60.9%

65

12.9%

Strategies for engaging teens as

teachers in 4-H Scienceprogramming

15

3.0%

77

15.3%

158

31.3%

191

37.9%

349

69.2%

63

12.5%

Designing 4-H Science programs

that integrate the mission

mandates

22

4.4%

116

23.0%

167

33.1%

136

27.0%

303

60.1%

63

12.5%

Other:

Five (5) respondents mentioned that they are less interested in sessions about things they already dosuch as the NYSD.

Two (2) respondents mentioned they need help in writing curricula and finding lesson plans.


22/33

14

Table 4.3

Interest in Evaluation Topics

Not

interested

Somewhat

interested

Moderately

interested

Very

interested

Moderately

and Very

Interested MissingOverview of evaluation basics 52

10.3%

113

22.4%

148

29.4%

123

24.4%

271

53.8%

68

13.5%

Planning and conducting an

evaluation of 4-H Science

programs

33

6.5%

97

19.2%

146

29.0%

159

31.5%

305

60.5%

69

13.7%

Overview of existing evaluation

instruments and tools

24

4.8%

91

18.1%

147

29.2%

173

34.3%

320

63.5%

69

13.7%

Accessing the 4-H online

evaluation basics course

29

5.8%

102

20.2%

152

30.2%

152

30.2%

304

60.4%

69

13.7%

Overview of recent 4-H research

that documents the impact of 4-

H Science programs

38

7.5%

110

21.8%

152

30.2%

134

26.6%

286

56.8%

70

13.9%

Overview of current

informal/non-formal science

learning research and how it

documents the impact of 4-H

Science programs

38

7.5%

119

23.6%

152

30.2%

127

25.2%

279

55.4%

68

13.5%

Preparing effective 4-H Science

Program success stories

31

6.2%

96

19.0%

151

30.0%

159

31.5%

310

61.5%

67

13.3%

Using photos/images to

document program success

34

6.7%

85

16.9%

153

30.4%

163

32.3%

316

62.7%

69

13.7%

Other:

Four (4) respondents mentioned that they want new ideas on evaluation which can include:developing/using tools that allow agents to use record books and demonstrations as evaluation

methods, animal science and environmental science evaluation, new ideas and programs to evaluate

and what else is out there besides YEAK.

One (1) respondent mentioned that they are not interested in the nationally available evaluation toolssince their state does not use them.

One (1) respondent mentioned that they need help in conveying the program stories throughevaluation.


23/33

15

Table 4.4

Interest in Resource Development Topics

Not

interested

Somewhat

interested

Moderately

interested

Very

interested

Moderately

and Very

Interested Missing

Mission to Market; The End of

Fund Raising

42

8.3%

125

24.8%

114

22.6%

140

27.8%

254

50.4%

83

16.5%

Implementing the 4-H Science

Fund Development Toolkit

39

7.7%

124

24.6%

136

27.0%

120

23.8%

256

50.8%

85

16.9%

The top 10 funding strategies for

4-H Science

32

6.3%

89

17.7%

138

27.4%

162

32.1%

300

59.5%

83

16.5%

Developing a 4-H Science

Advocacy card to use with

potential donors

59

11.7%

119

23.6%

126

25.0%

117

23.2%

243

48.2%

83

16.5%

Building partnerships for 4-H

Science

23

4.6%

91

18.1%

157

31.2%

151

30.0%

308

61.2%

82

16.3%

Using the NYSD to market your

4-H Science program

41

8.1%

101

20.0%

139

27.6%

139

27.6%

278

55.2%

84

16.7%

Volunteer and alumni funding

support

56

11.1%

130

25.8%

129

25.6%

104

20.6%

233

46.2%

85

16.9%

Examples of current successful

partnerships for 4-H Science

35

6.9%

115

22.8%

149

29.6%

121

24.0%

270

53.6%

84

16.7%

Developing an elevator speech

about your 4-H science program

to use with potential donors

56

11.1%

117

23.3%

115

22.8%

132

26.2%

247

49.0%

84

16.7%

Using 4-H Science impact

research to support your case

for potential funders

42

8.3%

126

25.0%

126

25.0%

125

24.8%

251

49.8%

85

16.9%

Creating a public valuestatement for your 4-H Science

program

38

7.5%

109

21.6%

138

27.4%

133

26.4%

271

53.8%

86

17.1%

Strategies for making a

successful ask of potential

donors

41

8.1%

124

24.6%

131

26.0%

121

24.0%

252

50.0%

87

17.3%

Other:

Two (2) respondents reported that training on fund development might be more helpful if it was tailored todifferent job positions so that each person knows their role in fund development based on their position. This

could help communication between co-workers regarding fund development.

Two (2) respondents stated that finding funding is an issue and would like more resources for finding funding thatarent competitive.

One (1) respondent would like to have these sessions available after the academy for references when they areneeded.

One (1) respondent would like more focus to be placed on other science topics such as animal and environmentalscience.

One (1) respondent would like help in convincing other extension staff that 4-H science is important and thatfundraising is needed.

One (1) respondent mentioned that they need more time to receive all of the training topics and implement them.


24/33

16

E-Academy Format

Respondents were asked to indicate their preference for the format of the E-academy. This is

the only place where the needs assessment data differ in important ways when the

respondents are separated out by group. For this analysis we separated out those who arecounty educators from all other respondents, and in doing so, found an important difference.

Interactive e-learning sessions consisting of presentations with interactive componentsreceived the highest overall average rating (4.0/5.0)

Facilitated interactive learning: consisting of presentations viewed as a group at a localsite with a facilitator trained to conduct breakout sessions received the greatest number

of high ratings

BUT- if you consider only the responses from county-level educators the FacilitatedInteractive Learning is the most preferred (4.8/5.0 mean rating)

Table 5.0 represents the preference of all respondents for the format of the E-Academy.Table 5.1 shows the preferences for respondents who are county-based educators.

County educators indicated a strong preference for an facilitated interactive e-learning

session as the format of the academy. All participants reported that they would prefer

to participate via distance technologies.


25/33

17

Table 5.0

Preference in Format of E-AcademyAll Respondents

Format TypesAverage

1=low agreement

5=high agreement

I would not

participate in this

format

AllResponses

On-line e-learning session: Presentation-based learning,

non-interactive presentations about each topic3.3

26

5.2%

Interactive e-learning session: Presentations with some

interactive component4.0

20

4.0%

Enhanced interactive e-learning: Interactive

presentations with required additional on-line

participation such as assignments, field trips, breakout

discussions

3.443

8.5%

Facilitated interactive learning: Presentations viewed as

a group at a local site with a facilitator trained to

conduct breakout sessions

3.929

5.8%

I would NOT participate if only offered via distance

technologies1.7

30

6.0%

Other:

Nine (9) respondents mentioned that they would prefer to have face to face training ratherthan the E-Academy but realized that logistics and costs are inhibitors. Also, E-learning has

advantages that are not available with a traditional conference format.

Two (2) respondents reported that the E-learning that blends live interaction is the bestformat.

One (1) respondent stated that they want information to be archived so that the informationcan be accessed when it is needed.

Table 5.0

Preference in Format of E-AcademyCounty Educators

CountyE

ducators

On-line e-learning session: Presentation-based learning,

non-interactive presentations about each topic3.3

24

5.9%

Interactive e-learning session: Presentations with some

interactive component3.9

18

4.4%

Enhanced interactive e-learning: Interactive

presentations with required additional on-line

participation such as assignments, field trips, breakoutdiscussions

3.434

8.3%

Facilitated interactive learning: Presentations viewed as

a group at a local site with a facilitator trained to

conduct breakout sessions

4.822

5.4%

I would NOT participate if only offered via distance

technologies1.8

27

6.6%


26/33

18

Time Commitment

Respondents were asked to indicate the amount of time they are willing to commit to participating in

the E-Academy. The following percentages indicated a yes for the indicated time:

69.8% for ONE part day e-academy (up to four hours total) 61.5% for ONE full-day academy (5+ hours) 59.7% for an academy series over 3-5 months with part days of up to 4 hours 14.9% for three or more full consecutive days 26.8% for Three or more part consecutive days

Pre and Post Academy Assignments

Respondents were asked the likelihood that they would participate in pre and post academy

assignments. Figure 5.0 presents the percentage of respondents who indicated they were likely or

very likely to complete these assignments.

Figure 5.0

Likelihood of Completing Pre and Post Academy Assignments

Format of E-Academy Sessions

Respondents were asked to rate their interest in participating and presenting at the E-Academy.

The top two topics respondents were interested in participating in are: video (64.9%) and

webinar (71.2%) presentations. Participants have the least interest in participating in the

Pechakucha sessions (39.7%). All respondents reported that they would be the most interested

48.4%

54.8%

48.0%

59.1% 56.5%10.7%

21.4%

9.7%

12.7% 12.9%

0%

20%

40%

60%

80%

100%

Complete

assignments

BEFORE the e-

academy

Complete

assignments

during the e-

academy

Complete any

post-academy

assignments

Attend any post

academy e-

learning

opportunities

Complete online

learning modules

pre or post e-

academy

Very Likely

Likely


27/33

19

in presenting lightning sessions.1 (22.8%) Respondents would be least interested in

presenting Pechakucha2

sessions (69.6%). Figure 6.0 presents the percentage of respondents

who indicated they were interested in a particular format. Figure 6.1 shows the percentage of

respondents who are interested in presenting a particular type of session at the E-Academy.

Figure 6.0

Percentage of Respondents Indicating Interest in Types of E-Learning Sessions

1A Lightning session is a short presentation given at a conference or similar forum that last only a few minutes;

several will usually be delivered in a single period by different speakers.2

Pechakucha is a simple presentation format where 20 images are shown, each for 20 seconds. The images

advance automatically and presenter talks along to the images.

47.243.7

59.964.9

59.7

71.2

0

10

20

30

40

50

60

70

80

90

100

On-line poster

presentations

Pechakucha

session

Lightning

sessions

Video

presentation

Voice-over

power point

presentation

(recorded)

Webinar

presentations

(live and

archived)
http://en.wikipedia.org/wiki/Presentationhttp://en.wikipedia.org/wiki/Academic_conferencehttp://en.wikipedia.org/wiki/Academic_conferencehttp://en.wikipedia.org/wiki/Presentation


28/33

20

Figure 6.1

Percentage of Respondents Indicating Interest in Presenting Types of E-Learning Sessions

Technological Capabilities

Finally, respondents were asked rate their technological capability to attend the e-academy successfully.

The most inhibitive piece of technology to the respondents attending an E-Academy is having a

functional voice over internetprotocol (VOIP) microphone system. (27.4%) Other inhibitive aspectsinclude having a webcam (25.0%) and having adequate technical support in the event of a

technological problem on your end. (26.6%).

It is important to note that while 70% indicated they have attended webinars successfully, thisalso means that almost 30% have not

In addition, only 58% indicated they have adequate technological support locally in the event ofproblems with participation that are caused by local equipment

And one out of four people (25%) report they do not have an appropriate setting, such as aquiet room for participating in the e-academy

Figure 7.0 presents the percentage of respondents who indicated they had the technological capability

to attend the e-academy. .

18.70%13.90%

22.80%

14.90%20.00% 18.50%

0%

10%

20%

30%

40%

50%

60%

70%80%

90%

100%

On-line poster

presentations

Pechakucha

session

Lightning

sessions

Video

presentation

Voice-over

power point

presentation

(recorded)

Webinar

presentations

(live and

archived)


29/33

21

Figure 7.0

Percentage of Respondents Indicating Sufficient Capability to attend the E-Academy Successfully

Summary and Recommendations

The results of the needs assessment are, overall, promising for the success of the E-Academy. In

particular, the respondents seem interested in the proposed content of the E-Academy, and

indicate they are willing to commit time to attending the academy. There are a few importantthings to note that should be taken into consideration by the E-Academy planners, however.

1. Respondents indicated the greatest interest in content related to science programimplementation (Doing Science) and Volunteer Development. Particular aspects of

Curriculum also were rated moderately high. There are several items that stand out

clearly as interest areas for this audience and should be kept in mind when the content

of the academy is planned.

2. The proposed content for the Evaluation and Fund Development areas received low tomoderate ratings. The planners should plan content in these areas with the audience inmind and not plan a program that is not a match with the skills and needs of county-

based educators.

3. County educators indicated a very strong preference for the format of the E-Academy tobe a facilitated session. While creating this type of session is beyond the purview of the

E-Academy per se, some attention should be paid to providing information to states and

70.2

58.3

84.3

56.259.9

75.2

0

10

20

30

40

50

60

70

8090

100

Past webinar

participation

without technical

difficulties on

your end

Adequate tech

support in the

event of a tech

problem on your

end

Access to a

computer for

online viewing

A functional

voice over

internet protocol

(VOIP)

microphone

system

Webcam Appropriate

settings, such as

a quiet room

with no

interruptions


30/33

22

counties about how to plan for and host the E-Academy at the local level, including a

recommendation that learning be done in a facilitated session if possible.

4. Care should be taken to be sure the technology used for the delivery of the E-Academyis tested for potential problems that may occur when the educator participates in the

academy. The results of the needs assessment revealed just a fair to moderatetechnological capacity of the educators. Especially noteworthy is that almost of

respondents reported having no on-site technical assistance to help them. Advanced

training, clear instructions for participation, and technical support will all be key to the

success of the E-Academy.


31/33

23

Appendix One

Needs Assessment Respondent Demographics

Frequency Percentage

Gender MaleFemale

78390

15.5%92.5%

State

Alabama

Arizona

Arkansas

California

Colorado

Connecticut

Delaware

Florida

Georgia

HawaiiIdaho

Illinois

Indiana

Iowa

Kansas

Kentucky

Louisiana

Maine

Maryland

Massachusetts

Michigan

MinnesotaMississippi

Missouri

Montana

Nebraska

Nevada

New Hampshire

New Jersey

New Mexico

New York

North Carolina

North DakotaOhio

Oklahoma

Oregon

Pennsylvania

Rhode Island

South Carolina

2

2

5

14

6

2

1

16

16

24

35

11

5

25

32

9

8

8

4

9

365

22

3

8

7

1

8

1

16

25

324

11

12

26

1

6

0.4%

0.4%

1.0%

2.8%

1.2%

0.4%

0.2%

3.2%

3.2%

0.4%0.8%

6.9%

2.2%

1.0%

5.0%

6.3%

1.8%

1.6%

1.6%

0.8%

1.8%

7.1%1.0%

4.4%

0.6%

1.6%

1.4%

0.2%

1.6%

0.2%

3.2%

5.0%

0.6%4.8%

2.2%

2.4%

5.2%

0.2%

1.2%


32/33

24

Tennessee

Texas

Utah

Vermont

Virginia

Washington

West Virginia

Wisconsin

Wyoming

Missing

7

16

10

7

5

9

4

11

1

3

1.4%

3.2%

2.0%

1.4%

1.0%

1.8%

0.8%

2.2%

0.2%

0.6%

Region

Northeast

North Central

South

West

1890s

Missing

88

180

139

78

10

9

17.5%

35.7%

27.6%

15.5%

2.0%

1.8%

Number of years

in 4-H

First year

2-5 years6-10 years

11-15 years

16-20 years

21+years

Missing

52

123112

79

47

88

3

10.3%

24.4%22.4%

15.8%

9.4%

17.6%

0.6%

State Science

Liaison

Yes

No

Missing

36

444

24

7.1%

88.1%

4.8%

Official Role

Volunteer

Partner

State 4-H Science LiaisonState 4-H Science Specialist

State 4-H Specialist (not science)

County 4-H Educator (agent role)

County 4-H Educator (non-agent role)

State 4-H Program Leader

State 4-H Foundation Staff

State 4-H Foundation Director

Missing

4

3

614

35

345

64

14

1

2

16

0.8%

0.6%

1.2%2.8%

6.9%

68.5%

12.7%

2.8%

0.2%

0.4%

3.2%

Science Specific

Role

Yes

No

Missing

53

443

8

10.5%

87.9%

1.6%

Period Hired,

Science Specific

Role Only

December 2009 or earlier

January to June 2010

July to December 2010


July to December 2011


41

7

2

10

11

11

48.8%

8.3%

2.4%

11.9%

13.1%

13.1%


33/33

25

July to December 2012 2 2.4%

Period Hired,

Science Specific

Role Only

Pre December 2009 to December 2010

January 2011 to December 2012

50

34

59.5%

40.5%

Where is time

spent?

Direct Programming to youth

Teaching and preparing volunteers and partners

Teaching and preparing 4-H Staff

Developing curricula

Adapting current curricula

Creating science lessons and activities

Fund development

Evaluation

Developing programs

Developing partnerships

Missing

204

87

25

6

22

22

7

2

53

29

47

40.5%

17.3%

5.0%

1.2%

4.4%

4.4%

1.4%

0.4%

10.5%

5.8%

9.3%

Where SHOULD

time be spent?

Direct Programming to youth

Teaching and preparing volunteers and partners

Teaching and preparing 4-H StaffDeveloping curricula

Adapting current curricula

Creating science lessons and activities

Fund development

Evaluation

Developing programs

Developing partnerships

Missing

100

235

405

7

5

5

6

40

29

32

19.8%

46.6%

7.9%1.0%

1.4%

1.0%

1.0%

1.2%

7.9%

5.8%

6.3%

Attended National

Academy

Yes

No

Missing

49

431

24

9.7%

85.5%

4.8%

If yes, which track

did you attend?

Curriculum

Evaluation

Professional Development

Fund Development

8

18

14

6

17.4%

39.1%

30.4%

13.0%

If yes, what was

your role?

Attendee (non-presenter)

Presenter

Both

44

0

2

95.6%

0

4.4%

Attended

REGIONAL

Academy 2012

Yes

No

Missing

161

315

28

31.9%

62.5%

5.6%

If yes, what wasyour role?

Attendee (non-presenter)

Presenter

Both

115

1

44

71.9%

0.6%

27.5%

Documents

FINAL Needs Assessment Report