Evaluation Summary Report of NIMBioS Activities Year · PDF fileEvaluation Summary Report of NIMBioS Activities Year Three ... Tennessee Junior Science and Humanities Symposium

Evaluation Summary Report of NIMBioS Activities Year Three September 2010-April 2011 National Institute for Mathematical and Biological Synthesis April 2011

NIMBioS | Evaluation Summary of Major NIMBioS Activities 1

Evaluation Summary of Major NIMBioS Activities

Executive Summary

This is a report of NIMBioS evaluated activities during the third annual reporting period (RP 3) to the

National Science Foundation. The report covers the period of September 2010-April 2011. During RP 3,

567 participants (500 different people) from 262 institutions participated in NIMBioS sponsored

activities (see Diversity Report for details on participants). Research program activities during RP 3

included:

11 Working Groups (with a total of 12 meetings)

4 Investigative Workshops

20 Short-term visitors

10 Postdoctoral Fellows

2 Sabbatical Fellows

Education and outreach program activities during RP 3 included (see Annual Report for more details on

these events):

2 Tutorials

3 Biology in a Box Teacher Workshops

A NIMBioS Seminar Series

Tennessee Junior Science and Humanities Symposium

Research Experiences for Undergraduates/Veterinary Students Program

Sharing Adventures in Engineering and Science Program (SHADES)

Undergraduate Research Conference at the Interface of Biology and Mathematics

EcoED Webinar-- Math, Computing, Undergraduate Ecology Education and Large Datasets: An

example from a Citizen Science Program

Gadget Girls: Adventures in STEM

Teacher Collaboration Program

Other events included:

2 Advisory Board Meetings

By the end of August 2011 NIMBioS will have hosted 17 working group meetings, 9 Investigative

Workshops, 4 Tutorials, 26 Short-term Visitors, and 3 Sabbatical Fellows.


Of the 1,230 different participants to date, 288 have visited NIMBioS for more than one event (Figure 1).

Figure 1. Number of events per participant

NIMBioS conducted evaluations of its 12 working group meetings,4 workshops, 2 tutorials,

Undergraduate Research Conference at the Interface of Biology and Mathematics, EcoED Webinar, and

Research Experiences for Undergraduates/Veterinary Students programs. An evaluation of the pilot

Teacher Collaboration program is ongoing as well. Evaluations were carried out via electronic surveys

sent to all participants either after participation in a NIMBioS event, or both before and after

participation if a pre/post comparison of responses was warranted. Evaluation findings, along with

suggestions for improvement, were shared with event organizers, as well as NIMBioS staff as needed.

Improvements to program content and format, as well as NIMBioS’ overall operations, are made

accordingly. Following is a brief synopsis of the evaluations of NIMBioS’ major activities during RP 3.

942

161

77

28

4 45

1 1

7

1

10

100

1000

1 2 3 4 5 6 7 8 9 >10

Nu

mb

er

of

Par

tici

pan

ts

Number of Events Per Participant


Research Activities

Working Group and investigative workshop evaluation highlights are aggregated across all events in

their respective categories. Evaluations of working groups and workshops sought to answer the

following common questions:

1. Were participants satisfied with the event overall?

2. Did the event meet participant expectations?

3. Do participants feel the group made adequate progress toward their stated goals?

4. Do participants feel they gained knowledge about the main issues related to the research

problem?

5. Do participants feel they gained a better understanding of the research across disciplines related

to the group’s research problem?

6. What impact do participants feel participating in the event will have on their future research?

7. Were participants satisfied with the accommodations offered by NIMBioS?

8. What changes in accommodations, group format, and/or content would participants like to see

at future meetings?

Working Groups

NIMBioS Working Groups are chosen to focus on major scientific questions at the interface between

biology and mathematics that require insights from diverse researchers. The questions to be addressed

may be either fundamental, applied or both, and may be focused around a particular biological topic, or

one from mathematics that is driven by biological insight. NIMBioS is particularly interested in questions

that integrate diverse fields, require synthesis at multiple scales, and/or make use of or require

development of new mathematical/computational approaches.

Working Groups are relatively small (10-12 participants, with a maximum of 15), focus on a well-defined

topic and have well-defined goals and metrics of success (e.g., publications, database, software).

Selection of Working Groups is based upon the potential scientific impact and inclusion of participants

with a diversity of backgrounds and expertise that match the scientific needs of the effort. Organizers

are responsible for identifying and confirming participants with demonstrated accomplishments and

skills to contribute to the Working Group. Given this emphasis, working group activities rarely involve

recently-trained researchers such as postdocs and graduate students. Participation by international


researchers is encouraged; though generally there will not be more than 2-3 individuals from outside

North America in a Working Group. Working Groups typically meet 2-4 times over a two year period,

with each meeting lasting 3-5 days; however the number of participants, number of meetings, and

duration of each meeting is flexible, depending on the needs and goals of the Group. Plans can include

visits to NIMBioS for subsets of Working Group members to collaborate with NIMBioS IT staff and

researchers on Working Group needs.

First Meetings

During RP 3, NIMBioS hosted the first meetings of three working groups, with a total of 35 participants

(Table 1) (See http://www.nimbios.org/workinggroups/ for more details about specific working groups).

Evaluation surveys were sent to all participants, with the exception of working group organizers and

NIMBioS employees who were participating in the working groups. A total of 14 participants took part

in the evaluation of the first meetings of their working groups (the evaluation form for the Optimal

Control for Agent-based Models group was still open at the time of this report, and is not included in the

total or summary).

Table 1. Working Group First Meetings Hosted by NIMBioS

Title of Working Group Dates

#

Participants

Species Delimitation 12/2-4/10 12

Gene Tree Reconciliation 12/16-18/10 12

Optimal Control for Agent-based Models 4/26-28/11 11

Synopsis of First Meeting Evaluation Results

Overall satisfaction with first meetings was relatively high among survey respondents, the

majority of whom (86%) indicated they either agreed or strongly agreed that their respective

meetings were very productive and met their expectations.

75% of respondents thought the presentations were useful and 83% thought the presenters

were very knowledgeable about their topics.

The majority of respondents (93%) agreed that they had a better understanding of the main

issues related to their group’s research problem as a result of participation.

http://www.nimbios.org/workinggroups/


A large majority (93%) said they planned to take the knowledge they gained during the working

group and apply it to their own research.

64% of respondents reported they developed unanticipated plans for collaborative research

with other working group participants.

93% of respondents either agreed or strongly agreed that they would recommend participating

in NIMBioS working groups to their colleagues.

86% of respondents agreed that the format of their working group was very effective for

achieving the group’s goals.

88% of respondents felt their working group made adequate progress toward its stated goals for

the first meeting.

67% of respondents said they left their respective meetings with a good idea of what their

contribution will be at the next meeting.

Overall, respondents reported being satisfied with the travel, housing, and other amenities

provided by NIMBioS.

Second Meetings

During the reporting period, NIMBioS hosted the second meetings of four working groups, with a total

of 47 participants (Table 2). Evaluation surveys were sent to all participants, with the exception of

working group organizers and NIMBioS employees who were participating in the working groups. A

total of 18 participants took part in the evaluation of the second meetings of their working groups.

Table 2. Working Group Second Meetings Hosted by NIMBioS

Title of Working Group Dates # Participants

Synthesizing Predictive Modeling of Forest Insect Dynamics Across Spatial and Temporal Scales

10/18-21/10 14

Cross-Topology Registration 1/8-10/11 11

Food Web Dynamics 1/11-15/11 8

Modeling Bovine Tuberculosis 1/31-2/1/11 14


Synopsis of Second Meeting Evaluation Results

Overall satisfaction with the working group meetings was high among survey respondents,

100% of whom indicated they either agreed or strongly agreed that their meeting was very

productive and met their expectations.

100% of respondents thought the presentations were useful and that the presenters were very

knowledgeable about their presentation topics.

100% of respondents agreed that participating in the meeting increased their understanding of

the work being done in by others in the group, and that it increased their understanding of how

everyone’s work would come together to achieve the goals of the group.

All respondents said that participating in their working group had influenced their research

agendas. Several participants noted that the group had lead to collaborations that otherwise

may not have occurred.

100% of respondents agreed that the format of their working group was very effective for

achieving its goals.

Most respondents (85%) felt that their working group made adequate progress toward reaching

its intended goals.

Most (85%) respondents said they left their second meeting with a good idea of what their

contribution would be at the next meeting.



Third Meetings

During the reporting period, NIMBioS hosted the third meetings of four working groups, with a total of

40 participants (Table 3). Evaluation surveys were sent to all participants, with the exception of working

group organizers and NIMBioS employees who were participating in the working groups. A total of 21

participants took part in the evaluation of the second meetings of their working groups (the evaluation

form for the Synthesizing Predictive Modeling of Forest Insect Dynamics Across Spatial and Temporal

Scales group was still open at the time of this report, and is not included in the total or summary). .


Table 3. Working Group Third Meetings Hosted by NIMBioS


Integrating Functional and Evolutionary Dynamics at Multiple Scales 9/13-17/10 9

Coalitions and Alliances 11/4-6/10 9

Population and Community Ecology Consequences of Intraspecific Niche Variation

2/10-12/11 14

Synthesizing Predictive Modeling of Forest Insect Dynamics Across Spatial and Temporal Scales

4/14-20/11 8

Synopsis of Third Meeting Evaluation Results

All respondents indicated they were very satisfied with their working groups overall.

95% of respondents indicated being satisfied with the diversity of disciplinary expertise of their

working group’s participants.

All respondents felt their groups were making adequate progress toward their goals.

85% of respondents agreed that participating in the working group meeting increased their

understanding of the work being done in by others in the group, and that they had a better

understanding of how everyone’s work would come together to achieve the goals of the group.

All respondents said they felt the expectations for the next working group meeting were clear, in

the sense that they were leaving this meeting with a good idea of what they needed to

accomplish before the next meeting.

Respondents indicated the greatest differences between their collaborations within the working

group and their other research collaborations were in the disciplinary topics involved, research

methods used, and scientific questions addressed.

Fourth Meetings

During the reporting period, NIMBioS hosted the fourth meeting of one working group, with a total of 5

participants (Table 4). Evaluation surveys were sent to all participants, with the exception of working

group organizers and NIMBioS employees who were participating in the working groups. A total of 3

participants took part in the evaluation of the second meetings of their working groups. Because of the

small number of respondents in this group, a summary of results will not be provided.


Table 4. Working Group Fourth Meetings Hosted by NIMBioS


Biological Problems using Binary Matrices 12/14-17/10 5

Investigative Workshops

NIMBioS Investigative Workshops differ from working groups in that they focus on a broader topic or set

of related topics at the interface of biology and mathematics and have relatively large size (30-40

participants). Workshops attempt to summarize/synthesize the state of the art and identify future

directions, and they have potential for leading to one or more future working groups. Organizers invite

15-20 key participants, and the remaining 15-20 participants are filled through open application from

the scientific community.

NIMBioS hosted four investigative workshops during RP 3, with a total of 129 participants (Table 5).

Evaluation surveys were sent to participants of all workshops, with the exception of workshop

organizers and NIMBioS employees who were participating in the workshops. A total of 75 participants

took part in the evaluation of the workshops (the evaluation form for the Synchrony in Biological

Systems Across Scales was still open at the time of this report, and is not included in the total or

summary).

Table 5. Investigative Workshops Hosted at NIMBioS

Title of Workshop Dates # Participants

Modeling Wildlife Zoonoses 11/8-10/10 45

Solid Tumor Modeling 1/19-21/11 38

Modeling Infectious Disease 1/23-25/11 12

Synchrony in Biological Systems Across Scales 4/11-13/11 34

Synopsis of Workshop Evaluation Results

Overall satisfaction was high among survey respondents, the majority of whom (93%) indicated

they either agreed or strongly agreed that their workshop was very productive. Most (92%) also

agreed it met their expectations.


Almost all respondents thought the presentations were useful (97%,) and all felt that the

presenters were very knowledgeable about their topics.


in NIMBioS workshops to their colleagues.

Most respondents (98%) felt that participating in their workshops helped them to better

understand the research going on in other disciplines regarding the research problems.

Many (62%) said the exchange of ideas that took place during their workshop would influence

their future research, while 31% said it would “possibly” influence their research.

31% of respondents said they developed plans for collaborative research with other workshop

participants, while 30% said they would “possibly” collaborate with other participants.

The majority of respondents (90%) agreed that they had a better understanding of the main

issues related to their workshop’s research problem as a result of participation.

90% of respondents agreed that the format of their workshop was very effective for achieving

its goals.

85% of respondents felt the workshop made adequate progress toward finding a common

language across disciplines for research on their workshop’s topic.

A large majority (92%) said they were satisfied with the opportunities provided during the

workshop presentations and discussions to ask questions and/or make comments.



Education and Outreach Program Activities

Tutorials

NIMBioS hosted two tutorials during RP 3, with a total of 76 participants (Table 6). Evaluation surveys

were sent to all participants, with the exception of tutorial organizers and NIMBioS affiliates who were

participating in the tutorials. A total of 56 participants took part in the evaluation of the tutorials.

Table 6. Tutorials Hosted at NIMBioS

Title of Tutorial Dates # Participants

Phylogenetics Tutorial 10/13-15/10 39

Stochastic Tutorial 3/15-18/11 37


The evaluation of the tutoriasl sought to answer the following questions:

1. Were participants satisfied with the tutorial overall?

2. Did the tutorial meet participant expectations?

3. Was the tutorial appropriate to the participants’ levels of expertise?

4. Did participants feel they learned an appropriate amount of information?

5. How did participants feel about the amount of content and format of the tutorial?



at future similar meetings?

Synopsis of Tutorial Evaluation Results


in NIMBioS tutorials to their colleagues.

78% of respondents agreed the tutorials met their expectations and 76% agreed they were

appropriate to their level of expertise.

98% of respondents thought the instructors were knowledgeable about their topics, and 91%

thought the presentations were useful.

78% of participants thought the hands-on exercises were useful, while a smaller majority felt the

group discussions were useful (73%).

89% of respondents agreed that the formats of the tutorials were very effective for achieving its

goals.

49% of participants indicated they felt the amount of content offered during the tutorial was

“just right,” while 38% felt there was too much material presented for the allotted time.

All participants indicated having a better understanding of tutorial content result of attending

the tutorials.

A large majority (95%) said they were satisfied with the opportunities provided during the

tutorial to ask questions and/or make comments.




Research Experiences for Undergraduates/Veterinary Students

The NIMBioS Research Experiences for Undergraduates (REU) and Research Experiences for Veterinary

Students (REV) programs took place simultaneously on the University of Tennessee, Knoxville (UT)

campus June 7-July 30, 2010. Thirteen undergraduates, three veterinary students and two high school

teachers from 15 different institutions across the United States were chosen to participate in the

program.

During the eight-week long program, participants lived on campus at the University of Tennessee,

Knoxville, (UT) and worked in teams with UT faculty to conduct research at the interface of mathematics

and biology. The award included a stipend, housing and some funding to support travel.

Research topics for the 2010 program were modeling the effects of climate change on ant foraging

behavior in the Great Smoky Mountains National Park; predicting the park's biodiversity; modeling

Johne's disease in cattle; modeling the growth and development of plant pathogens; and modeling the

dynamics of cat populations in the community. Mentors in the program included UT professors Suzanne

Lenhart (Professor in Applied Mathematics, Associate Director of NIMBioS), Shigetoshi Eda (Center for

Wildlife Health, Department of Forestry, Wildlife and Fisheries), Paul Armsworth (Assistant Professor,

Department of Ecology and Evolutionary Biology), Steven Wise, (Assistant Professor, Department of

Mathematics), Kim Gwinn (Associate Professor, Entomology and Plant Pathology), and John New

(Professor, Department Head, Comparative Medicine). NIMBioS postdoctoral fellows William Godsoe,

and Sharon Bewick also served as mentors. Evaluation surveys were sent to all participants in the

program, with the exception of mentors. All 18 participants took part in the evaluation.

The evaluation of the REU/REV sought to answer the following questions:

1. Were participants satisfied with the program overall?

2. Did the research experience meet participant expectations?

3. Did the research experience impact participant plans to go to graduate school?

4. To what extent did participants increase their research skills during the program?

5. To what extent do participants feel they gained knowledge about the research process?

6. How satisfied were participants with their mentors?

7. How satisfied were participants with the accommodations offered by NIMBioS?

8. What changes do participants feel NIMBioS should make in the program for next year?


Synopsis of REU/REV Evaluation Results

Overall satisfaction with the program was high among participants, 94% of whom said they were

“satisfied” or “very satisfied” with their experiences and would recommend the program to

others.

Fifteen of the eighteen participants said most or all of their expectations were met or exceeded

during the program, while three participants said only “some” of their expectations were met.

The majority of participants (89%) thought the overall workload during the program was “just

about right,” while 11% thought either “too much” or “too little” work was assigned.

Overall, participants were highly satisfied with their mentors, indicating that they were very

helpful and supportive during the research experience. Participants rated their mentors highly,

with the average mentor rating at 1.71 (on a scale of -2 to 2, with 2 being the most favorable).

Participants rated the usefulness of the BCMB information sessions highly as well. Ratings

ranged from an average of 0.11 for the “Diversity in Science” session, to an average of 0.44 for

the “Speaking to a Professional Audience” session (on a scale of -1=not useful to 1=very useful).

Gains in several research-related skills were reported by participants, with an average rating for

all skills of 0.75 on the pre-survey and 1.07 on the post-survey (on a 5-point Likert scale from -

2=extremely poor at the skill to 2=excellent at the skill).

Participants reported gains in knowledge regarding several research-related topics. Before the

program, participants on average rated themselves 0.55 on a 5-point Likert scale from

-2=extremely poor understanding to 2=excellent understanding of the topics. After

participation, the average rating was 1.28.

Sixty-seven percent of participants said that participating in the program impacted their plans to

go to graduate school in some way. Some participants said that the experience reinforced their

previous decisions to attend graduate school, while others said that the experience made them

more interested in integrating math and biology into their graduate school plans.

Undergraduate Research Conference at the Interface of Biology and Mathematics (URC)

The NIMBioS second annual Undergraduate Research Conference at the Interface of Biology and

Mathematics took place at the University of Tennessee's Conference Center in downtown Knoxville

November 19-20, 2010. The event was organized by the NIMBioS Education and Outreach Associate


Director for Education, Outreach, and Diversity, Suzanne Lenhart, and the Education and Outreach

Coordinator Kelly Moran.

Nearly 130 undergraduate students and faculty mentors participated from more than 30 institutions

throughout the United States. Dr. Abdul-Aziz Yakubu, Professor and Chair of the Department of

Mathematics from Howard University, kicked off the conference with a plenary talk on fish population

dynamics. Students contributed more than 60 oral and poster presentations to the program on topics in

ecology, evolution, disease, biomedical applications and genetics. Participants learned about

educational and professional opportunities at NIMBioS and also at NESCent, the National Evolutionary

Synthesis Center, located in Durham, NC. A panel discussion featuring professionals in mathematical

ecology and biology, bioinformatics and computational biology fielded questions on graduate school and

career advice. Evaluation surveys were sent to all participants in the program, with the exception of

NIMBioS affiliates and event organizers. A total of 102 participants took part in the evaluation.

The evaluations for the conference sought to answer the following questions:

1. Were participants satisfied with the conference overall?

2. Did the conference meet participant expectations?

3. Do participants feel the conference allowed them to make new connections with others in math

and biology?

4. Do participants feel they gained a better understanding of undergraduate research happening at

the interface of mathematics and biology?

5. What impact do undergraduate participants feel the conference will have on their future career

plans?



at future similar meetings?

Synopsis of URC Conference Evaluation Results

Overall satisfaction with the conference was high among respondents, the majority of whom

indicated they either agreed or strongly agreed that the conference was productive (89%) and

met their expectations (85%).

Most respondents thought the presentations were useful (81%), while a slightly smaller majority

felt the panel discussion was useful (79%).


Ninety-two percent of respondents either agreed or strongly agreed that they would

recommend participating in NIMBioS conferences to their colleagues.

Overall, respondents reported being satisfied with the conference accommodations provided by

NIMBioS.

Respondents reported high levels of learning about how to present scientific research and

career opportunities available at the interface of mathematics and biology.

Most respondents felt the most useful aspect of the conference was the career panel, followed

by the poster session and networking opportunities.

Ninety-eight percent of undergraduate respondents said they felt that participating in the

conference helped them become more knowledgeable about undergraduate research going on

at the interface of biology and math.

The majority of respondents (89%) agreed that the conference was successful in achieving its

goal of creating a forum through which undergraduates could present research and make new

connections at the interface of math and biology.

Twenty-five percent of undergraduate respondents said they felt that the exchange of ideas that

took place during the conference would (or potentially would) influence their career plans, while

44% said it “possibly” would influence their career plans.

The majority of respondents (99%) said they felt that participating in the conference helped

them make connections with others within the interdisciplinary field of math and biology.

EcoEd Webinar-- Math, Computing, Undergraduate Ecology Education and Large Datasets: An

example from a Citizen Science Program

A 45-minute webinar entitled “Math, Computing, Undergraduate Ecology Education and Large Datasets:

An Example from a Citizen Science Program” was presented by NIMBioS Director Louis Gross and

Postdoctoral Fellow William Godsoe on September 8, 2010. The webinar, hosted via LiveOnline@UT,

was offered as one of several events prior to the Oct. 14-15 2010 Ecology and Education Summit,

sponsored by ESA and the National Education Association in Washington, D.C.

The webinar focused on math and computational education for ecology undergrads and illustrated how

a large field dataset can be used to motivate hypothesis formulation and assessment by

undergraduates. This included a discussion of NIMBioS’ Research Experience for Undergraduates (REU)

program linking biology and math undergrads; discussion of a large citizen science project based in the

Great Smoky Mountains National Park (Discover Life in America's All Taxa Biodiversity Inventory – ATBI);


description of one ATBI large biodiversity dataset and how a small group of undergrads chose what to

analyze and how to do so; and interview comments from the REU students about their learning process.

The presenters emphasized how the data and analysis involved a multiplicity of concepts of biodiversity,

the variety of questions raised by the students and the constraints on addressing them using the

available data.

The evaluation of the webinar sought to answer the following questions:

1. Did participants find the information presented in the webinar useful?

2. Did the webinar meet participants’ expectations?

3. Did participants felt they learned about the central topics of the webinar?

4. Did participants feel the presenter adequately addressed audience questions?

5. Were there any technical problems with the format of the webinar?

6. What topics would participants have liked to cover if given more time?

7. What topics would participants like to cover at future webinars?

Synopsis of Webinar Evaluation Results

Participant expectations for the webinar included learning how to use large datasets in

undergraduate ecology classes and how to incorporate citizen science projects into the

classroom.

64% of respondents felt as though the webinar met their expectations.

The majority of respondents (52%) said they felt that participating in the webinar helped them

better understand the importance of mathematics in undergraduate biology education.

97% of respondents indicated they felt sufficient opportunity was given for questions and

comments from the audience, and that the questions from the audience were answered well.

40% of respondents indicated having some sort of technical experience while accessing the

webinar. The most common complaint dealt with audio problems.

If given more time, participants would have liked to have gotten more specific information

about how to translate the REU example into a project for a large undergraduate ecology

classroom.

Other suggestions for future webinar topics included information about other available datasets

available for classroom use, statistics for undergrad teaching, and how to overcome differences

in mathematics skills of undergrad biology students.

Documents

Evaluation Summary Report of NIMBioS Activities Year · PDF fileEvaluation Summary Report of NIMBioS Activities Year Three ... Tennessee Junior Science and Humanities Symposium