Preparation “A Glass Half Full”

PreparationPreparation“A Glass Half Full”“A Glass Half Full”

Please organize yourselves into Please organize yourselves into groups of 5-6.groups of 5-6.

Note cards for comments will be Note cards for comments will be distributed prior to the beginning of distributed prior to the beginning of the session and are on your tables.the session and are on your tables.

Please select a different recorder for Please select a different recorder for each group activity.each group activity.

A Glass Half Full: A Glass Half Full: CETP Core EvaluationCETP Core Evaluation

Frances LawrenzFrances Lawrenz

Karen AppeldoornKaren Appeldoorn

University of MinnesotaUniversity of Minnesota

What are the various sources of What are the various sources of Core data and what are their Core data and what are their strengths and weaknesses?strengths and weaknesses?

Briefly discuss the above question. Briefly discuss the above question. Record your responses on note Record your responses on note

cards. cards. We will pick them up from each We will pick them up from each

group.group.

Data Sources Used by the CoreData Sources Used by the Core

Higher Education: PIs or Evaluators, Higher Education: PIs or Evaluators, Deans or Department Chairs, STEM Deans or Department Chairs, STEM and STEM Education Faculty, and STEM Education Faculty, Students, NSF Student Scholars Students, NSF Student Scholars

K-12: Principals, CETP and NonCETP K-12: Principals, CETP and NonCETP Teachers, CETP and NonCETP Teachers, CETP and NonCETP Students, NSF Graduated ScholarsStudents, NSF Graduated Scholars

Both: Classroom Observers, Both: Classroom Observers, Classroom Activities, Student Classroom Activities, Student AssessmentsAssessments

Make and record predictions about the Make and record predictions about the following – we will return to these at the following – we will return to these at the

end of the presentation:end of the presentation: How often did HE faculty think students worked How often did HE faculty think students worked

on real-world issues?on real-world issues? How helpful did HE students find doing data How helpful did HE students find doing data

collection and analysis?collection and analysis? How often did K-12 teachers believe students How often did K-12 teachers believe students

had enough time to learn what was required? had enough time to learn what was required? How helpful did K-12 students find completing How helpful did K-12 students find completing

portfolios?portfolios? What % of Deans do you think said their faculty What % of Deans do you think said their faculty

had formalized interaction with K-12?had formalized interaction with K-12? What did PIs say was the most important What did PIs say was the most important

outcome?outcome?

What is your definition of What is your definition of standards-based reform?standards-based reform?

Determine a definition in your group.Determine a definition in your group. Write your definition on a note card.Write your definition on a note card. We will collect the cards and ask for We will collect the cards and ask for

a show of hands on selected items.a show of hands on selected items.

What is standards-based reform?What is standards-based reform?

Standards-based reform: identifying Standards-based reform: identifying challenging academic standards for challenging academic standards for what all students should know and what all students should know and be able to do and aligning other be able to do and aligning other aspects of the system, such as aspects of the system, such as testing, accountability, teacher testing, accountability, teacher certification, and professional certification, and professional development, with the new development, with the new standardsstandards

National Teaching StandardsNational Teaching Standards

Science and MathematicsScience and Mathematics SCIENCE AND MATH STANDARDS: Inquiry-SCIENCE AND MATH STANDARDS: Inquiry-

based program: Select content to meet based program: Select content to meet interests, knowledge, understanding, interests, knowledge, understanding, abilities and experience of studentsabilities and experience of students

Student-centered rather than teacher-Student-centered rather than teacher-centered teaching strategies are implicit in centered teaching strategies are implicit in the National Teaching Standardsthe National Teaching Standards

National Teaching StandardsNational Teaching Standards

SCIENCE AND MATH STANDARDS: Engage SCIENCE AND MATH STANDARDS: Engage in ongoing assessment of teaching and of in ongoing assessment of teaching and of student learning: Use multiple methods student learning: Use multiple methods and systematically gather data about and systematically gather data about student understanding and abilitystudent understanding and ability

Improving TeachingImproving Teaching

86% Education faculty said they observed 86% Education faculty said they observed colleagues teaching colleagues teaching

67% STEM faculty reported observing 67% STEM faculty reported observing colleagues teachingcolleagues teaching

Most often, observations were done Most often, observations were done because of interest in improving one’s own because of interest in improving one’s own teachingteaching

49% say their courses influenced changes 49% say their courses influenced changes in other coursesin other courses

About 50% of the faculty felt that half of About 50% of the faculty felt that half of their colleagues were engaged in reformtheir colleagues were engaged in reform

Comments about InstructionComments about Instruction ““I believe my dept has crossed an important I believe my dept has crossed an important

threshold thanks in part to CETP. Reformed threshold thanks in part to CETP. Reformed instruction is now the norm and we have instruction is now the norm and we have reached a critical number of faculty who use reached a critical number of faculty who use reformed instruction. We hire new faculty reformed instruction. We hire new faculty based in part on their understanding of based in part on their understanding of reforms and their willingness to teach science reforms and their willingness to teach science in a reformed manner.” in a reformed manner.” Dean/Dept ChairDean/Dept Chair

““There is more disposition on the part of the There is more disposition on the part of the faculty to use constructivist approaches in faculty to use constructivist approaches in their teaching, as well as to integrate their teaching, as well as to integrate information technologies in their courses.” information technologies in their courses.” Dean/Dept ChairDean/Dept Chair

HE Teaching ActivitiesHE Teaching Activities

Most activities were used between Most activities were used between SeldomSeldom and and OccasionallyOccasionally

Most common activities were real world Most common activities were real world problems, writing descriptions of problems, writing descriptions of reasoning, making models and using reasoning, making models and using technology to communicatetechnology to communicate

Students thought most Students thought most helpfulhelpful activities activities were students working on real-world were students working on real-world problems, students using or making problems, students using or making models, and using technology to models, and using technology to understand concepts taught in class.understand concepts taught in class.

Time Spent and Time Spent and Helpfulness of Standards-Helpfulness of Standards-

Based HE Teaching Based HE Teaching PracticesPractices

Survey Item Frequency Faculty

Weighted Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Students worked on problems related to real-world or

practical issues.2.69 3.15 2.61

Students wrote descriptions of their reasoning.

2.67 3.02 2.42

Students used or made models. 2.64 2.95 2.61

Students made connections to other fields. (STEM and non-

STEM)2.57 2.85 2.45

Students evaluated the extent of their own learning.

2.32 2.88 2.40


Weighted Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Students performed investigative activities that

included data collection and analysis.

2.57 2.90 2.44

Students designed and made presentations that helped them

learn class concepts.2.57 2.64 2.46

Students participated in whole-class discussions during which

the instructor talked less than the students.

2.46 2.63 2.31

Students worked with other students where the whole group

got one grade.2.37 2.43 2.31


Weighted Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Student assessment results were used to modify what was taught

and how.2.76 2.78 2.39

New information was based on what students already knew

about the topic.2.67 2.91 2.44

Students had a voice in decisions about course

activities.2.14 2.74 2.43


Weighted Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Students completed assessments/assignments that

included problems with complex solutions.

2.53 3.04 2.31

Students completed assessments/assignments that included multiple choice/short

answer items.

2.37 2.65 2.29


included portfolios.2.33 2.19 2.28


included full-length papers/reports.

2.28 2.41 2.15


Weighted Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Students used technology as a tool to communicate with their

instructor.2.72 2.47 2.51

Students used technology to understand or explore in more

depth concepts taught in class.

2.59 2.77 2.53

Students used technology as a tool in investigations to gather

and analyze scientific or mathematical data.

2.48 2.74 2.50

Students used technology as a tool for assessment.

2.26 2.58 2.40

Capsule Description of the Quality of the Lesson (HE)

Level: Description of LevelCETP

N %

Level 1: Ineffective Instruction 2 5.3

Level 2: Elements of Effective Instruction 3 7.9

Level 3: Beginning Stages of Effective Instruction 14 36.8

1. Low 5 35.7

2. Solid 3 21.4

3. High 6 42.9

Level 4: Accomplished, Effective Instruction 13 34.2

Level 5: Exemplary Instruction 6 15.8

Most Important Outcomes (PI)Most Important Outcomes (PI)

38% said increased collaboration 38% said increased collaboration among higher education institutionsamong higher education institutions

31% reported improved interaction 31% reported improved interaction between STEM and education between STEM and education departmentsdepartments

31% saw improved interaction 31% saw improved interaction between higher education and K-12 between higher education and K-12 schoolsschools

Faculty Reporting Faculty Reporting

73% said more interaction with faculty 73% said more interaction with faculty from other institutions from other institutions

72% STEM faculty reported more STEM 72% STEM faculty reported more STEM and education dept interactionand education dept interaction

83% Education faculty found more STEM 83% Education faculty found more STEM and education dept interactionand education dept interaction

46% faculty said they have formalized 46% faculty said they have formalized interaction with K-12 schoolsinteraction with K-12 schools

32% reported field based experiences in K-32% reported field based experiences in K-1212

To what extent has CETP Improved To what extent has CETP Improved (4=substantial):(4=substantial):

STEM faculty interactions about STEM faculty interactions about instruction, mean=3.9instruction, mean=3.9

STEM and Education faculty STEM and Education faculty interactions, mean=3.8interactions, mean=3.8

2yr and 4 yr college interactions, 2yr and 4 yr college interactions, mean=2.6mean=2.6

Faculty and k-12 interactions, Faculty and k-12 interactions, mean=3.7mean=3.7

Comments about InteractionComments about Interaction ““The funding has tremendously encouraged The funding has tremendously encouraged

collaboration. I will never go back to working collaboration. I will never go back to working alone.” alone.” Faculty MemberFaculty Member

““We have much greater interaction with We have much greater interaction with mathematics and science faculty from other mathematics and science faculty from other institutions. CETP is the cause of it.” institutions. CETP is the cause of it.” Faculty Faculty MemberMember

““We visit colleagues in similar programs at other We visit colleagues in similar programs at other institutions and we communicate more, in person institutions and we communicate more, in person and by email, with colleagues in other and by email, with colleagues in other departments at my home institution. This is a departments at my home institution. This is a direct consequence of the CETP.” direct consequence of the CETP.” Faculty MemberFaculty Member

Deans/Department ChairsDeans/Department Chairs

53% reported changes in interaction 53% reported changes in interaction with faculty in other institutionswith faculty in other institutions

86% said their faculty had formalized 86% said their faculty had formalized interaction with K-12interaction with K-12

77% said classes in their depts. had 77% said classes in their depts. had field based experiencesfield based experiences

42% said their depts. offered special 42% said their depts. offered special programs to increase diversityprograms to increase diversity

Comments about InteractionComments about Interaction

““The two year colleges were virtually The two year colleges were virtually unknown prior to CETP. Now they unknown prior to CETP. Now they are a part of the mix when are a part of the mix when discussions, roundtables, seminars or discussions, roundtables, seminars or professional development workshops professional development workshops are organized.” are organized.” Dean/Dept ChairDean/Dept Chair

Percent of Time Spent on Teaching and/or Curriculum ReformTwo-Year Four-Year

STEM Education STEM Education

N % N % N % N %

1 (<10%) 1 3.8

2 (10-19%) 1 3.8

3 (20-29%) 2 2.7

4 (30-39%) 5 6.8 1 3.8

5 (40-49%) 1 6.3 7 9.5 1 3.8

6 (50-59%) 6 8.1 4 15.4

7 (60-69%) 1 6.3 1 50.0 11 14.9 2 7.7

8 (70-79%) 1 6.3 19 25.7 9 34.6

9 (80-89%) 5 31.3 1 50.0 7 9.5 5 19.2

10 (90-100%) 8 50.0 17 23.0 2 7.7

Weighted Mean/SD

9.06/1.39 8.50/1.11 7.65/1.96 7.27/2.07

Value of TeachingValue of Teaching

84% dept chairs rated teaching as 84% dept chairs rated teaching as very important in hiring decisionsvery important in hiring decisions

89% dept chairs said promotion and 89% dept chairs said promotion and tenure included work on instructional tenure included work on instructional improvementimprovement

34% 4yr STEM faculty found teaching 34% 4yr STEM faculty found teaching more valued than research more valued than research

59% 4 yr Education faculty reported 59% 4 yr Education faculty reported teaching more valued than researchteaching more valued than research

Comments about K-12 InteractionComments about K-12 Interaction

““At the same time new courses were At the same time new courses were developed and implemented, teachers developed and implemented, teachers from the School District served as from the School District served as ‘critical friends’ in observing college ‘critical friends’ in observing college teaching.” teaching.” Dean/Dept ChairDean/Dept Chair

““STEM faculty are working between STEM faculty are working between disciplines to garner funds and develop disciplines to garner funds and develop innovative preservice programming. innovative preservice programming. They are creating linkages with K-12 They are creating linkages with K-12 schools as never before.” schools as never before.” Dean/Dept Dean/Dept ChairChair

Relationships with Higher Relationships with Higher EducationEducation

74% of the principals said their 74% of the principals said their schools have ongoing relationships schools have ongoing relationships with institutions of higher educationwith institutions of higher education

56% found these relationships have 56% found these relationships have not changed but all but one comment not changed but all but one comment about them was positiveabout them was positive

47% of teachers reported having 47% of teachers reported having relationships with higher educationrelationships with higher education

CETP teachers reported more CETP teachers reported more relationshipsrelationships

Teacher Preparation Programs: Teacher Preparation Programs: Principal ViewPrincipal View

90% of principals believed teacher 90% of principals believed teacher preparation programs make a preparation programs make a differencedifference

Components of high quality Components of high quality programs were: programs were: standards-based practices, standards-based practices, strong content knowledge base, strong content knowledge base, understanding of child development and understanding of child development and

behaviorbehavior

Barriers to ExcellenceBarriers to Excellence

62% of principals reported barriers to 62% of principals reported barriers to excellenceexcellence

58% of teachers reported barriers58% of teachers reported barriers Lack of resources was the major Lack of resources was the major

barrierbarrier CETP teachers perceived more CETP teachers perceived more

barriers than nonCETP—perhaps they barriers than nonCETP—perhaps they have higher expectationshave higher expectations

Principal Ratings of Principal Ratings of Importance of Teaching Importance of Teaching

StrategiesStrategies

N %

Students writing lengthy descriptions of their reasoning.

1. Not very important 1 .8

2. Somewhat important 3 2.4

3. Important 31 25.2

4. Very important 61 49.6

5. Extremely important 27 22.0

Weighted Mean/SD 3.93/.76

Investigative activities that include data collection and analysis.

1. Not very important

2. Somewhat important 1 .8





Whole-class discussion during which the teacher talks less than the students.







N %

Students working in groups; receiving one grade per group.

1. Not very important 11 9.0





Weighted Mean/SD 3.08/1.04

Presentation of new information that is based on students' prior knowledge.







Students gathering information to answer their own questions.







N %

Using computers to support deep conceptual understanding.







Using a variety of assessment techniques: multiple choice tests, portfolios, projects, etc.


2. Somewhat important





Use of national science or mathematics standards.







CETP vs. NonCETP K-12 TeachersCETP vs. NonCETP K-12 Teachers

CETP teachers rated their preparation CETP teachers rated their preparation programs higher than nonCETP teachersprograms higher than nonCETP teachers

CETP teachers were more likely to say CETP teachers were more likely to say they had received instruction on they had received instruction on standardsstandards

CETP teachers attended more CETP teachers attended more professional meetings professional meetings

CETP teachers were more likely to view CETP teachers were more likely to view themselves as facilitators than nonCETPthemselves as facilitators than nonCETP

K-12 Teaching ActivitiesK-12 Teaching Activities

Most used between seldom and Most used between seldom and occasionally occasionally

Most common were activities that Most common were activities that include data collection and analysis and include data collection and analysis and that the students have enough timethat the students have enough time

CETP teachers and students reported CETP teachers and students reported using the techniques more than using the techniques more than nonCETPnonCETP

NonCETP students rated techniques as NonCETP students rated techniques as more helpful than CETP studentsmore helpful than CETP students

Time Spent and Time Spent and Helpfulness of Standards Helpfulness of Standards

Based K-12 Teaching Based K-12 Teaching PracticesPractices

Survey Item Frequency Teacher Weighted

Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Students worked on problems related to real-world or

practical issues.3.29 2.75 1.89* -

Students determined how much they knew about something.

3.18 2.97 1.94* -

Students made connections to other fields. (STEM and non-

STEM)3.03 2.36 1.42* -

Students used or made models, e.g., physical, conceptual or

mathematical.2.98* 2.43* 1.78* -

Students wrote about why they thought something.

2.76* 2.45* 1.52* -


Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Students did activities that included data collection and

analysis.2.97* 3.01* 2.06

Students participated in whole-class discussions during which the teacher talked less than the

students.

2.61* 2.34 1.39

Students designed and made presentations that helped them

learn class concepts.2.53* 2.40* 1.63* -

Students worked with other students where the whole group

got one grade.2.41* 2.50 1.75


Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Student assessment results were used to modify what was taught

and how.3.61

Not a student item

Not a student item

Students had enough time to learn what was required.

3.44 3.19 2.15* -

New information was based on what students already knew

about the topic.3.36 2.85 2.01* -

Students had a voice in decisions about class activities.

2.83 2.53* 1.70


Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Students completed assessments/assignments that included multiple choice/short

answer items.

3.22 2.96* 2.06* -

Students completed assessments/assignments that included problems with complex

solutions.

3.13* - 2.97 1.89* -


included full-length papers/reports.

2.21* 2.33* 1.38* -


included portfolios.1.92* 1.99* 1.06* -


Mean (1-4)

Frequency Student

Weighted Mean (1-4)

Helpfulness Student

Weighted Mean (0-3)

Students used technology to understand or explore in more depth concepts already taught

in class.

2.84 2.65 1.97* -

Students used technology as a tool for checking understanding

(assessment).2.75* - 2.60* 1.85* -

Students used technology as a tool in investigations to gather

and organize information.2.57* 2.72* 1.97

Students used technology as a tool to communicate with their instructor or other students.

2.15* 2.00 1.32* -

Capsule Description of the Quality of the Lesson (K-12)

Level: Description of LevelCETP NonCETP

N % N %

Level 1: Ineffective Instruction 10 13.3 6 11.8

Level 2: Elements of Effective Instruction 13 17.3 12 23.5

Level 3: Beginning Stages of Effective Instruction

26 34.7 20 39.2

1. Low 10 29.4 9 39.1

2. Solid 11 32.4 8 34.8

3. High 13 38.2 6 26.1

Level 4: Accomplished, Effective Instruction 14 18.7 11 21.6

Level 5: Exemplary Instruction 12 16.0 2 3.9

How important would the How important would the data sources rate basing data sources rate basing new information on what new information on what students already know students already know

about a topic? about a topic?

New information is based on what New information is based on what students already know about a students already know about a

topic:topic: The majority of Principals rated this as The majority of Principals rated this as

Very ImportantVery Important The majority of K-12 Teachers said it The majority of K-12 Teachers said it

happens happens RegularlyRegularly The majority of 6-12 Students said it The majority of 6-12 Students said it

happens happens OccasionallyOccasionally The majority of Faculty say it happens The majority of Faculty say it happens

RegularlyRegularly The majority of College Students say it The majority of College Students say it

happens happens OccasionallyOccasionally

Multiple AssessmentsMultiple Assessments

Principals think the use of multiple Principals think the use of multiple assessments (such as portfolios and assessments (such as portfolios and multiple choice items) is multiple choice items) is Very Very ImportantImportant

However, in regards to portfolios:However, in regards to portfolios: Faculty – Faculty – SeldomSeldom College Students – College Students – NeverNever K-12 Teachers – K-12 Teachers – NeverNever 6-12 Students – 6-12 Students – NeverNever

Understanding science/math requires special Understanding science/math requires special skills that only some possess.skills that only some possess.

(Scaled: 1-4, 1-->Strongly Disagree)(Scaled: 1-4, 1-->Strongly Disagree)

Higher Ed. (Weighted Mean/SD)Higher Ed. (Weighted Mean/SD)

Four-YearFour-Year

STEMSTEM EducationEducation

Faculty on ScienceFaculty on Science 1.51/.641.51/.64 1.25/.491.25/.49

Faculty on MathFaculty on Math 1.66/.671.66/.67 1.26/.491.26/.49

Students on ScienceStudents on Science 2.39/.832.39/.83 2.00/.792.00/.79

Students on MathStudents on Math 2.44/.852.44/.85 2.03/.822.03/.82

K-12 (Weighted Mean/SD)K-12 (Weighted Mean/SD)

CETPCETP NonCETPNonCETP

K-12 Teachers on K-12 Teachers on Science*Science* 1.71/.641.71/.64 1.90/.611.90/.61

K-12 Teachers on K-12 Teachers on Math*Math* 1.78/.611.78/.61 1.97/.621.97/.62

6-12 Students on 6-12 Students on ScienceScience 2.36/.902.36/.90 2.29/.912.29/.91

6-12 Students on 6-12 Students on MathMath 2.51/.932.51/.93 2.43/.952.43/.95

What Students Thought About Science and Mathematics

CETP NonCETP

N % N %

Mathematics is boring.

1. Disagree 788 60.3 378 61.5

2. Agree 519 39.7 237 38.5

Mathematics is important to everyone's life.

1. Disagree 205 15.5 102 16.7

2. Agree 1115 84.5 510 83.3

Science is boring.*

1. Disagree 920 70.7 396 65.0

2. Agree 382 29.3 213 35.0

Science is important to everyone's life.

1. Disagree 294 22.4 147 24.1

2. Agree 1021 77.6 462 75.9

How accurate were your initial How accurate were your initial predications?predications?

Go back to the note cards you made Go back to the note cards you made earlier. earlier.

How accurate were your initial How accurate were your initial predications?predications?

ScholarsScholars 85 % felt they benefited from the 85 % felt they benefited from the

scholarshipscholarship 45% felt the scholarship was not 45% felt the scholarship was not

influential in their decision (they influential in their decision (they wanted to be teachers even without wanted to be teachers even without the scholarship)the scholarship)

Scholars report they had good Scholars report they had good opportunities and had positive opportunities and had positive impact on their institutionsimpact on their institutions

27% of the PIs report funding to 27% of the PIs report funding to continue the scholarships.continue the scholarships.

Comments about ScholarsComments about Scholars

““A student congress of continuing A student congress of continuing and graduated students was formed and graduated students was formed by a CETP scholar. This Congress by a CETP scholar. This Congress offered the opportunity for offered the opportunity for experienced teachers, novice experienced teachers, novice teachers, and undergraduates to teachers, and undergraduates to exchange ideas and to discuss issues exchange ideas and to discuss issues related to teaching and the related to teaching and the profession.” profession.” Dean/Department ChairDean/Department Chair

Comments from ScholarsComments from Scholars

““I am proud to be a member of this I am proud to be a member of this wonderful community of dedicated wonderful community of dedicated professionals. I love knowing that I have professionals. I love knowing that I have such a strong system of support behind such a strong system of support behind me.”me.”

““Very, very positive – I know I have a life Very, very positive – I know I have a life long connection to a professor who is willing long connection to a professor who is willing to help me any way he can when I begin my to help me any way he can when I begin my teaching career.”teaching career.”

““As a refugee with low income, I couldn’t at As a refugee with low income, I couldn’t at all afford the tuition for (recertification)…But all afford the tuition for (recertification)…But this scholarship gave me back hope.”this scholarship gave me back hope.”

Concerns/Questions about Core Concerns/Questions about Core Data CollectionData Collection

What concerns or questions do you What concerns or questions do you have about collecting Core data?have about collecting Core data?

Record these on a note card and we Record these on a note card and we will pick them up.will pick them up.

Documents

Preparation “A Glass Half Full”