View
4
Download
0
Category
Preview:
Citation preview
William R. Veal, Ph.D.
College of Charleston
NSTA/NCATE SPA Coordinator
Objectives
See the relationship between NSTA 2003 Standards and the NCATE Assessment System
Understand the NSTA-specific items that make this report science-specific
Discuss exemplar assessments
Look at the 2012 NSTA Preservice Standards
Things to keep in mind for scoring guides
Do the criteria adequately cover the important defining elements of the desired performance?
Do the cell descriptors distinguish the different levels of performance in observable ways?
Are the cell descriptors likely to be interpreted in the same way by different evaluators? (inter-rater reliability)
Is there enough substance to the assessment to assure external reviewers that if the criteria are met, the candidate is prepared to teach to the standards?
Section I - Context
Describe state or institutional policies
Describe field and clinical experiences
Program of Study:
Most important for graduate program to show content alignment
Advising Sheet
Include 3 years of data on candidates
Disaggregate data in a table by licensure area (i.e., biology, chemistry, etc.) and program level (i.e., baccalaureate, post-bacc., masters, etc.)
Section II - NCATE Assessments
Maximum of 8 assessments
Minimum of 6 assessments, but NSTA requires 8
Numbers 1 through 5 are the same for all SPA’s
Assessments fall into 3 categories
Content
Pedagogy
K-12 Student Learning
NSTA/NCATE Alignment (2003)
Standards 1 2 3 4 5 6 7 8
Concepts & Principles
Unifying Concepts
Technology in Sc.
Research in science
Math in science
2. Nature of Science
3. Inquiry
4. Issues in Science
6. Curriculum
7. Science in the Community
8. Assessment
9. Safety & Welfare
1. S
c. c
on
ten
t
Section IV – Evidence for Meeting Stds.
Key assessments required for all candidates.
Data tables should reflect & align with the scoring guides
10 criteria in scoring guide = 10 data points in table
Narrative:
A. Description of the assessment and use in program
B. Alignment of assessment with stds. (scoring guide)
C. Brief data analysis
D. Interpretation and reflection on program
1 Content: Praxis II Test
Standards 1 2 3 4 5 6 7 8
Concepts & Principles
Unifying Concepts
Technology in Sc.
Research in science
Math in science
2. Nature of Science
3. Inquiry
4. Issues in Science
6. Curriculum
7. Science in the Community
8. Assessment
9. Safety & Welfare
1. S
c. c
on
ten
t
1 Content: Praxis II or state exam
Alignment of test with NSTA stds. if not Praxis II
Aggregated pass rates for all completers for the previous 3 years (includes most recent year) [2 years for resubmits]
Disaggregate data by year, program level, and licensure area
80% pass rate applies if more than 10 total (all science content tests for all 3 years)
Data include:
Mean scores
Range scores
Pass rates for state
Sub-scores if available
Year Number of
Completers Pass Rate (%)
2010-2011 6 100
2009-2010 5 100
2008-2009 7 87.5
Subscore 1 (Science and Technology): 278.5 (254-300) Subscore 2 (Life Science): 284.7 (269-300) Subscore 3 (Physical Science): 264.3 (238-279) Subscore 4 (Earth Systems &the Universe): 251.3 (222-279) Subscore 5 (Cell Biology, Heredity, &Evolution): 267.2 (240-283) Subscore 6 (Organismal Biology and Ecology): 274.8 (255-300)
Aggregated pass rates on ICTS Science: Biology test for past 3 years Mean (and range) of subscores on ICTS Science: Biology test for most recent year (2010-2011)
Assessment 1 Content: Praxis II Items to Consider
Are the data available?
How will you get the data?
Do you have 10 completers in years? If not, then do not have to meet 80%. Still have to show disaggregated data by program and level and subscores.
Reflections on data
Upon examination of the data, we were able to learn and improve our program. For example, the year that we dropped Evolutionary Biology from our required coursework, the subscores in that area dropped dramatically. We re-evaluated our program and brought the course back into the requirements.
2 Content: Science Content Alignment
NSTA 2003 Standards 1 2 3 4 5 6 7 8
Concepts &Principles
Unifying Concepts
Technology in Sc.
Research in science
Math in science
2. Nature of Science
3. Inquiry
4. Issues in Science
6. Curriculum
7. Science in the Community
8. Assessment
9. Safety & Welfare
1. S
c. c
on
ten
t
2: GPA and Content Exams
Grade distributions or GPAs must be disaggregated by licensure areas of the candidates (i.e., biology, general science, etc)
distinct levels of licensure (middle level, high school, secondary, etc
degree levels of preparation (undergraduate/postbaccalaureate/ masters)
Provide aggregated data from all science courses taken by candidates in the program.
Include science content alignment with the 2003 NSTA Standards using the Content Analysis Forms
GPA and Science Content
Year Program Number of
Completers
Mean
Major GPA
(4.0 scale)
Range of
Major GPAs
(4.0 scale)
2009-2010 Undergraduate 0 ---- ----
2008-2009 Undergraduate 3 3.11 2.60-4.0
2007-2008 Undergraduate 7 3.16 2.88-3.40
Table 1: Undergraduate Major GPA- Traditional Plan of Study
2: Content Analysis Forms Available on the NCATE website
Align the science content with the NSTA 2003 Standards
Must demonstrate a 90% alignment
Table 1: Unifying Concepts A. Core
Competencies
B. Required course number & name and
advising requirements
1. Multiple ways we
organize our
perceptions of the world
and how systems
organize the studies
and knowledge of
science.
CHE 161- Introduction to Teaching Science Seminar CHE140- General Chemistry I CHE 141- General Chemistry II BSC 197- Molecular and Cellular Basis for Life GEO 100 Earth Systems Science PHY 110- Physics for Engineering and Science Majors I PHY 111 Physics for Engineering and Science Majors II PHY 205- Origins of the Universe
2. Nature of scientific
evidence and the use of
models for explanation.
CHE 161- Introduction to Teaching Science Seminar CHE140- General Chemistry I CHE 141- General Chemistry II BSC 197- Molecular and Cellular Basis for Life GEO 100 Earth Systems Science PHY 110- Physics for Engineering and Science Majors I PHY 111 Physics for Engineering and Science Majors II PHY 205- Origins of the Universe
3 Pedagogy: Unit Plan
NSTA 2003 Standards 1 2 3 4 5 6 7 8
Concepts & Principles
Unifying Concepts
Technology in Sc.
Research in science
Math in science
2. Nature of Science
3. Inquiry
4. Issues in Science
6. Curriculum
7. Science in the Community
8. Assessment
9. Safety & Welfare
1. S
c. c
on
ten
t
3 Pedagogy: Unit Plan At least one major teaching unit
(not just a single lesson plan) includes requirements for activities with lesson plans and various assessments. The unit must include: Unifying concepts
Nature of science
Inquiry
Issues
Personal and technological applications
State and national (NSES) science standards
Assessment
Must be SCIENCE-specific (i.e., criteria must align with NSTA elements)
3 Pedagogy: Unit Plan
How does this assessment fit into what the rest of the Unit is doing?
Can you build on a Unit Plan assessment that is already a part of the program?
Are the scoring guides presently used at your institution specific to the 2003 NSTA standards?
Usually done with an Addendum.
Criteria / Sub-Standard
Inadequate –1 Adequate - 2 Target - 3
1a. Understand major science concepts.
One or more lessons contain scientific inaccuracies and/or are ineffective at conveying concepts or principles. The unit fails to teach how scientific theories are developed.
Lessons successfully and accurately convey to students concepts and principles of science.
Lessons very successfully and accurately convey to students concepts and principles of science. Creative planning is evident. The unit teaches how scientific theories are developed.
1b. Understand NSES unifying concepts.
The unit fails to teach the nature of scientific laws or to relate concepts to them.
The unit teaches the nature of scientific laws and relates concepts to them.
The unit teaches the nature of scientific laws and relates concepts to them in particularly effective ways.
1c. Understand technical applications of science.
The personal and technological applications of the scientific concepts and principles of the unit are inadequately addressed, or pedagogy is inadequate for student understanding.
The personal and technological applications of the scientific concepts and principles of the unit are effectively addressed and pedagogy is adequate to develop student understanding.
Candidates apply their understanding of the personal and technological applications of the scientific concepts and principles to their own life and surroundings.
2a. NOS Understanding of the historical and cultural development of science as related to the key ideas and major understandings of the unit are poorly addressed or pedagogy is inadequate to develop student understanding.
Understanding of the historical and cultural development of science as related to the key ideas and major understandings of the unit are explicitly addressed and pedagogy is adequate to develop candidate understanding.
Candidates use their understanding of historical and cultural development of science in multiple and creative ways in the lesson plans.
4 Pedagogy: Student Teaching Observation Form
NSTA 2003 Standards 1 2 3 4 5 6 7 8
Concepts & Principles
Unifying Concepts
Technology in Sc.
Research in science
Math in science
2. Nature of Science
3. Inquiry
4. Issues in Science
6. Curriculum
7. Science in the Community
8. Assessment
9. Safety & Welfare
1. S
c. c
on
ten
t
4 Pedagogy: Student Teaching Observation Form Addendum to regular form
Shows that candidates can implement safety knowledge
Include science specific areas
Safety elements 9a-d is a minimum to pass this Assessment.
4 Pedagogy: Example A
Standard Likert
9b. The candidate practices safe and proper techniques for the preparation, storage, dispensing, supervision, and disposal of all materials used in science instruction.
1 2 3 4 5
1 does not address standard 2 addresses but does not meet standard 3 meets standard at acceptable level 4 above minimum acceptable level 5 exceeds standard
Scoring guide needs improvement: The criteria do not differentiate different levels of performance in observable ways.
4 Pedagogy: Example B Unacceptable Acceptable Target
Safe techniques (NSTA 9b)
The intern fails to practice safe and proper techniques for the preparation, storage, dispensing, supervision, and disposal of all materials used in science instruction.
The intern practices safe and proper techniques for the preparation, storage, dispensing, supervision, and disposal of all materials used in science instruction.
The intern practices safe and proper techniques for the preparation, storage, dispensing, supervision, and disposal of all materials used in science instruction without additional prompting from mentor or university supervisor.
Candidate are rated using the following guidelines: Unacceptable: The candidate's performance is not improving and is not acceptable Acceptable: The candidate's performance meets the expectations for pre-service teachers Target: The candidate's performance meets expectations that would be characteristic of an experienced teacher
Scoring guide needs improvement: Criteria using operational terms are needed. More description in the criterion is needed.
5: Evidence of K-12 Student Learning
NSTA 2003 Standards 1 2 3 4 5 6 7 8
Concepts & Principles
Unifying Concepts
Technology in Sc.
Research in science
Math in science
2. Nature of Science
3. Inquiry
4. Issues in Science
6. Curriculum
7. Science in the Community
8. Assessment
9. Safety & Welfare
1. S
c. c
on
ten
t
5: Evidence of K-12 Student Learning
Candidate/Teacher Work Sample or Portfolio
Must be science specific and address each applicable NSTA standard and element.
Must have descriptors that are measurable.
2c. The candidate facilitates students learning the nature of science
Provides little to no evidence of student learning beyond memorization of at least one aspect of the nature of science.
Provides evidence of student learning beyond memorization of at least one aspect of the nature of science. Representative sample of student work included.
Provides evidence of student learning beyond memorization of two or more aspects of the nature of science and representative samples of student work. Includes a reflective analysis of students’ learning.
6 Content: Safety and Welfare
NSTA 2003 Standards 1 2 3 4 5 6 7 8
Concepts & Principles
Unifying Concepts
Technology in Sc.
Research in science
Math in science
2. Nature of Science
3. Inquiry
4. Issues in Science
6. Curriculum
7. Science in the Community
8. Assessment
9. Safety & Welfare
1. S
c. c
on
ten
t
This assessment shows and measures candidates knowledge of safety items 9a-d.
Usually involves a test of safety knowledge.
Test items must be specifically aligned with each NSTA element, or aggregated assignments and tests are aligned with each element.
Ex. 5. Hazardous chemicals enter the body the fastest by: (9a)
A. ingestion.
B unbroken skin absorption.
C inhalation.
D. broken skin absorption
Ex. Results of Flinn safety exams & chapters 12-18 (9b)
7 Content: Research in Science Discipline
NSTA 2003 Standards 1 2 3 4 5 6 7 8
Concepts & Principles
Unifying Concepts
Technology in Sc.
Research in science
Math in science
2. Nature of Science
3. Inquiry
4. Issues in Science
6. Curriculum
7. Science in the Community
8. Assessment
9. Safety & Welfare
1. S
c. c
on
ten
t
7 Content: Research in Science Discipline
Usually performed in the science content courses
A system for post degree or transfer candidates
Alternative course or
Evaluation of previous work
Action Research or science fair is not acceptable
What would your science departments deem acceptable?
Could be a short project
Example Scoring Guide Investigation parts Not acceptable (1) Acceptable (2) Excellent (3)
D. Data collection and organization (NSTA 1d)
Data is collected in a manner that is inconsistent with the science discipline. Data is poorly organized and/or displayed.
Data is collected consistent with the science discipline. Data is organized and presented using graphs, charts or tables with minor errors.
Data is collected consistent with the science discipline. Data is organized, clearly addresses research problem and presented with no errors.
F. Analysis and interpretation of data (NSTA 1d, 1e)
Inappropriate techniques used in data analysis and interpretation.
Appropriate techniques used in data analysis and interpretation with minor errors.
Appropriate techniques are used in data analysis and interpretation is clear and concise with no errors.
G. Final conclusions (NSTA 1d)
Conclusions not clearly related to problem or supported by data
Conclusions generally related to problem and supported by data
Conclusions clearly related to problem and fully supported by data
No part of this project should receive less than a “2” or it should be redone to the satisfaction of the instructor.
8 Content: Contextual Content
NSTA 2003 Standards 1 2 3 4 5 6 7 8
Concepts & Principles
Unifying Concepts
Technology in Sc.
Research in science
Math in science
2. Nature of Science
3. Inquiry
4. Issues in Science
6. Curriculum
7. Science in the Community
8. Assessment
9. Safety & Welfare
1. S
c. c
on
ten
t
8 Content: Contextual Content
Content understanding of topics important to science educators
Assessment #8 is usually addressed with a portfolio of appropriate assignments
If there are courses that help to meet this standard, there needs to be specific descriptions to tie the course directly back to the 2003 NSTA standards
Review: Things to keep in mind for scoring guides
Do the criteria adequately cover the important defining elements of the desired performance?
Do the cell descriptors distinguish the different levels of performance in observable ways?
Are the cell descriptors likely to be interpreted in the same way by different evaluators? (inter-rater reliability)
Is there enough substance to the assessment to assure external reviewers that if the criteria are met, the candidate is prepared to teach to the standards?
New 2012 NSTA Standards
These were approved by NSTA & SASB.
First submission in Fall 2012 as a pilot.
Programs can use either set of standards through Fall 2013.
Spring 2014 – All programs must use 2012 Standards
Discontinue Option C at this time.
NSTA Standard 1: Content Knowledge Effective teachers of science understand and articulate the knowledge and practices of contemporary science. They interrelate and interpret important concepts, ideas, and applications in their fields of licensure.
Preservice teachers will:
1a) Understand the major concepts, principles, theories, laws, and interrelationships of their fields of licensure and supporting fields as recommended by the National Science Teachers Association.
1b) Understand the central concepts of the supporting disciplines and the supporting role of science-specific technology.
1c) Show an understanding of state and national curriculum standards and their impact on the content knowledge necessary for teaching P-12 students.
NSTA Standard 2: Content Pedagogy Effective teachers of science understand how students learn and develop scientific knowledge. Preservice teachers use scientific inquiry to develop this knowledge for all students.
Preservice teachers will:
2a) Plan multiple lessons using a variety of inquiry approaches that demonstrate their knowledge and understanding of how all students learn science.
2b) Include active inquiry lessons where students collect and interpret data in order to develop and communicate concepts and understand scientific processes, relationships and natural patterns from empirical experiences. Applications of science-specific technology are included in the lessons when appropriate.
2c) Design instruction and assessment strategies that confront and address naïve concepts/preconceptions.
NSTA 2011 Standards and Elements Content Assmt
1 Stand. Tests
Content Assmt 2
Grades / Analysis
Forms
PCK Assmt 3
Unit Plan
PCK Assmt 4 Std.
Tchg. eval form
Std. Lrng. Assmt 5
Work Sample
PCK Assmt 6
Profess.
1a. Concepts, principles, theories, laws, and
interrelationships
1b. Supporting content
1c. State/Nat’l Standards
a. Variety of inquiry
2b. Active inquiry lessons
2c. Formative and summative assessment
2d. Instruction and assessment of naïve
/preconceptions
3a. Variety teaching strategies
3b. Inquiry, laboratories, field experiences,
NOS
3c. Assessment
3d. Safety
4a. Chemicals
4b. Procedures
4c. Living organisms
5a. Collect, organize, analyze, reflect on pre-
formative and summative evidence
5b. Distinguish science from other ways of
knowing
5c. Developmentally appropriate inquiries
6 a. Content Prof. Dev.
6b. Education Prof. Dev.
2003 NSTA Standard/Element
Assessment 2012 NSTA Standard/Element
1a Licensure Test 1a-c. Content Knowledge
1a Content Exams or Science Content GPA and Content Analysis Form
2a-c. Content Knowledge
1a-c, 2a-c, 3a-b, 4a-b, 6a-b, 7a-b, 8a-c
Unit Plan 3a-d. Learning Environments
9a-d Teaching Observation Instrument 4a-c. Safety
1a, 2b, 3b, 4b Teacher Work Sample or Portfolio 5a-c. Impact on Student Learning
9a-d Safety exams, papers, and/or projects
1d-e Science Research (subsumed in CAF)
2a-c, 3a-b, 4a-b Assignments, tests, and/or papers.
6a-b. Professional Knowledge and Skills
What do Programs have to do to adjust to 2012 Standards? Keep the same Assessments (1-5)
Alter the criteria and cell descriptors in the instruments to align with the new wording of the 2012 Standards.
Review the Rubrics online at NSTA.org/preservice for guidelines about how to interpret evidence.
New SASB guidelines state: Programs must meet Standards with a “preponderance of
evidence”. (subjective interpretation of reviewers)
Not all elements of a standard need to be met in order to pass the standard.
Recommended