72
NCATE NCATE Institutional Institutional Orientation Orientation Loren Blanchard, Vice President David C. Loren Blanchard, Vice President David C. Smith, Dean Emeritus Smith, Dean Emeritus Accreditation & Accountability College Accreditation & Accountability College of Education of Education Louisiana System of Universities Louisiana System of Universities University of Florida University of Florida Issues in Assessment Systems

NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

  • View
    219

  • Download
    0

Embed Size (px)

Citation preview

Page 1: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

NCATE Institutional NCATE Institutional OrientationOrientation

Loren Blanchard, Vice President David C. Smith, Dean Loren Blanchard, Vice President David C. Smith, Dean EmeritusEmeritus

Accreditation & Accountability College of EducationAccreditation & Accountability College of Education

Louisiana System of Universities University of FloridaLouisiana System of Universities University of Florida

Issues in Assessment Systems

Page 2: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 2

Accountability in Higher Accountability in Higher EducationEducation

Establishing criteria for the Establishing criteria for the systematic collection for the systematic collection for the collection and analysis of data.collection and analysis of data.

SCDEs often respond to policy SCDEs often respond to policy makers and accrediting bodies.makers and accrediting bodies.

The issues should be: program The issues should be: program efficacy and program efficacy and program improvement.improvement.

Page 3: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 3

Accountability and Accountability and Assessment are With UsAssessment are With Us

It is obvious that we are in an age It is obvious that we are in an age of accountability and that of accountability and that inevitably involves assessment. inevitably involves assessment. Like the Ice Age, there is little Like the Ice Age, there is little reason to believe that it will pass reason to believe that it will pass quickly.quickly.

Page 4: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 4

Questions and More Questions and More QuestionsQuestions

What kinds of questions do others What kinds of questions do others ask? (State DOE, Legislature, ask? (State DOE, Legislature, University, Feds.)University, Feds.)

Why do they ask those kinds of Why do they ask those kinds of questions?questions?

What do those kinds of questions What do those kinds of questions signal about their concerns?signal about their concerns?

Page 5: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 5

What questions do What questions do you wish others you wish others would ask?would ask?

Page 6: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 6

How do you respond to important How do you respond to important questions regarding program questions regarding program effectiveness and efficiency when effectiveness and efficiency when you are asked, “what evidence do you are asked, “what evidence do you have”, and “how do you you have”, and “how do you know”?know”?

What data do you have to support What data do you have to support your convictions and contentions?your convictions and contentions?

Page 7: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 7

Would you argue that SCDEs Would you argue that SCDEs should not be accountable for the should not be accountable for the competence of those they competence of those they recommend for licensure?recommend for licensure?

Page 8: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 8

A license is to teachers and other A license is to teachers and other professional educational personnel professional educational personnel what accreditation is to SCDEs.what accreditation is to SCDEs.

Page 9: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 9

How do you think that we How do you think that we should be held should be held accountable?accountable?

Page 10: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 10

The nature of change is such that The nature of change is such that the press for change will always be the press for change will always be greater from outside the greater from outside the organization than within the organization than within the organization.organization.

Page 11: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 11

Have you thought deeply about Have you thought deeply about the purpose, mission and vision of the purpose, mission and vision of the organization and related them the organization and related them to the assessment system? There to the assessment system? There are implications for what you are implications for what you choose to include in your choose to include in your assessment system.assessment system.

Page 12: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 12

Goods and Services and Goods and Services and AssessmentAssessment

The assessment of products The assessment of products and services is different.and services is different.

Products are tangible goods.Products are tangible goods. Services are intangible goods.Services are intangible goods. The distinction is important.The distinction is important.

Page 13: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 13

The assessment issues are quite different. The assessment issues are quite different. Education is not a “product”; it does not Education is not a “product”; it does not have weight and does not take up space.have weight and does not take up space.

Education involves providing services that Education involves providing services that are intangible but have value.are intangible but have value.

The measurement of services is The measurement of services is exponentially more difficult than exponentially more difficult than measuring products.measuring products.

This distinction should be made clear to This distinction should be made clear to policy makers.policy makers.

Page 14: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 14

We deride single-measure, We deride single-measure, simplistic measures of educational simplistic measures of educational performance. But what do we performance. But what do we offer as a better alternative?offer as a better alternative?

““To every complex problem there To every complex problem there is an answer that is short, simple is an answer that is short, simple and wrong.” H. L. Menkinand wrong.” H. L. Menkin

Page 15: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 15

Effectiveness and Effectiveness and EfficiencyEfficiency

Effectiveness has to do with how Effectiveness has to do with how well something is functioning.well something is functioning.

Efficiency has to do with the effort, Efficiency has to do with the effort, time, or cost involved in the time, or cost involved in the preparation or production.preparation or production.

Page 16: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 16

Inevitable Tension in Inevitable Tension in AssessmentAssessment

The need to accommodate your The need to accommodate your situation.situation.

The need to compare with others The need to compare with others in similar situations.in similar situations.

It is necessary to compare within It is necessary to compare within and across institutions. (Internal and across institutions. (Internal relative productivity and relative productivity and comparison with counterpart comparison with counterpart units.) units.)

Page 17: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 17

A fundamental principle in A fundamental principle in communication theory is that when communication theory is that when the system becomes overloaded the system becomes overloaded with information people will ignore with information people will ignore all of the information rather that to all of the information rather that to ignore the information selectively. ignore the information selectively. ((Consider that principle in the context of fact Consider that principle in the context of fact

books and assessment systems.)books and assessment systems.)

Page 18: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 18

People in an organization focus on People in an organization focus on what is measured not what is said what is measured not what is said to be importantto be important..

Page 19: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 19

Consider open versus closed Consider open versus closed systems. Isn’t it better to have systems. Isn’t it better to have people working from the same people working from the same numbers?numbers?

Page 20: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 20

It is necessary to compare within It is necessary to compare within and across institutions. (Internal and across institutions. (Internal relative productivity and relative productivity and comparison with counterpart comparison with counterpart units.)units.)

Page 21: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 21

Do not make high-stakes decisions Do not make high-stakes decisions based on soft data. based on soft data. 

Consider “directionality” in Consider “directionality” in analyzing data.analyzing data.

Page 22: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 22

A single assessment system to A single assessment system to serve multiple needs is preferable serve multiple needs is preferable to a set of assessment systems, to a set of assessment systems, each of which is designed to serve each of which is designed to serve a single purpose.a single purpose.

Page 23: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 23

You get big numbers by adding You get big numbers by adding little numbers. It is important to little numbers. It is important to aggregate data at meaningful aggregate data at meaningful levels. (Program, department, levels. (Program, department, college, institution.)college, institution.)

Page 24: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 24

What we measure is a profound What we measure is a profound statement about what we believe statement about what we believe is important.is important.

Page 25: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 25

It is one thing to collect data, It is one thing to collect data, another thing to be discriminating another thing to be discriminating in collecting data, and still another in collecting data, and still another thing to know how to use data.thing to know how to use data.

Page 26: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 26

What matters and what matters What matters and what matters most? (The need to know and the most? (The need to know and the nice to know.)nice to know.)

Page 27: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 27

Multiple measures are important. Multiple measures are important. It is valuable to look at a problem It is valuable to look at a problem from more than one angle. What from more than one angle. What are key points at which to access? are key points at which to access? (At a minimum, entrance, exit and (At a minimum, entrance, exit and follow-up.)follow-up.)

Page 28: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 28

We are not good at being proactive in We are not good at being proactive in generating data and we are not good at generating data and we are not good at being creative in generating data.being creative in generating data.

Be proactive – give people information Be proactive – give people information that they do not ask for but informs that they do not ask for but informs them more deeply about the them more deeply about the effectiveness of your organization. effectiveness of your organization. Think carefully about what that Think carefully about what that information might be.information might be.

Page 29: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 29

In higher education it is always In higher education it is always teaching, research, and service. teaching, research, and service.

Typically policy makers place a Typically policy makers place a higher value on professional public higher value on professional public service than does the academy.service than does the academy.

Page 30: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 30

Consider the role of the executive in Consider the role of the executive in assessment. Assessment phases: planning, assessment. Assessment phases: planning, implementing, maintaining, and revising the implementing, maintaining, and revising the system. Planning involves knowing what data system. Planning involves knowing what data are needed and meaningful. It also involves are needed and meaningful. It also involves assembling a design team. Implementation assembling a design team. Implementation involves providing the personnel, hardware involves providing the personnel, hardware and software. Maintenance involves using the and software. Maintenance involves using the data in a meaningful fashion. There is a data in a meaningful fashion. There is a difference between data-informed and data-difference between data-informed and data-driven decisions. Revision involves driven decisions. Revision involves recognizing and accommodating changing recognizing and accommodating changing needs for data.needs for data.

Page 31: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 31

Problems that are regularly Problems that are regularly seen:seen:

People have difficulty in creating an People have difficulty in creating an assessment system.assessment system.

People think more about collecting data than People think more about collecting data than they do about the structure of their they do about the structure of their assessment system and the kinds of data that assessment system and the kinds of data that they include in their assessment system.they include in their assessment system.

They often want to do it for the wrong reason – They often want to do it for the wrong reason – for accreditation rather than seeing it as a tool for accreditation rather than seeing it as a tool to evaluate and improve what they are doing.to evaluate and improve what they are doing.

Page 32: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 32

They have difficulty in using candidate data They have difficulty in using candidate data three ways. (The aggregation issue.)three ways. (The aggregation issue.)

People are not aware of meaningful data that People are not aware of meaningful data that already exists and can be imported into their already exists and can be imported into their assessment system. Then they can focus their assessment system. Then they can focus their effort on data that they need to generate.effort on data that they need to generate.

People often do not know how to use data well.People often do not know how to use data well.

Page 33: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 33

People often do not consider examining People often do not consider examining relationships among data sets. (FTE and relationships among data sets. (FTE and headcount enrollment, enrollment and cost to headcount enrollment, enrollment and cost to generate a SCH).generate a SCH).

Time is a problem. It is not realistic to expect that Time is a problem. It is not realistic to expect that busy people can create and maintain an busy people can create and maintain an assessment system “on top of everything else”. It assessment system “on top of everything else”. It is very difficult to develop, implement, maintain is very difficult to develop, implement, maintain and revise an assessment system without and revise an assessment system without additional resources.additional resources.

Resources – human and technological, are needed. Resources – human and technological, are needed. The allocation of resources is a measure of The allocation of resources is a measure of institutional priority.institutional priority.

Page 34: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 34

Creating an assessment system is Creating an assessment system is a creative task; it is also tedious a creative task; it is also tedious and time-consuming.and time-consuming.

Page 35: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 35

Think about assessment and the Think about assessment and the development of an assessment development of an assessment system as an opportunity rather system as an opportunity rather than a problem or burden.than a problem or burden.

Page 36: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 36

Assessment Assumptions: Assessment Assumptions: Bases for ChangeBases for Change

Assessment should be based on Assessment should be based on explicit outcomes.explicit outcomes.

Assessment must involve Assessment must involve performance.performance.

Assessment should provide Assessment should provide feedback to the learner.feedback to the learner.

Assessment should have a multiple Assessment should have a multiple context.context.

Page 37: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 37

Assessment Assessment Assumptions: Assumptions: Bases for ChangeBases for Change

Assessment should involve Assessment should involve external publics.external publics.

Assessment should be Assessment should be developmental.developmental.

Assessment should be cumulative.Assessment should be cumulative.

Page 38: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 38

Developing an Assessment System Developing an Assessment System that Culminates in a Change that Culminates in a Change OrientationOrientation

Identify what it is that you want to learn Identify what it is that you want to learn or know more about.or know more about.

Determine how you will go about Determine how you will go about learning more information.learning more information.

Establish a plan for what will happen to Establish a plan for what will happen to data once they are collected.data once they are collected.

Determine the tools, strategies and Determine the tools, strategies and resources that you have available or will resources that you have available or will need to foster change.need to foster change.

Page 39: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 39

Developing an Assessment System Developing an Assessment System that Culminates in a Change that Culminates in a Change OrientationOrientation

Be prepared to make changes and Be prepared to make changes and continue to study effects of these continue to study effects of these changes to ensure that there are changes to ensure that there are no adverse effects.no adverse effects.

Page 40: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 40

Two Systems of Two Systems of Performance AssessmentPerformance Assessment

Professional Accountability Support Professional Accountability Support System Using a Portal Approach System Using a Portal Approach (PASS-PORT) – Statewide Initiative (PASS-PORT) – Statewide Initiative

Teacher Education Assessment Teacher Education Assessment System (TEAS) – National InitiativeSystem (TEAS) – National Initiative

Page 41: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

PASS-PASS-PORTPORT

Development Funded by the Board of Regents CITAL

Funds

An Electronic An Electronic PProfessional rofessional

AAccountability ccountability SSupport upport SSystem Using a ystem Using a PORTPORTal al

Approach Approach

for Teacher for Teacher

EducationEducation

Page 42: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 42

Evolution of PASS-PORTEvolution of PASS-PORT

Professional collaborative project Professional collaborative project between a Louisiana public and a between a Louisiana public and a Louisiana private higher education Louisiana private higher education institutioninstitution

State/national demands for State/national demands for outcome- based assessments for outcome- based assessments for teacher educationteacher education

Page 43: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 43

Evolution of PASS-PORTEvolution of PASS-PORT

Professional Accountability Support Professional Accountability Support System Using a Portal Approach (PASS-System Using a Portal Approach (PASS-PORT)PORT) Highly comprehensive web-based toolHighly comprehensive web-based tool Designed primarily for Louisiana teacher Designed primarily for Louisiana teacher

preparation programspreparation programs Allows institutions to gather and statistically Allows institutions to gather and statistically

analyze performance data on candidates, analyze performance data on candidates, faculty and program operations at select faculty and program operations at select time frames to glean significant trend datatime frames to glean significant trend data

Page 44: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 44

Evolution of PASS-Evolution of PASS-PORTPORT

PASS-PORT is a system designed to PASS-PORT is a system designed to assist institutions in fulfilling NCATE assist institutions in fulfilling NCATE Standards, Title II Accountability Standards, Title II Accountability reporting standards, and select teacher reporting standards, and select teacher education redesign criteria.education redesign criteria.

PASS-PORT is designed to assist PASS-PORT is designed to assist institutions to better appreciate institutions to better appreciate systematic collection, analysis and use systematic collection, analysis and use of program data.of program data.

Page 45: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 45

Evolution of PASS-Evolution of PASS-PORTPORT

PASS-PORT has wealth of useful PASS-PORT has wealth of useful functionalitiesfunctionalities Defines KSD requirements and Defines KSD requirements and

accompanying assessments for accompanying assessments for candidates at four different candidates at four different checkpoints across initial and checkpoints across initial and advanced programsadvanced programs

Page 46: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 46

Page 47: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

47

PASS-PORT Functionality1. Creation of Standards-based

Electronic Portfolios (“working” portfolios) Upload artifacts to PASS-PORT Create standards-based portfolios (from

scratch or from template) that are aligned with any set of professional standards (NCATE, ISTE, INTASC, NCTM, etc.)

Request electronic feedback and evaluation Used by faculty as well as teacher candidatesBenefit: Standards-based Electronic Portfolios, Unit

ReportingExamples: Uploading Artifacts; Working Portfolio From Scratch; From Template

Page 48: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

48

PASS-PORT Functionality

2. Creation of “Portal folios” - special portfolios submitted at various checkpoints (unit specified requirements) in the program: Portal I: Declaration of Major Portal II: Entry into Teacher Education

Program Portal III: Entry into Student Teaching Portal IV: Exit from Teacher Education

Program Benefit: Meet NCATE expectations for performance-based data collection, analysis, unit improvement and student learning.

Examples: Creating a Portal Folio; Routing of Portal Folios; Evaluating a Portal Folio

Page 49: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 49

Evolution of PASS-Evolution of PASS-PORTPORT

PASS-PORT has wealth of useful PASS-PORT has wealth of useful functionalitiesfunctionalities Emphasizes field/clinical experiences Emphasizes field/clinical experiences

and corresponding sitesand corresponding sites

Page 50: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

50

PASS-PORT Functionality3. Management of Field Experiences

Database of field experience sites Students manage own field experiences Add reflections to field experiences Collects data on field experience site and

cooperating teacher for reporting Field experiences can become artifacts to

include in portal foliosExample: Creating Field Experiences

Benefit: Meet NCATE expectations for evaluation of field experiences, improve unit and student learning.

Page 51: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 51

Evolution of PASS-PORTEvolution of PASS-PORT

PASS-PORT has wealth of useful PASS-PORT has wealth of useful functionalitiesfunctionalities Draws out and statistically analyzes Draws out and statistically analyzes

specific data for varied reporting specific data for varied reporting purposespurposes

Page 52: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

52

PASS-PORT Functionality4. Analysis/Reporting and

Communication Maintain record of professional development

by candidates and faculty for reporting purposes

Faculty create “taskfolios” (functional portfolios)

Generate reports to meet NCATE, Board of Regents, and/or university requirements, etc.

Create query to analyze any data field (i.e., field experiences)

Examples: Student Gender Analysis, Survey Example, Portal Folio Analysis

Benefit: Meet NCATE expectations for collecting and analyzing performance data for the entire college.

Page 53: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 53

PASS-PORT FunctionsPASS-PORT Functions

Encourages faculty to develop portfolios Encourages faculty to develop portfolios that align with NCATE’s Faculty that align with NCATE’s Faculty Performance elementsPerformance elements

Is a recording mechanism to chronicle Is a recording mechanism to chronicle program and unit improvements made program and unit improvements made as a consequence of data analysesas a consequence of data analyses

Provides individualized assessment Provides individualized assessment results at each checkpoint for results at each checkpoint for candidates and faculty that include candidates and faculty that include strengths, weaknesses and prescriptionsstrengths, weaknesses and prescriptions

Page 54: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

54

PASS-PORT Functionality5. NCATE Electronic Documents Room

Ability to create a secure web-based viewing area for displaying documents / artifacts for the NCATE review team

Post example portalfolios for review Share individual candidate’s artifacts at

different points to show growth and feedback/prescriptions

NCATE committee tools, communication

Benefit: Facilitate NCATE visit through easy creation of electronic documents room.

Page 55: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 55

EDR FunctionalityEDR Functionality Creation of Document RoomsCreation of Document Rooms

Create an unlimited number of secure Create an unlimited number of secure web-based viewing rooms web-based viewing rooms

Rooms are built by creating pages Rooms are built by creating pages (from templates) that can be nested (from templates) that can be nested and linked together with a simple and linked together with a simple point and click interface [no web point and click interface [no web master required]master required]

Provides access for NCATE reviewers Provides access for NCATE reviewers prior to site visitprior to site visit

Page 56: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 56

PASS-PORT ConnectionConnects to Louisiana’s K-12 Initiative and Enhances the PK-16+ Education Initiative

The Department of Education is developing a similar electronic portfolio system based on PASS-PORT functionality and the seamless exchange of data. As a result, a pre-service teacher’s performance data, initially developed at the undergraduate level, will be accessible upon employment and entrance into the state’s K-12 professional portfolio system. When the teacher enters graduate school, he/she will re-enter PASS-PORT at the university.

Page 57: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Questions? Contact…

Doug Williams, [email protected]

Loren Blanchard, [email protected]

Juanita Guerin, [email protected]

John Fulwiler, [email protected]

PASS-PASS-PORTPORT

Page 58: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 58

What kinds of data might be What kinds of data might be included in a Teacher Education included in a Teacher Education Performance Assessment Performance Assessment System©? System©? (Establishing a conceptual (Establishing a conceptual framework.)framework.)

Resource DataResource Data Productivity DataProductivity Data Candidate Performance Candidate Performance

DataData Faculty Performance DataFaculty Performance Data

Page 59: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 59

Structure of a Teacher Structure of a Teacher Education Assessment Education Assessment System©System©

First Symbol -First Symbol - Classification Classification√ A = Resource DataA = Resource Data√ B = Productivity DataB = Productivity Data√ C = Candidate Performance C = Candidate Performance

DataData√ D= Faculty Performance DataD= Faculty Performance Data

Second Symbol - Second Symbol - LevelLevel√ 1= Unit Data1= Unit Data√ 2= Institutional Data2= Institutional Data√ 3= Calculated Data3= Calculated Data

Third Symbol - Third Symbol - CategoryCategory√ Sub-systemsSub-systems

Fourth Symbol - Fourth Symbol - NumberNumber√ Element NumberElement Number

Page 60: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 60

TEAS© System TEAS© System andandSub-systemsSub-systems

A. ResourcesA. Resources

1. Unit1. Unit

1. Human Resources1. Human Resources

2. Fiscal Resources2. Fiscal Resources

3. Demographic 3. Demographic DataData

2. Institution2. Institution

1. Human Resources1. Human Resources

2. Fiscal Resources2. Fiscal Resources

3. Demographic 3. Demographic DataData

3. Derived3. Derived

B. ProductivityB. Productivity

1. Unit1. Unit

1. Enrollment1. Enrollment

2. Loads and Assignments2. Loads and Assignments

3. Program3. Program

4. Title II Data4. Title II Data

5. External $ Generated5. External $ Generated

6. Student Demographic Data 6. Student Demographic Data

2. Institution2. Institution

1. .Enrollment1. .Enrollment

2. Loads and Assignments2. Loads and Assignments

3. External $ Generated3. External $ Generated

4. Student Demographic Data4. Student Demographic Data

3. Derived3. Derived

1. Enrollment1. Enrollment

2. Loads and Assignments2. Loads and Assignments

3. External $ Generated3. External $ Generated

4. Student Demographic Data4. Student Demographic Data

Page 61: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 61

TEAS© System and Sub-TEAS© System and Sub-systemssystems

C. Candidate PerformanceC. Candidate Performance1. Unit1. Unit

1. Initial Program Admission1. Initial Program Admission2. Early in Program2. Early in Program3. Mid-program3. Mid-program4. Summative4. Summative5. Graduate Follow-up5. Graduate Follow-up6. Advanced Programs 6. Advanced Programs

AdmissionAdmission7. Mid-program7. Mid-program8. Summative8. Summative9. Graduate Follow-up9. Graduate Follow-up

2. Institution2. Institution1. Undergraduate1. Undergraduate2. Graduate2. Graduate

3. Derived3. Derived1. Undergraduate1. Undergraduate2. Advanced Programs2. Advanced Programs

D.FacultyD.Faculty

1. Unit1. Unit

1. Teaching Evaluations1. Teaching Evaluations

2. Scholarly Productivity2. Scholarly Productivity

3. Professional Service3. Professional Service

4. Institutional Service4. Institutional Service

2. Institution2. Institution

1. Teaching Evaluations1. Teaching Evaluations

2. Scholarly Productivity2. Scholarly Productivity

3. Professional Service3. Professional Service

4. Institutional Service4. Institutional Service

3. Derived3. Derived

1. Scholarly Productivity1. Scholarly Productivity

2. Institutional Service2. Institutional Service

Page 62: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 62

Kinds of Information in TEAS.Kinds of Information in TEAS.

AACTE – NCATE Annual Report data.AACTE – NCATE Annual Report data. Resource Information – Salaries, Budget, Faculty Positions…Resource Information – Salaries, Budget, Faculty Positions… Comparative Salary Information – Unit, Comparative Salary Information – Unit,

Institution…Institution… Productivity Data – SCH, Headcount Enrollment, Productivity Data – SCH, Headcount Enrollment,

FTE Enrollment, Service Activity, Graduate Impact FTE Enrollment, Service Activity, Graduate Impact DataData

Data for Staff PlanningData for Staff Planning Data for Budget AnalysisData for Budget Analysis Advocacy Data - Equity in the System, Monitoring Advocacy Data - Equity in the System, Monitoring

the Systemthe System

Page 63: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 63

Using Productivity Data - Using Productivity Data - Examples.Examples.

B.1.2.7 Hours Taught/FTE FacultyB.1.2.7 Hours Taught/FTE Faculty #DIV/0!#DIV/0!B.2.2.4 Hours Taught/FTE FacultyB.2.2.4 Hours Taught/FTE Faculty #DIV/0!#DIV/0!B.1.2.8 Sch/FTE FacultyB.1.2.8 Sch/FTE Faculty #DIV/0!#DIV/0!B.2.1.10 Sch/FTE FacultyB.2.1.10 Sch/FTE Faculty #DIV/0!#DIV/0!B.1.3.1 Cost/SCHB.1.3.1 Cost/SCH #DIV/0!#DIV/0!B.2.2.5 Cost/SCHB.2.2.5 Cost/SCH #DIV/0!#DIV/0! Compare productivity over time!Compare productivity over time!

B.1.3.2 $ Prepare Elem. Teacher Approx.B.1.3.2 $ Prepare Elem. Teacher Approx.#DIV/0!#DIV/0!

B.1.3.3 $ Prepare Sec. Teacher Approx.B.1.3.3 $ Prepare Sec. Teacher Approx.#DIV/0!#DIV/0!

Page 64: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 64

Candidate Impact Data.Candidate Impact Data.

How important are schools, colleges and How important are schools, colleges and departments of education?departments of education?

C.1.4.16 Potential Candidate ImpactC.1.4.16 Potential Candidate Impact

=(D345*25*25)+(D346*125*25)=(D345*25*25)+(D346*125*25)

Movers and shakers. One years’ graduating class.Movers and shakers. One years’ graduating class.

Page 65: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 65

Faculty Performance Data.Faculty Performance Data.

Professional ServiceProfessional Service

D.1.3.1 No.State Wrkshps, ElemD.1.3.1 No.State Wrkshps, Elem

D.1.3.2 No. State Wrkshps, SecD.1.3.2 No. State Wrkshps, Sec

D.1.3.3 No. State Wrkshps, TotalD.1.3.3 No. State Wrkshps, Total 00

D.1.3.4 Elem. Tchrs. @ State Conf.D.1.3.4 Elem. Tchrs. @ State Conf.

D.1.3.5 Sec. Tchrs. @ State Conf.D.1.3.5 Sec. Tchrs. @ State Conf.

D.1.3.6 Total Tchrs. @ State conf. (PS)D.1.3.6 Total Tchrs. @ State conf. (PS) 00

D.1.3.7 Potential Student ImpactD.1.3.7 Potential Student Impact 00

D.1.3.8 No. State Wrkshps/FacultyD.1.3.8 No. State Wrkshps/Faculty#DIV/0!#DIV/0!

Page 66: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 66

Attributes of this Attributes of this system.system.

366 pieces of information are contained in 366 pieces of information are contained in the system. the system.

216 pieces of information (59%) are 216 pieces of information (59%) are entered.entered. Many of the data are already routinely collected.Many of the data are already routinely collected. It is very likely that the institution and unit It is very likely that the institution and unit

currently collects some of the data.currently collects some of the data. 150 pieces of information (41%) are 150 pieces of information (41%) are

generated.generated.

Page 67: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 67

Consider the kinds of information generated.Consider the kinds of information generated. Comparative data within the system.Comparative data within the system. Comparative data over time – multiple years.Comparative data over time – multiple years. Evaluative data.Evaluative data. Data for planning.Data for planning. Proactive data.Proactive data.

Graphs and tables can be developed from Graphs and tables can be developed from the data in this system. the data in this system. (Enrollment and (Enrollment and Budget, % of Faculty and Budget, % of Grad. SCH Budget, % of Faculty and Budget, % of Grad. SCH and Budget)and Budget)

Page 68: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 68

What You Have and What You What You Have and What You NeedNeed

0

20

40

60

80

100

120

140

160

180

Prob. Have Prob. Need Derived

Series1

Page 69: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 69

This System and This System and YourYour System System

The data in this performance assessment The data in this performance assessment system can be used for:system can be used for: Unit planning, advocacy,annual reports and Unit planning, advocacy,annual reports and

budgeting.budgeting. Institutional planning and evaluation.Institutional planning and evaluation. External advocacy and information.External advocacy and information. Accreditation, program approval and review.Accreditation, program approval and review.

This system is easily modified.This system is easily modified. Remember, you do not need to create a Remember, you do not need to create a

performance assessment system annually.performance assessment system annually. The entire assessment system does not need The entire assessment system does not need

to be put into operation simultaneously.to be put into operation simultaneously.

Page 70: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 70

Your Assessment Your Assessment System.System.

Think about the framework for your system.Think about the framework for your system. All data elements do not need the same All data elements do not need the same

baseline.baseline. You can implement your assessment system You can implement your assessment system

systematically and gradually.systematically and gradually. An assessment system can help make informed An assessment system can help make informed

decisions about improving your programs.decisions about improving your programs. There is a relationship between resources and There is a relationship between resources and

productivity. Make unconventional comparisons. productivity. Make unconventional comparisons. (Compare SCH/FTE with candidate evaluations of (Compare SCH/FTE with candidate evaluations of programs over time.)programs over time.)

Page 71: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 71

YourYour System System

This system is available to you.This system is available to you. Email: [email protected]: [email protected] But you need your own system.But you need your own system. This system is easily modifiable.This system is easily modifiable. There are advantages in having some There are advantages in having some

commonality in data sets - within the state, commonality in data sets - within the state, region and nationally.region and nationally.

Page 72: NCATE Institutional Orientation Loren Blanchard, Vice President David C. Smith, Dean Emeritus Accreditation & Accountability College of Education Louisiana

Loren Blanchard and David C. Smith

Issues in Assessment Systems 72