Using Electronic Portfolios to Assess Learning at IUPUI

Trudy Banta

Sharon Hamilton

Susan Kahn

Indiana University-Purdue University Indianapolis

Plan

Implement

Evaluate

ImproveCulture of Evidence

PLANNING

1. Campus mission, goals

2. Unit goals aligned

3. Programs based on assessable goals with PIs

4. Annual reports on the Web

PAII Evaluation Services

1. Assessment of learning2. Surveys3. Program reviews4. Performance indicators5. Program cost analysis6. Web-based evaluation tools7. Program evaluation/action research8. Accreditation

IMPROVEMENT AGENDA

1. Reporting to internal constituents

2. Demonstrating accountabilitywww.iport.iupui.edu

3. Proposing improvement initiatives based on findings

4. Improving assessment methods

Assessment of Individual Student Development

1. Assessment of basic skills for use in advising• Placement• Counseling

2. Periodic review of performance with detailed feedback

3. End-of-program certification of competence• Licensing exams• External examiners

Key Results of Individual Assessment

1. Faculty can assign grades

2. Students learn their own strengths and weaknesses

3. Students become self-assessors

A Second Look

1. Across students

2. Across sections

3. Across courses

1. Where is learning satisfactory?

2. What needs to be retaught?

3. Which approaches produce the most learning for which students?

Group Assessment Activities

1. Classroom assignments, tests, projects2. Questionnaires for students, graduates,

employers3. Interviews, focus groups4. Program completion and placement5. Awards/recognition for graduates6. Monitoring of success in graduate

school7. Monitoring of success on the job

Use of Results of Group Assessment

1. Program improvement

2. Institutional and / or state peer review

3. Regional and / or national accreditation

Assessment of Learning

1. Faculty agreement on outcomes and performance standards

2. Evidence in syllabi and assignments that outcomes are taught

3. Collective faculty review of group performance

4. Use of findings to improve instruction and curriculum

Methods of Assessment

1. Paper and pencil tests2. Individual or group projects3. Portfolios4. Observation of practice5. Observation of simulated practice6. Analysis of case studies7. Attitude or belief inventories8. Interviews and focus groups9. Surveys

Now We Have

the

Press to assess with a test

2006Commission on the Future of

Higher Education

1. Need simple way to compare institutions

2. Institutions should be more accountable for student learning

What happens to outcomes

assessment

undertaken

to guide improvements

in

teaching and learning?

Using Electronic Assessment Methods

We can

1. Track progress in assignments

2. Evaluate contributions to group projects

3. Conduct immediate process checks to evaluate instruction

4. Assess the quality of written work

What is an e-portfolio?

A selection of purposefully organized artifacts that support retrospective and prospective reflection and that document, assess, and enhance student learning over time.

Why e-portfolios?

1. For students Track growth and development Develop capacities for metacognition and self-

directed learning Integrate and apply learning

2. For faculty Enrich learning experiences Base assessment on authentic performances

3. For programs and institutions Assessment for admissions, improvement and

effectiveness, accreditation

Principles of Undergraduate Learning

1. Core communication and quantitative skills

2. Critical thinking

3. Integration and application of knowledge

4. Intellectual depth, breadth, and adaptiveness

5. Understanding society and culture

6. Values and ethics

ePort Learning Matrix

ePort levels of competence

What all undergraduate students at IUPUI should know and be able to do within their:

1. Introductory: First 26 credit hours

2. Intermediate: First 56 credit hours

3. Advanced: Major, profession, or academic program

4. Experiential: Co-curricular, extra-curricular, work-based learning

Inside a Matrix Cell

Goal: To document introductory competence in written communication

Expectations:

1.You have identified a different audience and purpose for at least 2 documents

2.You have selected information appropriate to your audience and purpose.

3.You have used two different organizational styles appropriate to audience and purpose

4. You have used appropriate language, style, writing conventions, and formats.

Features of ePort

1. Resources

2. Learner Profile

3. MatricesPULCustomized

4. Presentation Tool

5. Reporting Tool

Dashboard/Learner Profile

• Tasks

• Profile

• Pre-Survey

• Preflection

• Upload resources

• Reflection

Profile Information

Connections to self-assessment tools

Connections to Career Center; Student Life and Diversity

Connections to other helpful sites

CommunicationSkills

CriticalThinking

Integration &Application of

Knowledge

UnderstandingSociety &Culture

Values &Ethics

Etc ….

Intro

Intro

Intro

Intro

Intro

Intermed

Intermed

Intermed

Intermed

Intermed

Adv.

Adv.

Adv.

Adv.

Adv.

Exper

Exper

Exper

Exper

Exper

Artifact Artifact Artifact Artifact

reflect reflect reflect reflect

Crs grade

assess

interact

Crs grade

assess

interact

Crs grade

assess

interact

Crs grade

assess

interact

•

Time/level/Linear

Objec

ts

Cri

teri

a

Agg

rega

te(R

epor

ts)

Adoption strategies

1. Small grants to departments and to freshmen learning communities

2. Collaboration with users to develop needed features and modify or customize existing ones

3. Extensive faculty development and technology support

Student Self-Assessment in the IUPUI ePort

From Gateway Through Graduation

From Gateway Through Graduation

1. First year: Preflection Reflection

2. Capstone: Retrospective Prospective

Preflection

1. What are your important goals for your life after college? Think about your goals in areas like career, citizenship, relationships, personal life, and spiritual/religious development.

2. Thinking of one or two of these goals, what do you need to do and learn between now and graduation to get there? What abilities, skills, knowledge, and other characteristics do you need to improve or develop?

3. Which of the PULs seem(s) most important for you to improve upon to reach your goals? Why?

Reflection

1. Reflect upon the goals you had during your first weeks at IUPUI.

Have your goals, or your ideas about how to reach them, changed?

What have you learned that is relevant to the goal(s) discussed in your preflection? What is the evidence for this learning?

Are there additional PULs that you now see as relevant to your goals?

Capstone

Matrix

LIFELONG

LEARNING

POSSIBLE CAREERS

CURRENT ROLE IN

SOCIETY

PRINCIPLE OF UNDERGRADUATE LEARNING

Artifacts Artifacts Artifacts

REASON FOR BECOMING AN ENGLISH MAJOR

Artifacts Artifacts Artifacts

Matrix Thinking

1. Movement from focus on original document to focus on original document in relation to a new conceptual framework

2. See work in new focus

3. Results in greater depth of understanding of original work

4. Students can assess their intellectual growth since the original writing.

5. Development in reflective thinking

Development in Reflection

Ability to self assess

Awareness of how one learns

Developing lifelong learning skills

Ability to Self-Assess

Observing One’s Own Performance

Using Feedback and Evidence

Finding and Analyzing Patterns

Making Judgments

Islam Y107.doc exemplifies my ability to be a critical thinker because I had to put forth significant effort to separate my emotion from the facts and research….I believe that objectivity and rationality are at the core of every serious student – this paper shows me that I can be a serious student. Every class that I have taken in political science, English, and philosophy has emphasized the importance of looking past the surface of things. Additionally, my education in the liberal arts has taught me that there is much more to things than what my emotions tell me there are….My emotions are central only to my own experiences, and my critical thinking skills allow me to leap outside of my own experiences.

Critical thinking is my most important PUL because, no matter what goal I am pursuing, critical thinking will be a necessary step before that goal will ever be reached….Thinking about working as an advocate of the law in a globalizing world is intimidating. The documents in this cell, however, have calmed my angst so that I feel as though I am capable of accomplishing my goals. Two of the papers in this cell are only one page in length. I’ve come to learn that brevity and conciseness coupled with poignancy and clarity serve as valuable evidence that one has thought well and critically. To be brief is difficult if there is a lot to say. There is also a very long document in this cell.…Longevity of thought is sometimes necessary before brevity of a statement can be made clear.

I no longer see what I have to offer as an English job hunter in mere terms of degree possessed and years of experience…I look at what I have to offer in a larger context. Beyond the essentials in my resume that I share with all other graduates, I now see capacities in critical thinking, communications, and multi-project analyses. All these capacities can be supported with the creative and scholarly material in my matrix.”

CommunicationSkills

CriticalThinking

Integration &Application of

Knowledge

UnderstandingSociety &Culture

Values &Ethics

Etc ….

Intro

Intro

Intro

Intro

Intro

Intermed

Intermed

Intermed

Intermed

Intermed

Adv.

Adv.

Adv.

Adv.

Adv.

Exper

Exper

Exper

Exper

Exper

Artifact Artifact Artifact Artifact

reflect reflect reflect reflect

Crs grade

assess

interact

Crs grade

assess

interact

Crs grade

assess

interact

Crs grade

assess

interact

•

Time/level/Linear

Objec

ts

Cri

teri

a

Agg

rega

te(R

epor

ts)