1

Assessing Foundations Programs

K. Lynn Taylor

University Teaching ServicesThe University of Manitoba,

Winnipeg, Manitoba, Canada R3T 2N2

Foundations of University Teaching Colloquium

Flinders University, Adelaide, April 10-11, 2003

Acknowledgements: This study is supported by a SSHRC Standard Research Grant (#410-2002-1584) to Schonwetter (PI), Ellis, Griffin, Perry, Taylor & Wright, 2002-2005

2

Session Objectives

Outline our longitudinal study

Explore selected bench mark data

Seek feedback on the design and findings

3

University of Manitoba: Program Requirements

Theory: Credit Course - 129.745 Teaching and Learning in Post-

Secondary Instruction (3 credits) or Extended Workshops & Papers/Research Project

Practice: TA work reflection and feedback or Mentor supervised teaching

Professional Development: 20 Hours Teaching Dossier and Curriculum Vitae (CV) Optional: WebCT training; Accent Modification.

4

University of Waterloo:Program Requirements

GS 901: 6 workshops 6 reflective response papers

GS 902: 2 more workshops 2 more response papers, research paper,

presentation on paper, teaching dossier GS 903:

3 observed teaching events 3 reflective response papers

5

Longitudinal Study: Gaps in the Literature

Gap 1: Lack of research and comprehensive program evaluations based on empirical design (Chism, 1998; Weimer &

Lenze, 1997) Gap 2: Target instructional interventions to specific

groups across institutions (Levinson-Rose & J. & Menges, 1981; Weimer &

Lenze, 1997) Gap 3: Inclusion of different kinds of inquiry,

including both qualitative methods and quantitative methods (Levinson-Rose & J. & Menges, 1981; Weimer & Lenze, 1997)

Gap 4: Limited models and theories to direct research (Blackburn, Beiber, Lawrence & Trautvetter, 1991; Menges & Svinicki, 1989;

Perry et al., 1997; Weimer & Lenze, 1997)

6

ConceptualFramework

GTA Studies (Abbott, Wulff & Szego, 1989;

Carroll, 1980; Chism, 1998; Weimer

& Lenze, 1997)

GTA Development

Theories (Abbott, Wulff & Szego, 1989; Marincovitch, Prostko & Stout,

1998; McKeachie, 1997)

New Hires Research

(Boice, 1992; Menges, 1994; 1996; Perry et al., 1997; Smith & Kalivoda,

1998)

Control Theory(Bandura, 1986; Perry et al., 1997; 1999)

Pilot Studies of CUT-CHET Programs

(Ellis & Schonwetter, 2001; Schonwetter & Taylor, 2000)

CHET-CUT Study

(SSHRC Grant to Schonwetter (PI), Ellis, Griffin, Perry, Taylor &

Wright, 2002-2005)

7

PROCEDURE

Ethical Approval

Access to Graduate Students

Pre-Test prior to entry into CHET-CUT

CHET-CUT Longitudinal Study

University of Manitoba

University of Waterloo

ControlGroup(s)

Post-Test after CHET-CUT completion

18 month follow-up into a tenure track position

8

Pre-Test Perceptions of:

Preparation for academic career Importance to academic career

Perceived control questions: having to teach a new course this term

Open-ended questions: How did you come to know about the Program? What motivated you to sign-up for the Program? Effects of program on professional and personal

development?

9

Who Has Been Participating in Our Study? (M = Means)

65 (53.7%) female; 56 (46.3%) males 20 to 48 years (M = 28.01) 56 seeking a Master’s degree; 64 seeking a

Ph.D. 1 - 12 graduate courses completed (M =

3.42) 0 - 9 terms of studying (M = 3.29 terms) Markers for 0-14 courses (M = 1.98 courses) TAs for 0 - 14 courses (M = 2.10 courses) Instructors for 0 - 6 courses (M = .76

courses).

4.524.43 4.41 4.4 4.4

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Responding tostudents'

questions in theclassroom

Understandingyour areas toimprove in

teaching

Adjustingteaching

techniques toimprove student

learning

Deliveringcontent in the

classroom

Keeping studentattention in the

classroom

Top 5 Most Important Teaching ItemsV

ery

Low

Very

Hig

h

11

2.382.21

2.13 2.081.97

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Understandingissues in post-

secondaryteaching

Developing acourse

Awareness ofcurrent research

literature onstudent learning

Awareness ofcurrent research

literature oneffectiveteaching

Preparing ateaching dossier

5 Least Prepared for Teaching ItemsV

ery

Low

Very

Hig

h

12

2.172.11

1.811.75

1.7

0

0.5

1

1.5

2

2.5

Developing acourse

Preparing ateaching dossier

Adjustingteaching

techniques toimprove student

learning

Selectingappropriate

teachingtechniques

Awareness ofcurrent research

literature oneffectiveteaching

Difference Between Importance and PreparedIM

PO

RTA

NC

E

- L

EA

ST

PR

EP

AR

ED

13

5 Least Important Teaching Items

3.74 3.71 3.67 3.613.5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Presenting yourteaching

philosophy

Understandingcurrent theories

of teaching

Using computersin the classroom

Awareness ofcurrent research

literature onstudent learning

Discussingteaching with

colleagues

Very

Low

Very

Hig

h

14

What Most Influenced or Motivated you toSign-Up for the CHET/CUT Program?

0 10 20 30

Certificate/ Transcript

Desire to Teach

Become a Professor

Learn AboutTeaching

Improve TeachingSkills

Frequencies

15

How will Your Experience in the CHET/CUT Program Contribute to Your ProfessionalDevelopment?

0 10 20 30 40

Useful in Career

Improve Teaching Skills

Learn Comm'n Skills

Useful in Getting Jobs

Learn Teaching Skills

Frequencies

16

How will Your Experience in the CHET/CUT Program Contribute to Your PersonalDevelopment?

0 10 20 30 40 50

Satisfaction/ Enjoyment

Self-Awareness

TeachingSkills/ Knowledge

Communication Skills

Increased Confidence

Frequencies

17

Responses to Data How can these data be used to

inform the development or fine tuning of our programs?

How can research like this be used to enhance the scholarship of faculy development?

18

ReferencesAbbott, R.D., Wulff, D.A., & Szego, C. K. (1989). Review of research on TA training. In J.D>

Nyquist, R.D> Abbott, & D. A. Wulff (Eds.), New Dirrections for Teaching and Learning, Number 39, (pp. 111-124). San Francisco, Jossey-Bass.

Bandura, A. (1986) Social foundations of thought and action. Englewood Cliffs, NJ: Prentice Hall.

Blackburn, R.T., Beiber, J.P., Lawrence, J.H., & Trautvetter, L. (1991). Faculty at work: Focus on research, scholarship and service. Research in Higher Education, 32, 385-413.

Boice, R. (1992) The new faculty member: Supporting and fostering professional development. San Francisco: Jossey-Bass.

Carroll, J.G. (1980). Effects of training programs for university teaching assistants. Journal of Higher Education, 51, 167-183.

Chism, N. V. N. (1998). Evaluating TA Programs. In M. Marincovich, J. Prostko, & F. Stout (Eds.), The professional development of graduate teaching assistants (pp. 249-262). Bolton: Anker.

Ellis, D., & Schonwetter, D.J. (2001). Help though the gateway: Assessing teaching certificate programs. Paper presented at the Professional and Organizational Development Network in Higher Education, St Louis, MI.

Levinson-Rose, J. & Menges, R. (1981). Faculty Development and the adoption and diffusion of classroom innovation. Review of Educational Research, 51, 403-434.

19

More ReferencesMarincovich, M., Prostko, J., & Stout, F. (1998). The professional development of

graduate teaching assistants. Bolton: Anker. McKeachie,. W. J., (1997). Critical elements in training university teachers. The

International Journal for Academic Development, 2, 67-74. Menges, R. J. (1994). Preparing new faculty for the future. Thought and Action, 10,

81-95.Menges, R.J. and Associates (1999). Faculty in new jobs. San Francisco: Jossey-Bass. Menges. R.J., & Svinicki, M. (1989). Designing program evaluation. To Improve the

Academy, 8, 81-97.Perry, R.P., Menec, V.H., Struthers, C.W. (1999). Feeling in control. In R. Menges and

Associates (Eds.), Faculty in new jobs (pp. 186-215). San Francisco: Jossey-Bass. Perry, R.P., Menec, V.H., Struthers, C.W., Hechter, F.J., Schonwetter, D.J., & Menges,

R.J. (1997). Faculty in transition: A longitudinal analysis of the role of perceived control and type of institution in adjustment to post-secondary institutions. Research in Higher Education, 38, 519-556.

Rice, R. E. (1996) Making a place for the new American scholar. Washington, DC: American Association for Higher Education.

20

More References

Schonwetter, D.J., & Taylor, K.L. (2000). Identifying critical components of effective instructional development. Paper presented to the Society for Teaching and Learning in Higher Education, Brock University, St. Catherines, Ontario.

Smith, K.S., & Kalivoda, P.L. (1998). Academic morphing: Teaching assistant to faculty member. To Improve the Academy, 17, 85-101.

Weimer, M. & Lenze, L. F. (1997). Instructional interventions: A review of the literature on efforts to improve instruction. In R.P. Perry & J. P. Smart (Eds.), Effective teaching in higher education: Research and practice (pp. 205-240). New York: Agathon Press.