Faculty Perceptions

of Fall 2011 IDEA

Preliminary Report to CNU Faculty Senate

April 2012

By Deborah Moore, Director, OAEA

for IDEA TaskForce & CNU Faculty

Purposes of the Survey

• gather information about Fall 2011 IDEA experience . . . targeting the online. . .

• learn what information you would like to have about IDEA timeline, reports, etc.

• identify what you need to know about the reports to help you improve instruction in your classroom and those planned collectively by your department.

Background Information

Response Rate: 192 submitted of 394 invited (49%); 8 reported “none of my classes were evaluated this fall”)

By subgroup: (not all respondents provided a response to the status item)

Adjunct, 30 of 148 (20%)Restricted, 44 of 78 (56%)Probationary, 31/53 (58%)Tenured, 80 of 115 (70%)

Which best describes your

status in Fall 2011?

Answer Response %

adjunct 29 15%

restricted 47 24%

probationary, hired before Fall 2011 26 14%

probationary, new hire as of Fall 2011 8 4%

tenured 82 43%

Total 192 100%

Home Departments

Which type of administration

did you experience this Fall?

Answer Res-

ponse

%

none of my classes were evaluated this fall 8 4%

online only, short form 82 43%

online only, diagnostic form 40 21%

online, both short and diagnostic forms 34 18%

paper only, diagnostic form 24 13%

paper and online, short form 2 1%

paper and online, both short & diagnostic 2 1%

Total 192 100%

Knowledge about the IDEA

0%10%

15%

25%

50%

Importance: Teaching Effectiveness

Overall, how important is IDEA to you with respect to

your own teaching efforts and improvement practices.

Satisfaction: Teaching Effectiveness

Overall, how satisfied are you with IDEA with respect to your

own teaching efforts and improvement practices.

Importance: Evaluation

Overall, how important is IDEA to you with respect to CNU's

process for instructor/course evaluation?

Satisfaction: Evaluation

Overall, how satisfied are you with IDEA with respect to CNU's process

for instructor/course evaluation?

Teaching Effectiveness:

Importance by Satisfaction

(Q1.6)Overall, how satisfied are you with IDEA with respect to your own teaching efforts

and improvement p...

Very Dissatisfied DissatisfiedSomewhat Dissatisfied

NeutralSomewhat

SatisfiedSatisfied Very Satisfied Total

Not at all Important

4 0 0 0 0 0 0 4

(Q1.5)Overall, how important is IDEA to you

with respect to your own teaching efforts

and improvement prac...

Very Unimportant 0 1 0 1 1 3 0 6

Somewhat Unimportant

1 4 3 2 4 0 1 15

Neither Important nor Unimportant

0 0 0 3 0 0 0 3

Somewhat Important

1 3 6 15 14 17 1 57

Very Important 2 3 5 3 19 41 1 74

Extremely Important

0 0 1 0 3 8 5 17

Total

8 11 15 24 41 69 8 176

Evaluation: Importance by Satisfaction

(Q1.8)Overall, how satisfied are you with IDEA . . .instructor/course evaluation

Very Dissatisfied Dissa-tisfied

Somewhat Dissatisfied Neutral

Somewhat Satisfied Satisfied

Very Satisfied Total

Not at all Important

4 1 0 0 0 0 0 5

(Q1.7)Overall, how important is IDEA .

. . for instructor/course

evaluation

Very Unimportant

2 4 0 2 0 0 0 8

Somewhat Unimportant

3 3 1 0 0 0 0 7

Neither Important nor Unimportant

0 2 4 7 1 1 0 15

Somewhat Important

1 5 6 7 20 4 0 43

Very Important1 6 9 6 17 20 3 62

Extremely Important

3 5 9 2 8 7 2 36

Total 14 26 29 24 46 32 5 176

Please rate the following aspects of the your IDEA experience:

Statistic

I knew

how

to se

lect

obje

ctive

s for e

ach o

f my

cours

es.

The on

line F

IF w

as e

asy t

o com

plete

.

My c

olleag

ues w

ere a

ble to

help

me w

ith th

e FIF,

if I

needed

it.

The O

ffice

of A

ssess

ment

was

able

to re

solve

issu

es

I had

with

com

pletin

g the F

IF.

I knew

when

the ID

EA ad

min

istra

tion w

as

sche

duled.

I know h

ow to u

se m

y repo

rts to

impro

ve in

stru

ct-

tion.

I know h

ow to u

se m

y IDEA

repor

ts fo

r ten

ure an

d

perso

nnel a

ctivi

ties.

Overa

ll I th

ought t

hat th

e onlin

e exp

erienc

e was

more

conv

enient t

han inco

nvenie

nt.

Online, Short Form

Mean 4.14 4.14 3.63 3.36 4.03 3.82 3.60 3.28

SD 0.90 0.63 0.80 0.81 0.93 0.94 0.99 1.25

N 73 72 71 72 73 72 72 72

Mean 3.77 4.05 3.92 3.21 3.81 3.74 3.42 3.33

SD 1.04 0.69 0.75 0.57 0.60 0.97 1.06 1.42

N 39 39 38 39 39 39 38 39

Mean 3.78 4.09 3.65 3.41 4.00 3.79 3.47 3.26

SD 0.88 0.58 0.98 0.88 0.95 0.77 0.90 1.11

N 34 33 32 33 34 34 34 34

Mean 4.17 3.06 3.37 3.11 4.04 3.96 3.83

SD 0.76 0.64 1.07 0.68 0.91 0.91 1.01 NA

N 24 18 18 18 24 24 24

Scale values

Strongly

Disagree

=1

Disagree

=2

Neither

Agree/

Disagree

=3

Agree=4Strongly

Agree=5

Online, Short Form

Online, Short &

Diagnostic Forms

Online, Diagnostic

Form

Paper, Diagnostic

What did you do to encourage or ensure that your students completed their IDEAs for your classes? (Select all that

apply.)

Statistic

Discussed the

importance of student

participation in IDEA

evaluations

Discussed my choice

of objectives and how

they related to the class

Offered rewards for

participation

Set aside classroom time for students to bring laptops

and complete

the evaluation

I didn't do anything to encourage

IDEA participation

Online, Short Form (N=72)

% 88 21 4 6 10

Online, Diagnostic Form (N=39)

% 72 26 18 15 15

Online, Short & Diagnostic Forms

(N=34)% 76 24 9 9 12

Paper, Diagnostic (N=21)

% 76 33 10 10 10

Stat-

istic All 1st Page

Stats at

end

and

items I

added Other

Comments

primarily

Online, Short Form

(N=74)% 51 18 7 1 23

Online, Diagnostic

Form (N=39)% 77 5 0 3 15

Online, Short &

Diagnostic Forms

(N=34)

% 59 12 12 3 15

Paper, Diagnostic

(N=24)% 50 8 13 17 13

Which elements of your individual course reports do

you use?

Statis

tic

weeks

11&12

weeks

12&13

weeks

13&14

(don't

change

the

timefra

me)

weeks

14&15 other

Online, Short Form (N=71) % 3 6 61 25 6

Online, Diagnostic Form

(N=38)% 3 5 45 42 5

Online, Short & Diagnostic

Forms (N=34)% 9 6 50 26 9

Paper, Diagnostic (N=24) % 8 13 50 21 8

Currently the IDEA is administered weeks 13 and 14

of the term. What weeks would you prefer?

IDEA Webpages

• http://www.cnu.edu/assessment/idea/index.asp

• http://www.theideacenter.org/

Preliminary Summary/Highlights

• Survey response rates were acceptable,

although low (56%-70%) for subgroups

except adjunct (20%).

• Nearly every department had faculty

participating in the survey.

• As expected more respondents indicated

having participated with an online

experience (160/192 or 83%) and paper

was limited to a small subgroup (26/192

or 14%). A small group had no classes

evaluated and ended their survey after

reporting their faculty status (8/192 or

4%).

• Although not ideal, these values can

serve as a benchmark for comparison

about faculty perceptions of the online

application of the IDEA.

• The majority of the respondents indicate

good knowledge about IDEA and related

procedures (75%); however 25% indicate

the need information about both the form

and procedures.

• Faculty were asked to provide an overall

rating of importance and satisfaction

about two contexts of IDEA use: IDEA for

personal teaching improvement, and IDEA

as used by CNU for course/instructor

evaluation.

• In the 1st

of 2 contexts, use of IDEA for

personal teaching improvement efforts,

84% or 148/176 respondents indicated

the tool was important (EI + VI + SI).

Within that subgroup, 74% were satisfied

and 14% were dissatisfied with IDEA.

Although room for improvement, overall

this is a positive outcome.

• Similarly 80% or 141/176 respondents

indicated the tool was important (EI + VI

+ SI) in the context of CNUS use of IDEA

for course/instructor evaluation. Within

this subgroup of those who indicate

CNU’s use of IDEA for course/instructor

evaluation is important, 57% were

satisfied and 32% were dissatisfied.

Continued

• Efforts to determine what is

dissatisfying—and addressing the

responses should help to shift

these ratios in a more positive

direction

• Strategies instructors used to

encourage student completion of

IDEA form are similar except

those involved with the

Diagnostic form (either paper or

online) used slightly more

strategies than those involved

with the Short form.

• Weeks 13 and 14 (the current

timeframe) was preferred by the

largest percent of respondents in

the various subgroups (by form),

however there was also support

for pushing the administration to

weeks 14 and 15.

• Regardless of format subgroup,

comments were fewer in quantity

and less helpful qualitatively.