MANUAL FOR GENERATING REPORTS FOR

STUDENT FEEDBACK ON TEACHING

(updated August 31, 2016)

This manual demonstrates how faculty members, chairs, and directors may create various reports of student

feedback from courses in Digital Measures: Course Response.

Logging into Digital Measures: Course Response ................................................................................................ 1

Viewing student response rates for a current or previous feedback period ............................................................ 2

Suggestions for increasing response rates during a current feedback period ......................................................... 4

Viewing student feedback (ratings and comments) for each course section .......................................................... 5

Viewing response rates and overall course/instructor items ................................................................................... 9

Viewing each student’s unique pattern of feedback (ratings and comments) ...................................................... 11

1

Logging into Digital Measures: Course Response

There are three ways to log into Digital Measures Course Response:

1. Visit https://cr.digitalmeasures.com/login/kennesaw/faculty/authentication/showLogin.do.

Enter your KSU Email (NET ID) and Password, and then click Log In.

2. A direct link to this login page appears as a Campus Bookmark on the KSU D2L Brightspace home page:

Feedback on Teaching: Faculty Link. There is also a link for students to use during an open review

period.

3. A direct link to the Course Response login page also appears in the left column of the Digital Measures

Manage Activities Home Page: Click on the link to Course Response.

You’ll be taken to the Course Response login page (see option 1 above).

Enter your KSU Email (NET ID) and Password, and then click Log In.

2

Viewing student response rates for a current or previous feedback period

To view the schedule for upcoming feedback periods, visit:

http://digitalmeasures.kennesaw.edu/course-response/schedule.php.

FACULTY MEMBERS: You can view how many students have visited the Digital Measures Web site to

provide feedback for your current courses or previous courses. To improve response rates, consider sharing

this information with students to encourage them to provide feedback before the feedback period ends.

CHAIRS/DIRECTORS: You can view similar information for all courses taught by faculty members in your

department/school. During a current feedback period, you can use this information to remind faculty to

encourage their students to provide feedback.

Login to Digital Measures Course Response (refer to page 2 of this manual).

In the menu at left, click on View Respondents.

The screen will display the response rates for the current feedback period (if the feedback period is open) or

the most recent feedback period (if no feedback periods are currently open). The example below shows what

a faculty member might have seen after Fall 2015.

Total Instruments = Number (Percent) of students who received invitations to provide feedback for a course.

Responses = Number (Percent) of students who viewed the Digital Measures feedback form for a course.

Instruments filled out = Number (Percent) of students who submitted feedback for a course. Students may

decide not to submit feedback, which is why there may be fewer Instruments filled out than Responses.

NOTE: This report will indicate whether a previous course had been open for feedback from students, even if

no student chose to provide feedback. In this case, the number of Instruments filled out will be 0 (0.00%).

Use the Term and Course dropdown menus to display information for a specific term or course. Click View after selecting a term or course.

Click on the Worksheet option to download a report that provides information for each course.

The following page describes the content of the Worksheet report.

3

WORKSHEET REPORT: RESPONSE RATE SUMMARY

If you click the Worksheet option (see previous page), you will receive a spreadsheet that contains a Response

Rate Summary for each course. The following columns appear in this summary report:

COLUMN DESCRIPTION

Term Fall, Spring, or Summer term.

Year Calendar year of the term.

Session KSU Registrar’s code for the session the course was offered.

Course Id The CRN assigned to the course.

Course Prefix Course prefix assigned to the course.

Course Number Course number assigned to the course.

Section Course section assigned to the course.

Course Title Title of the course.

College (Course) KSU Registrar’s two-letter code for the College in which the course is taught.

Department (Course) KSU Registrar’s code for the department/school in which the course is taught.

Instructor Name(s) and Net ID(s) of the course instructor(s) assigned to the course.

Total Total number of students who receive invitations to provide feedback for a course.

Completed Number of students who viewed the Digital Measures feedback form for a course.

(Note: Students have the option to complete the form without answering any item.)

Completed % Percent of students who viewed the Digital Measures feedback form for a course.

Answered Number of students who submitted feedback for a course.

Answered % Percent of students who submitted feedback for a course.

4

Suggestions for increasing response rates during a current feedback period1

Student Ratings2 Administered Online versus on Paper

The use of online student ratings has increased steadily with the growth of Web-based surveys. Online delivery

offers several advantages over paper-and-pencil administration. Students can respond outside of class at their

convenience, freeing class time for other activities (Dommeyer, Baum, & Hanna, 2003; Layne, DeCristoforo, &

McGinty, 1999). Response rates to open-ended questions posted online tend to be higher (Johnson, 2003) and

written comments lengthier (Hardy, 2003; Johnson, 2003, Layne et al., 1999). Moreover, online directions and

procedures can be uniform for all classes, enabling instructors to be less involved in the administration process

(Layne et al., 1999).3

The chief disadvantage of online ratings is lower student response rates to the fixed items, which threaten class

representativeness (Sorenson & Reiner, 2003). Lower response rates occur for several reasons, among them

student concern about anonymity, computer technical difficulties, and the time required to respond outside of

class (Dommeyer et al., 2003). Some instructors fear lower response rates create a negative bias because students

who are dissatisfied with the course or instructor might be more likely than others to respond (Johnson, 2003).

However, correlations between response rate and overall ratings of the instructor and course are, on average, quite

low (Benton, Webster, Gross, & Pallett, 2010a; Johnson, 2003), which suggests response bias is less likely.4

In spite of the disparity in response rates, researchers have consistently found no meaningful differences between

online and paper surveys. When the same students respond under both formats, the correlations are high between

global ratings of the instructor (.84) and course (.86) (Johnson, 2003). Further, no meaningful differences are

found in individual item means, number of positive and negative written comments (Venette, Sellnow, &

McIntire, 2010), scale means and reliabilities, and the underlying factor structure of the ratings (Leung & Kember,

2005). Similarly, when different students respond to online and paper surveys, no meaningful differences are

found in student progress on relevant course objectives, global ratings of the course and instructor, frequency of

various teaching methods (Benton et al., 2010a), subscale means (Layne et al., 1999), the proportion of positive

and negative written comments (Hardy, 2003), and the underlying factor structure (Layne et al., 1999).5

Suggestions for increasing online response rates: Higher online response rates are more likely

when instructors clearly communicate their expectations for compliance. Online response rates also tend to be

higher when students complete ratings for more than one course (2003). Ensuring student confidentiality,

monitoring response rates, encouraging instructor follow-up, sending reminders, acknowledging and rewarding

high response rates, and integrating the process into the campus culture may also be associate with higher response

rates (see The IDEA Center, 2008).6 [Please do not reward students for high response rates; see Footnote 6]

1 The excerpt below appears on page 11 of the following report:

Benton, S. L., & Cashin, W. E. (2012). Student ratings of teaching: Summary of research and literature (IDEA

Paper 50). Manhattan, KS: The IDEA Center. Retrieved from http://ideaedu.org/research-and-papers/idea-

papers/idea-paper-no-50/ *** Click on the URL above for the full article and for a reference list that includes the citations below.

*** Tom Pusateri, CETL Associate Director, added footnotes that comment on KSU’s system for student feedback. 2 The article on this page frequently refers to student ratings. At KSU, we prefer to refer to these data as student feedback. 3 To read summary reports on KSU’s experiences with online student feedback, visit:

http://digitalmeasures.kennesaw.edu/course-response/reports.php 4 Refer to the URL in Footnote #2 for information on response rates at KSU. KSU maintained no records of response

rates with its paper-and-pencil forms, so KSU has no data comparing response rates in the print and online versions. 5 KSU did not collect data from its paper-and-pencil forms, so there is no way to conduct comparable studies at KSU.

However, it is unlikely that KSU’s results would deviate substantially from the findings discussed in this paragraph. 6 Please do not reward students for high response rates as there is no university-wide policy for offering rewards to

all students. However, please do adopt any of the other suggestions in this paragraph to increase response rates.

5

Viewing student feedback (ratings and comments) for each course section

Login to Digital Measures Course Response (refer to page 2 of this manual) and select “Run Reports”.

Click on the drop-down menu and select Student Feedback: Ratings and Comments – by Course.

You will now see a list of options for generating the report.

The following page describes each option. NOTE: Some options may not be available to you.

6

Term(s) to Use: To select one or more academic terms/semesters, click on the link to Change selection

A pop-up window will open with a list of options, with the

most recent term at the top of the list. Select the boxes for

the term(s) you want to examine for your report. Note that

some terms have multiple options. If you want to view

your responses from Fall 2014, select all options that

begin with the words “Fall 2014” and then click “Save” at

the bottom of the full list.

Whom to Include: If you have this option, click on Change selection to view various options for your report.

Click on the small arrow next to an option to expand that option.

College (Instructor) refers to the home College of an instructor. You will

see codes for instructors who teach in your department/school.

Course lists all courses by Prefix, Number and Section. You may see

courses from other programs if any instructor teaches a course for another department/school. Use this option to select course sections that have unique identifiers (e.g., laboratory sections, sections offered online, sections that are part of a learning community).

Course Number permits you to create a report that includes only

Graduate courses or only Undergraduate courses.

Course Prefix lists the common prefixes of courses taught by instructors

in your department/school, including prefixes from programs in other departments/school. This option will permit you to exclude courses taught in departments/schools other than your own.

Course Prefix and Course Number permits you to select the specific

prefix/number combinations (e.g., only 1000- or 2000-level courses).

Department (Instructor) enables you to create a report that includes

only instructors in your department/school and excludes instructors from other units who teach courses for your department/school.

Instructor lists all of the instructors who teach courses in your

department or school in alphabetical order.

Section lists all of the section numbers in alphanumerical order. To

create a report on Online courses, select sections that begin with a W.

7

Report Options: Select the statistics to include in the report in addition to frequency data. Different Colleges,

departments, or programs may be interested in different statistics, so consult with your chair or colleagues.

File Format: Select the File Format (Word, Excel, PDF, html) and Page Size (Letter, A4) from the menus.

Run Report: After you have made your selections, click on Run Report (at the top right of the screen).

While your report is being processed, you will see an animation:

After the animation ends, you will be able to view, print or save the report.

The next page provides a sample report using one of the College of Humanities and Social Sciences forms.

8

Course Evaluation - University This sample report has been modified for illustrative purposes. Term: Spring 2014 - Full Term Comments are highlighted in yellow. Course: PREFIX Number-Section: Course Title This is the type of data you will receive if you select Mean/Average Instructor: Last Name, First Name as the statistics to include on the Run Reports screen.

Responses: 32 out of 58 (55.2%) This is the response rate for the course.

Below are examples of some of the items that appear on the College of Humanities and Social Science form.

Question

Strongly Agree

(4) Agree

(3)

Disagree

(2)

Strongly Disagree

(1)

Do Not Know

(0) Mean /

Average

The instructor is knowledgeable about the course material. 18

56% 11

34% 2

6% 1

3% 0

0% 3.44

The instructor has an effective style of presentation. In the example above, all 32 students who submitted the form responded to this item. The calculations of the percentages and Mean/Average include all 32 students who submitted the form and responded to this item. The instructor is available for consultation.

The instructor challenges me to think. 16

55% 13

45% 0

0% 0

0% 0

0% 3.55

The instructor treats students with respect. In the example above, 3 students who submitted the form did not respond to this item. The calculations of the percentages and Mean/Average include only the 29 students who responded to this item. The instructor makes course objectives clear.

The questions below are the university-wide questions approved by Faculty Senate for inclusion on all versions of the

forms. Question

Strongly Agree

(4) Agree

(3)

Disagree

(2)

Strongly Disagree

(1)

Do Not Know

(0) Mean /

Average

Overall the content of this course contributed to my knowledge and intellectual skills.

12 38%

13 41%

2 6%

3 9%

2 6%

3.13

In this example, 2 of 32 students responded “Do Not Know.” The calculation of percentages includes all 32 students who responded, but the Mean/Average omits the 2 students who responded “Do Not Know” and is based on 30 students.

Please comment on the course content

• Comments from different students will appear in bulleted format.

• Comments appear verbatim, with no corrections in spelling, format, or grammar.

Question Strongly

Agree (4)

Agree (3)

Disagree

(2)

Strongly Disagree

(1)

Do Not Know

(0) Mean /

Average

The instructor was effective in helping me learn. 18

64% 7

25% 2

7% 0

0% 1

4% 3.59

In this example, 28 students responded, 1 who responded “Do Not Know.” The calculation of percentages includes 28 students who responded. The Mean/Average omits 1 who responded “Do Not Know” and is based on 27 students.

Please comment on the instructor’s strengths

• In this report, you cannot determine which comments are associated with responses to other items on the form.

Please comment on ways the instructor might improve.

• In order to determine which comments are associated with other items on the form, follow the directions below for “Viewing each student’s pattern of responses in each course”.

9

Viewing response rates and overall course/instructor items

The following report will provide student response rates and summary data on two university-wide items that

appear on all of KSU’s ratings forms, one item on course content and one on instructor effectiveness:

“Overall the content of this course contributed to my knowledge and intellectual skills.”

“The instructor was effective in helping me learn.”

Students rate each item on a 4-point scale (1=Strongly disagree, 2=Disagree, 3=Agree, 4=Strongly agree):

Login to Digital Measures Course Response (refer to page 2 of this manual) and select “Run Reports”.

Click on the drop-down menu and select Response Rates and Overall Course/Instructor Items– by Course.

You will now see a list of options for generating the report:

Term(s) to Use: To select one or more academic terms/semesters, click on the link to Change selection. A

pop-up window will open with a list of options, with the most recent term at the top of the list. Select the boxes

for the term(s) you want to examine for your report. Note that some terms have multiple options. If you want

to view your responses from Fall 2014, select all options that begin with the words “Fall 2014” and then click

“Save” at the bottom of the full list.

File Format: Select the File Format: Microsoft Word (.doc), Microsoft Excel (.xls), PDF, or Web Page (.html).

Run Report: After you have made your selections, click on Run Report (at the top right of the screen).

You will receive a report that contains the following columns for each course (see next page):

10

COLUMN DESCRIPTION

Term Fall, Spring, or Summer term.

Year Calendar year of the term.

Session KSU Registrar’s code for the session the course was offered.

Course Id The CRN assigned to the course.

Course Prefix Course prefix assigned to the course.

Course Number Course number assigned to the course.

Section Course section assigned to the course.

Course Title Title of the course.

College (Course) KSU Registrar’s two-letter code for the College in which the course is taught.

Department (Course) KSU Registrar’s code for the department/school in which the course is taught.

Instructor Name(s) and Net ID(s) of the course instructor(s) assigned to the course.

Total Total number of students who receive invitations to provide feedback for a course.

Completed Number of students who viewed the Digital Measures feedback form for a course. (Note: Students have the option to complete the form without answering any item.)

Completed % Percent of students who viewed the Digital Measures feedback form for a course.

Answered Number of students who submitted feedback for a course.

Answered % Percent of students who submitted feedback for a course.

“Overall the content of this course contributed to my knowledge and intellectual skills.”

Content Strongly Agree # Number of students who indicated “Strongly Agree” to the item above.

Content Agree # Number of students who indicated “Agree” to the item above.

Content Disagree # Number of students who indicated “Disagree” to the item above.

Content Strongly Disagree # Number of students who indicated “Strongly Disagree” to the item above.

Content Strongly Agree % Percent of students who indicated “Strongly Agree” to the item above.

Content Agree % Percent of students who indicated “Agree” to the item above.

Content Disagree % Percent of students who indicated “Disagree” to the item above.

Content Strongly Disagree % Percent of students who indicated “Strongly Disagree” to the item above.

“The instructor was effective in helping me learn.”

Instructor Strongly Agree # Number of students who indicated “Strongly Agree” to the item above.

Instructor Agree # Number of students who indicated “Agree” to the item above.

Instructor Disagree # Number of students who indicated “Disagree” to the item above.

Instructor Strongly Disagree # Number of students who indicated “Strongly Disagree” to the item above.

Instructor Strongly Agree % Percent of students who indicated “Strongly Agree” to the item above.

Instructor Agree % Percent of students who indicated “Agree” to the item above.

Instructor Disagree % Percent of students who indicated “Disagree” to the item above.

Instructor Strongly Disagree % Percent of students who indicated “Strongly Disagree” to the item above.

11

Viewing each student’s unique pattern of feedback (ratings and comments)

Note: Options and contents of drop-down menus may differ from the illustrations in this manual depending on

your College and the permissions you have to view data.

Login to Digital Measures Course Response (refer to page 2 of this manual) and select “Run Reports”.

Click on the drop-down menu and select Create a new report.

You will see the following options:

For some options, click “Change Selection” to open up a separate window where you can change that option.

For other options, click the drop-down menu to change that option.

Term(s) to Use: Select any or all of the academic terms as described more fully on page 7 of this manual.

Whom to Include: If you see this option, you can select the options described on page 7 of this manual.

Data to Include: If you see this option, click Change Selection to open up a pop-up window.

12

You can choose to Include All data in your report

(both numerical data and student comments) or

Only Quantitative Data (numerical data, omitting student

comments).

By choosing Only Selected Items, you can choose to include

data only from one of KSU’s forms or, by clicking on the arrow

next to a form, you can choose only specific items from that

form. For a list of forms and items on each form, visit

http://digitalmeasures.kennesaw.edu/course-response/view-

forms.php

After you have made your selections, click Save.

Grouping Method: You can organize the report by the options described more fully on page 7 of this manual.

Statistics to Include: Click Change Selection to select the statistics to include in the report.

File Format: You can select CSV, DOC, PDF, HTML, or XML format, and you can select whether to generate

a Single report with all data or Multiple reports that separate Course items from Instructor items. When you

have completed your selections, click Run Report in the upper right of the screen.

Here is an example of part of a report you will receive if you select the CSV and Single options.

Each row in the report provides one student’s responses to all of the items on the feedback form. You can use

Excel to sort the data and examine patterns in student responses to the multiple-choice and open-ended items.