View
899
Download
1
Category
Tags:
Preview:
DESCRIPTION
This course is designed to provide the advanced knowledge, procedures, and hands-on experience required for the successful implementation and operation of online course evaluations with the Class Climate software. Topics for this workshop include: creating faculty and departmental reports. This course is recommended for personnel responsible for creating, managing and data collection for course evaluation surveys and already have basic skills with Class Climate.
Citation preview
Center for Excellence in Learning and Technology
Student Evaluation of Teaching: Creating Reports with Class
ClimateLaura Bestler, Instructional Support Specialist
294-5357 • course-evaluation@iastate.edueLearning:
http://www.celt.iastate.edu/elearning/Class Climate:
https://classclimate.its.iastate.edu/
Objectives
• Introductions• Demonstrating different
reports• Preparing reports• Creating reports• Sending results
Establishing Indicators• Effects Every
Report in Class Climate
• Indicators May be Added/Changed with Running Surveys
• Allows for 2 – 11 Options With or without Abstention
Establishing Indicators• Question Group(s) = Question Categories
• Indicators are an Attribute of the Group
• Indicators May be Added while Questionnaire in use
• Indicators Calculate the Mean Value of all Scaled Questions within the Group (Subtotal)
• Any Group Containing a Scaled Question Should have the Indicator Defined
• All Questions to be Considered in the Sub-Total Calculation MUST Match the Number of Options Set in the Indicator Value for that Group
Questions with a Different Scale (number of options) will be individually calculated but excluded from the Sub-Total Calculation.
Indicators in Questionnaires
Weighted Computation of Indicators
• Total Number of Answers for Each Question used for Computation of the Indicator Value
• Example: Question Group with Two (2) Questions, each with a 5-Point Scale, 4 respondents• Q1=4 Respondents / 5, 5, 4, 3 (Question Average =
4.25)• Q2=3 Respondents / 5, 5, 5 (Question Average =
5.0)• Indicator = 5 + 5 + 4 + 3 + 5 + 5 + 5 = 32 Divided
by # of Responses = 32 / 7 Average of Question Group = 4.57
• Not 4.25 + 5 = 9.25 / 2 = 4.62• Question Group Averages are Averaged into Global
Indicator
Calculation of the Average vs. Median• Total Number of Answers for Each Question used
for Computation of the Median• All question responses are sorted• Example of a Question with 5 respondents (odd
#)• Q1=5 Respondents / 5, 5, 4, 3, 1• Average of Question = 3.6
(5+5+4+3+1=18/5=3.6)• The Median is 4 (the absolute middle number)
• Example of a Question with 4 respondents (even #)• Q2=4 Respondents / 5, 4, 2, 2• Average of Question = 3.75 (5+4+2+2=13/4=
3.75)• The Median is 3 (4+2=6/2=3)
Calculation of the Average vs. Median• Always specify when a Group contains scaled
questions• Question Scales within a Group should have
Identical Options• 2 – 11 Options• With or Without Abstention• Mirrored Scales Required if
Positive on Left• Should be set at Question
Level if ‘Traffic Lights‘ will be used
Report Creator• Change your role to [Report Creator]• Click on [Folders] Think: File Cabinet• Click on [Create new Folder]
• Name folder [Example: F2011]
Report Creator: Single Category
• Include / Exclude questions as desired• Produce Reports by Single Category
• First 3 options in pull-down• Only ONE Choice per Selection
• SubUnit: Pick: 1 Subunit / 1 Period• Program of Study (if Activated and
populated): Pick: 1 POS / 1 Period• Instructor Profile – Pick: 1 Instructor / 1
Period
Report Creator: Any Compilation
Any Compilation (Combination) Last 2 options in pull-down
• Most Flexible• Combine Any
Surveys (except General Surveys)
• Choose multiple criteria from a standard choice box
• Allows Multiple Subunits / Periods / Types / Questionnaires
Report Creator: Profile Line• Create profile line emails (may be printed or saved)
• Compares scaled questions across selected surveys• 2-3 optimal (more becomes difficult to read)
Compare Instructor to their DEPT Peers
Up to 8 (eight)courses taught bysingle Instructor
Up to 8 (eight)Instructors whotaught same/similarCourse(s)
Line 1 Entire Faculty Course 1 Instructor A
Line 2 DEPT Averages Course 2 Instructor B
Line 3 Instructor/Course Averages
Course 3 Instructor C
Line 4 Total Average ofCourses 1,2,3
Total Average ofInstructor A,B,C
Report Creator: Profile Line
Mapping Questions Across Surveys• Used to Combine Surveys with Similar Questions
• Automatic• Use When:
• Questionnaires are Identical• Manual
• Use When:• Questionnaires are Somewhat Similar• Worded Differently, and Re-Sequenced
• You decide which questions are similar enough to average together
• Ability to define the number of character differences
Mapping Questions Across Surveys• Include Answers to Open Questions
• Not Extremely Useful when Combining Different Instructors / Courses
• Weighted Averages• Weighted is Default• Number of Responses will be used to Calculate the
Averages• Un-Weighted Averages
• Number of Surveys will be used to Calculate the Averages
• Not Eligible for Export to CSV nor SPSS• Raw Score Data is not Provided, only the Average• Not Widely Used
Weighted vs. Unweighted• By default, the reports are summarized
weighted, meaning that the calculation is effected according to the number of returns.
• Example: If you summarize a survey with 100 return questionnaires and one with 10, the survey with 100 questionnaires counts for more than the one with 10.
• Mathematically expressed: All individual results are summed for each question (each single value that was selected) and divided by the total number of returns (in this example: 110).
Weighted vs. Unweighted• You must explicitly select unweighted compilation
when creating a report. Unweighted calculation precludes the possibility to adequately undertake further processing in statistics programs.
• With unweighted compilations, each survey counts as 1, regardless of how many questionnaires have been processed for a given survey.
• Mathematically expressed: You add the respective total average of each question and divide by the number of the surveys that you are summarizing (in the example above: by 2).
• Individual response data is lost.
Evaluate Subgroup Data• Used with SINGLE-CHOICE questions
Example: If your survey asked: “Reason for taking this course” and the responses included: “Required” and “Elective”, you could produce a report which included ONLY the responses from those who attended as a required part of their major.
Evaluate Subgroup DataOpens a 2nd screen to allow multiple criteria
Name report as desired
Evaluate Subgroup: Multiple Criteria
Name report as desired
Merging Survey Data• Combine Reports Created by Evaluating a Subgroup
Example: Combine any reports, including those reports created using: Evaluate Subgroup
Single Profile Line Comparison• Combine any 2 Subgroup Reports
Example: Compare the responses from the 2 reports: “Required” and “Elective”, you could see which group evaluated the course higher / lower.
Report for Comparison on Questionnaire
Report for Comparison
Report for Comparison on Questionnaire
Report for Comparison on Questionnaire
The “Hook” Profile line into standard Report
Administrator Reports• Change your role to
[Administrator]• Note: You are unable to do
anything with surveys/questionnaires as a Report Creator
“Printing” Results• Click on [Batch Events] (Left Menu)
Creates PDFs to save on your local CPU or network
For surveys that have already been evaluated, the evaluations of a selection of courses can be requested. There are two possibilities:• The PDF reports of the selected
courses are created together in a common PDF file. This is advantageous for example, when the PDF reports for multiple courses are to be printed at the same time.
• The PDF reports requested are summarized as single PDF files in a ZIP file. This is advantageous for example, when the single PDF files are to be passed on in electronic format, deposited in a public folder or, when one wants to print them individually.
“Sending” Results
• Click on [Batch Events]
Send PDF ReportsYou can determine that an instructor receives all evaluations of his courses accumulated in one email. To do this, please activate the option “Reports of all courses of an instructor collected in one email”. Should this not be the case, the instructor will receive a separate email for each course. Please observe the defined maximum size for email attachments (“System Settings/Class Climate Settings/Network Settings/ Maximum document size”). In “System Settings/Text Templates” the relevant email text may be edited (“EMAIL: Results of a survey” and “E-MAIL: Results of a survey (Summary)”).
IMPORTANT: Discuss with your Department Chair how the reports should be sent to
Instructors
IMPORTANT: Discuss with your Department Chair how the reports should be sent to
Instructors
“Sending” Results
• Click on [Batch Events]
Sending PDF Reports as Instructor Profiles• You can send your instructor profile for surveys that
have already been evaluated as a batch email. By activating the option “Include courses with secondary instructors in instructor profiles”, courses with additional report recipients are also included.
• You can determine whether or not the answers to open questions are to be included in your instructor profile. To do this, please activate the option “Include answers to open questions”.
• If the respective instructor, in addition to his profile, is to receive a detailed report for each of his courses, simply activate the option “Also send single PDF reports”.
• Furthermore, you can determine whether profiles and PDF reports are sent solely to the respective instructor (“Send to instructors”) or only to a specified email address (“Send all to the following email address”).
CELT Training Adapted from:
• Oliver, M. (2011, October 25). Advanced topics in Class Climate: The Report Creator(s) Version 4 & 5 [Scantron webinar].
• Scantron Corporation. (2011). Class Climate Course Evaluation System User Guide v5.0. Eagan, MN: Scantron Corporation. http://classclimate.its.iastate.edu
Recommended