Management Department- Graduate Program Assessment …

Management Department- Graduate Program Assessment Reports 2018/2019

Figure 4.2 – Assessment Report | Management and Leadership (MA) AY 1819

Program: MA Management and Leadership What changes did you make to the program in AY1819? MNGT 5000 is currently being revised to provide a stronger introduction to the four learning themes of strategy, leadership, human capital, and problem solving. A basic simulation, Capstone Core, is being added to the course to better prepare students for the Capstone Simulation in MNGT 5650 and Comp-XM in MNGT 6000. MNGT 6000 is currently being revised to be a case-study course with assessment using Comp-XM and will roll out in Spring 2, 2020. What updates do you anticipate in AY 1920? We would like to replace the current BUSN 5200 (Finance for Managers) with a new course that reinforces the four learning themes and better prepares students for the more advanced courses of MNGT 5650 and MNGT 6000. PLO 1: Students will be able to explain the important terminology, concepts, principles, analytic

techniques and theories used in management and leadership. Goal Process/Measurem

ent instrument Current Results

Analysis of Results Action Taken Graph/ table

At least 80% of students achieve ‘medium” or better results for PLO 1.

Case Study. Direct, Internal, Summative.

90.3% of the students achieved minimum or better scores. This represents a 1.7% decline over the prior year.

All students across all campus types, except Online, are achieving this goal at a satisfactory rate. We see the same pattern of results with PLO 1 & 2. Online students were successful at 79th percentile, which falls below our 80% criteria.

Beginning Spring 2020 we will utilize COMPXM in MNGT6000 as our new assessmenttool.

See below

PLO 2: Students will be able to effectively apply important terminology, concepts, principles, analytic techniques and theories used in management and leadership when analyzing situations.

Goal Process/Measurement instrument

Current Results

Analysis of Results Action Taken Graph/ Table

At least 80% of students achieve ‘medium” or “high” results on PLO 2.


85% of the students achieved minimum or better scores. This represents a 7% decline over the prior year.

The largest decline came from PLO 2b. PLO 2c showed a drop of 7%. However, PLO 2a did not show a significant drop.

Beginning Spring 2020 we will utilize COMPXM in MNGT6000 as our new assessment tool.

See below.

PLO 3: Students will be able to effectively integrate important concepts, principles and theories used management and leadership when developing solutions to multifaceted problems in complex situations


Current Results

Analysis of Results Action Taken Graph/ Table

At least 80% of students achieve ‘medium” or “high” results on PLO 3.


85% of the students achieved minimum or better scores. This represents a 7% decline over the prior year.

The results show that military and online students are performing slightly below 80% goal. We believe that the changes to the new assessment tool will improve these results.

Beginning Spring 2020 we will utilize COMPXM in MNGT6000 as our new assessment tool.

See below

Figure 4.2 – Assessment Report | Human Resources Management (MA) AY 1819

Program: MA Human Resources Management PLO 1: Students will be able to explain the important terminology, facts, concepts, principles, analytic techniques, and theories used in human resources management.

1a. Shows that he or she understands the relevant terminology concepts and theories. Goal Process/Measurement

instrument Current Results

Analysis of Results

Action Taken

Graph/table

At least 80% of students will achieve “Moderate” or “Good” results on the assessment case study.

Case study assignment at the end of HRMG 6000. Direct, Summative, Internal.

The current assessment results, while lower than previous years are still well within the 80% targeted goal.

The overall lower results in large part stem from a more rigorous interpretation of the current case study.

A new more up to date and relevant case study is currently being evaluated for assessment purposes.

See below

PLO 2: Students will be able to effectively apply important terminology, facts, concepts, principles, analytic techniques, and theories used in human resources management when analyzing situations.

2a. Identifies important topics and issues 2b. Identifies and correctly applies relevant concepts and theories in the analysis 2c. Supports conclusions or claims with relevant facts and sound reasoning


Current Results

Analysis of Results

Action Taken

Graph/table

At least 80% of students will achieve “Moderate” or “Good” results on the assessment examination.


The current assessment results, improved over last year and are still well within the 80% targeted goal.

These subtle changes result from more of an overall concentration on practical application of core HRMG concepts.


See below.

PLO 3: Students will be able to effectively integrate important facts, concepts, principles, and theories used in human resources management when developing solutions or analyzing situations.

3a. Recognizes interrelationships among relevant concepts or theories and can integrate them in the analysis


Current Results

Analysis of Results

Action Taken

Graph/table

At least 80% of students will achieve “Moderate” or “Good” results on the assessment examination.


The current assessment results, while lower than previous years are still within the 80% targeted goal.

The overall lower results in large part stem from a more rigorous interpretation of the current case study.


See below.

Figure 4.2 – Assessment Report | Information Technology Management (MA) AY1819

Program: MA Information Technology Management What changes did you make to the program in AY1819?

• Standardized ITM textbooks per course across Webster Campuses • Built ITM faculty teaching review process • Built procedure guide for instructors with steps for teaching via WebEx+ • Converted ITM program from classroom and online to classroom/ WebEx+ and Online for all sites • Positioned program to successfully sustain itself at any site with any enrollment size per class without cancelling classes • Tested the implementation of WebEx+ • Updated ITM 5000 to new textbook version • Updated ITM 5200 to new textbook version

What updates do you anticipate in AY 1920? • Update two ITM program courses for online and WebEx+ • Execute onsite ITM faculty review process • Talk to student advisors about the ITM Program features, value and future.

PLO 1: Students will be able to explain the important terminology, facts, concepts, principles, analytic techniques, and theories used in the field of information technology management.


Current Results

Analysis of Results

Action Taken Graph/ Table

At least 80% of students achieve “medium” or “high” results on assessment exam.

The capstone exam in the ITM 6000 Capstone course reviews direct, summary, external and comparative terms and concepts in Information Technology Management

The AY 201819 results indicate that the average for the test was above the 80% mark in all four categories

The four categories break down as follows: - 85.98% - 86.59% - 86.59% - 87.20% 86.67% overall average as a group

The “Metro” and “Online” groups scored above the “Military” and “St. Louis” groups

Review the content of the consistency test and introduce the pre-assessment test in ITM 5000.

See below

PLO 2: Students will be able to effectively apply important terminology, facts, concepts, principles, analytic techniques, and theories used in the field of information technology management when analyzing complex factual situations.


Current Results

Analysis of Results Action Taken Graph/table

At least 80% of students achieve “medium” or “high” results on assessment exam

Capstone Exam in ITM 6000 Capstone Course.

The scores for PLO2 were above the 80% level with scores as follows: - 87.80% - 86.59% 87.19% overall average as a group

The “Metro” and “Online” groups performed above the 80% measure with the “Military” and “St. Louis” groups performing below the 80% level as a group

Promoting to advisors that the students complete the theory courses rather than trying to waive theory courses based upon previous experience and courses completed prior to attending Webster University

See below.

Page Break

PLO 3: Students will be able to effectively integrate (or synthesize) important facts, concepts, principles, and theories in the field of information technology management when developing solutions to multifaceted information technology management problems in complex factual situations.


Current Results

Analysis of Results

Action Taken Graph/table

Students will integrate facts, concepts, principles, theories and practice in the development of the final information technology management project and score 80% or more.

Capstone Exam in ITM 6000 Capstone Course, Direct, Summative, External, Comparative

AY 201819 results indicate the average was above the 80% mark. The categories breakdown as follows: - 87.20% - 87.20% - 86.59% 87.00% overall average for the group.

The “Metro” and “Online” groups performed above the 80% measure with the “Military” and “St. Louis” groups performing below the 80% level as a group

The instructor needs to work beyond the technical subject matter use to integrate the technical skills and management principles into a technical manager as the industry is moving to more and more Information Systems management. The future of the manager is going to become more and more important as we project manage company tasks to fulfill industry goals instead of just managing people doing technical tasks

See below.

Figure 4.2 – Assessment Report | Organizational Development (MA) AY 1819

Program: MA Organizational Development What changes did you make to the program in AY1819? We initiated ORDV WebNet+ delivery systems in South Carolina, Henderson Hall campus in the Washington, D. C., and for the St. Louis, MO, main and extended campuses. This led to a higher level of cooperation between campuses offering the two programs. What did occur was the use of learnings from previous more informal assessments of program effectiveness to develop formal assessment plan for AY 2019-2020. What updates do you anticipate in AY 1920? Formalized ORDV Assessment Portfolio made-up of course and assessment timetables for both the Graduate Certificate and M. A. in ORDV Programs. Instructions, tools and rubrics introduced to assist instructors teaching ORDV 5000, 5300, and 6000 where assessment for all three learning outcomes will take place. Possible integration of three ORDV courses into the HRDV program: ORDV 5000, 5400, and 5950. (Deletion of three HRDV courses that are legacy courses without significant updating in 10 years to provide space in the curriculum for introduction of the ORDV courses.)

PLO 1: Students can assess a learning portfolio documenting organizational development competency acquisition over the course of the program.

Goal Process/

Measurement instrument

Current Results Analysis of Results Action Taken

All ORDV students

receiving the M. A. degree will do a self-

rating on competency proficiency attainment

at the end of the ORDV

6000 capstone course.

Professional OD Competency Self-Rating Tool, covering 17 competencies, developed from an extensive survey of experienced OD practitioners in the field.

Only three students finished the assessment tool in St. Louis. Program Lead worked with them to share results with each other. They were graduating in May, 2019. The instrument had not been fully perfected at that point. There was no rubric. We used it for professional development, not assessment purposes. It was agreed that 2019-2020 would be the first year that the ORDV Program would be formally assessed against all three program learning outcomes. Spring 2, 2018 and Spring 2, 2019, were both used to prototype instruments and their utility for formal assessment in 2019-2020.

In May 2018, and 2019, a total of five students took ORDV 6000. These were the first two graduate students to formally graduate from the program. Their performance on the tool indicated that it was robust and did a good job of differentiating students who were outstanding students in the program. According to informal faculty comments and formal grades, four students would have been expected to know what competencies were most important to the field and to know how to measure their own level of proficiency on those competencies. Having taken 12 M. A. courses, they were all familiar with all aspects of the field. They also had experience applying the concepts, tools, and methods in practical situations. Results showed that all five students had a command and an understanding of the theoretical/conceptual aspects of the 17 competencies. They were able to rate both importance and self-proficiency on the instrument. Two of the students had jobs where they practiced OD. As expected, they had achieved higher levels of competency attainment than the three students whose experience was limited to ORDV course work and applications.

The next steps were to perfect the tool itself so that we could measure summary results across the seventeen competencies and create summary quantitative measures we could use to submit for individual course sections. The first utilization of the refined tool took place during the assessment year of 2019-2020 in ORDV 5300 OA, Fall One, 2019. Results are currently being summarized according to gap scores between importance and proficiency measures across the class. This number will be expected to fall within a range for each individual in the class. The range is computed based on an approximated proxy for competency attainment measured against completion of four core courses in the program. The criterion score on the applied rubric will be that 80% of ORDV majors will fall within the range of measures predicted for each program.

PLO 2: Students possess an organizational development personal and professional development plan by the end of the program.


Current Results

Analysis of Results Action Taken

During the years of prototyping, AY’s 2017-2018, 2018-2019, the goal was to have all students preparing to graduate in May of each year, complete an ORDV Personal and Professional Development Plan for ongoing use in their professional development. The goal was to have this take place in ORDV 6000, in Weeks 8-9, just before their program was complete and they graduated. There were five graduate students who fell into this category.

Professional Development Plan document procured from professional sources. The tool encourages the student to set measurable goals for two ORDV competencies they wish to develop. These competencies can come from the PLO#2 Competency Self-Assessment instrument individual results or from other sources related to the professional field of OD. For the previous two academic years mentioned, there was no use of formalized rubrics. Instead, the faculty member teaching the ORDV 6000 course reviewed the completed plans in class sharing to help each student think through the implications of a professional development plan and determine how they could use peers, self-reports, and supervisor/mentors to develop and track the plan once it was formalized.

As stated, five graduate students completed their PP&D Plans just before completing their Capstone ORDV 6000 course and graduating. Students collaborated on developing the plans and sharing them in class. All five students fully completed their plans with two competency development goals, timetables, use of needed resources and measurable outcomes.

The primary results were detailed completion of the plan elements as mentioned under current results. For both AY, in ORDV 6000, the class reviewed the deliverables in class and along with the instructor, helped to ensure that the goals were clear, specific, measurable, accurate, reliable, and time sensitive. We also helped each graduate student to create detailed plans with timetables, review, measures, and behavioral specifics. Each of these five students met the criteria due to high quality class discussion and teamwork.

The action taken was to build-in the learning from these pilot administrations into a rubric that could be used to measure assessment during the AY 2019-2020 In both ORDV 5300 and ORDV 6000. In ORDV 5300, both the Graduate Certificate and M. A. programs in ORDV will be assessed. In ORDV 6000, the M. A. Program Learning Outcome #2 will be assessed using a different criterion measure. Students in ORDV 5300 OA, Fall One, 2019, have just submitted their completed plans. This instrument will also be administered three more times when ORDV 5300 is offered Spring One, 2020, and ORDV 6000 is offered in two separate sections in Spring Two, 2020.

PLO 3: Students can demonstrate acquisition of individual course learning objectives through theoretical application papers, case discussions, written case analyses, class role plays and simulations, use of electronic and social media, and organizational development project interventions papers and logs.

Goal Process/ Measurement

instrument

Current Results Analysis of Results Action Taken

Tool to be developed to measure performance against this learning outcome. The intention was to develop a schedule of testing in multiple class sections where expected differences could be measured.

20 item multiple choice test created and prototyped in AY 2018-19

Tool was prototyped in ORDV 6000, Spring Two, 2019, for the M. A. degree in ORDV. Two of the three students in the class produced 20 correct answers (20 pts as measured by raw scores) out of 20 possible. The third student scored 11 pts out of 20 possible points.

The instructor in this class was the Program Lead. He taught the small group of upcoming ORDV graduates in a very close collaborative fashion and came to know their OD professional styles, understandings, and skillsets in an intimate way. in this class and in other classes where he had had these students, the instructor had learned that the two who scored high were the ones who were the best students, got the highest grades, and were able to apply the professional models to their work most effectively. The one who scored outside of the acceptable/criterion range had been deeply involved in alternative change management approaches in his workplace. He was a well-credentialed practitioner who had been immersed in a related but in some ways very different professional approach to change. He had difficulty with the program all along because he tended to oversimplify complex issues.

For the AY 2019-2020, the Program Lead, with input from other ORDV faculty, decided to embed the 20 item test into the curriculum at three points: (1) At the beginning of ORDV 5000, before any course content had been introduced, as a benchmark measure; (2) again, in Week Eight of ORDV 5300, after the four foundational classes had been taught, and (3) then in Week Nine, of ORDV 6000, after all course content had been provided for the entire program. Instructors and students would receive directions at each point of administration and results would be automatically be submitted through the course software.

Figure 4.2 – Assessment Report | Master of Public Administration (MPA) AY 1819

Program: MPA Master of Public Administration

What changes did you make to the program in AY1819?

Text updated with most current publications available.

What updates do you anticipate in AY 1920?

Continue to update new text to most current.

PLO 1: Students will be able to explain the important terminology, facts, concepts, principles, analytic techniques and theories used in the field of public administration.


Current Results

Action Taken

At least 75% of students will score medium or high on the exam.

Multiple choice exam in PADM 6000. Direct, Summative, Internal.

No data to review

Exploring and moving toward a new assessment tool.

PLO 2: Students will be able to effectively apply important terminology, facts, concepts, principles, analytic techniques and theories used in the field of public administration when analyzing complex factual situations.


Current Results

Action Taken

Students will be able to demonstrate an understanding and application of the above mentioned at a rate of medium to high results. It is expected that at least 75% of students achieve the goal

Multiple choice exam in PADM 6000. Direct, Summative, Internal, Comparative

No data to review


PLO 3: Students will be able to effectively integrate (or synthesize) important facts, concepts, principles and theories in the field of public administration when developing solutions to multifaceted public administration problems in complex factual situations.


Current Results

Action Taken

Students will be able to demonstrate an understanding and application of the above mentioned at a rate of medium to high results. It is expected that at least 75% of students achieve the goal.

Multiple choice exam in PADM 6000. Direct, Summative, Internal, Comparative

No data to review


Figure 4.2 – Assessment Report | Business and Organizational Security Management (MA) AY 1819

Program: MA Business and Organizational Security Management What changes did you make to the program in AY1819? What updates do you anticipate in AY 1920?

PLO 1: Students will be able to explain the important terminology, facts, concepts, principles, analytic techniques and theories used in the field of business and organizational security management.


Current Results

Analysis of Results

Action Taken

Graph/table

At least 80% of the students score in the “Moderate” or “Good” category on the multiple choice exam.

Multiple choice cumulative exam and case study in Capstone course. Summative, internal. A case study which requires responses to an individual scenario for each of the six core course subject matter areas. Summative, internal.

See below

PLO 2: Students will be able to effectively apply important terminology, facts, concepts, principles, analytic techniques and theories used in the field of business and organizational security management when analyzing complex situations.


Current Results

Analysis of Results

Action Taken

Graph/table

At least 80% of the students score in the “Moderate” or “Good” category on the case study analysis

Multiple choice cumulative exam and case study in Capstone course. Summative, internal. A case study which requires responses to an individual scenario for each of the six core course subject matter areas. Summative, internal

See below.

PLO 3: Students will be able to effectively integrate important facts, concepts, principles and theories used in business and organizational security management when developing solutions to multifaceted business and organizational security management problems in complex situations.


Current Results

Analysis of Results

Action Taken

Graph/table

At least 80% of the students score in the “Moderate” or “Good” category on the case study analysis.

Multiple choice cumulative exam and case study in Capstone course. Summative, internal. A case study which requires responses to an individual scenario for each of the six core course subject matter areas. Summative, internal

See below.

Figure 4.2 – Assessment Report | Human Resources Development (MA) AY 1819

Program: MA Human Resources Development What changes did you make to the program in AY1819? What updates do you anticipate in AY 1920? PLO 1: Students will be able to explain the important terminology, facts, concepts, principles, analytic techniques, and theories used in the field of human resources development. a. Human Resources Development diagnosis as critical first step in organizational change b. Application of Force Field Analysis to Gulfport Transit

c. Analysis of Readiness for change Goal Process/Measurement

instrument Current Results

Analysis of

Results

Action Taken

Graph/table

. At least 80% of students achieve "medium" or "high" results on assessment exam.

Complex Case Study in Human Resources Development. Direct, Indirect, Summative, Internal, Comparative

See below.

PLO 2: Students will be able to effectively apply important terminology, facts, concepts, principles, analytic techniques, and theories in the field of human resources development when analyzing complex factual situations.

a. Training and Development b. Group Development and Change c. Organizational Development and Change d. Career Management


Current Results

Analysis of Results

Action Taken

Graph/table

At least 80% of students achieve ‘medium” or “high” results on comprehensive case study completed in capstone course, HRDV 6000

Complex Case Study in Human Resources Development. Direct, Indirect, Summative, Internal, Comparative

See below.

PLO 3: Students will be able to effectively integrate (or synthesize) important facts, concepts, principles, and theories in the field of human resources development when developing solutions to multifaceted human resources development problems in complex factual situations.

a. Use of quantitative evaluation criteria to determine effectiveness of Human Resources Development project.


Current Results

Analysis of Results

Action Taken

Graph/table

At least 80% of students achieve ‘medium” or

“high” results on Program Learning Outcome #3

Complex Case Study in Human Resources Development. Direct,

Indirect, Summative, Internal, Comparative

See below.

Figure 4.2 – Assessment Report | Space Systems Operations Management

Program: MS Space Systems Operations Management What changes did you make to the program in AY1819? What updates do you anticipate in AY 1920? PLO 1: Students will be able to explain the important terminology, facts, concepts, principles, analytic techniques, and theories used in the field of space systems operations management.


Current Results

Analysis of Results

Action Taken

Graph/table

At least 80% of students achieve ‘medium” or “high” results on the program assessment.

Multiple Choice and 2 Short Essays, Summative, Direct, Internal

See below

PLO 2: Students will be able to effectively apply important terminology, facts, concepts, principles, analytic techniques, and theories used in the field of space systems operations management when analyzing complex factual situations.


Current Results

Analysis of Results

Action Taken

Graph/table


Multiple Choice and 2 Short Essays, Summative, Direct, Internal

PLO 3: Students will be able to effectively integrate (or synthesize) important facts, concepts, principles, and theories in the field of space systems operations management when developing solutions to multifaceted space systems operations management problems in complex factual situations.


Current Results

Analysis of Results

Action Taken

Graph/table


Students averaged 81.00%, which is an improvement over the last 2 years.

Figure 4.2 – Assessment Report | Doctor of Management Program AY 1819

Program: Doctor of Management

PLO 1: Students will be able to synthesize and apply relevant theories to explain domestic or global phenomena.


Current Results

Analysis of Results

Action Taken

Graph/table

All students develop, submit, and have a theoretical paper accepted in a target journal at the completion of the fourth class, DMGT 7350.

Paper publication--Direct, Formative, External, Comparative

PLO 2: Students will be able to demonstrate linkages between theoretical findings and the practical application of these findings.


Current Results

Analysis of Results

Action Taken

Graph/table

All students submit a theoretical paper (after working independently with a faculty member in DMGT 7900, the seventh class) ) to either a regional or a national management conference

Paper presentation-- Direct, Summative, External, Comparative Conference paper reviews are typically peer-reviewed as well as reviewed by the Conference Program Director

PLO 3: Students will be able to identify and justify a novel research niche, design ideas, propose hypotheses and appropriate methodology, and conduct a rigorous study.


Current Results

Analysis of Results

Action Taken

Graph/table

To have 100% of students who have completed coursework be engaged in dissertation research. To have at least 3 graduates of the program each year be awarded their doctoral degree.

Dissertation Learning Outcomes 1. Show mastery of relevant literature and be able to field questions using extant research findings 2. Demonstrate mastery of research methods 3. Provide logical arguments based on theoretical support 4. Provide a comprehensive literature review on the relevant constructs and provide meaningful connections between the literature and the student’s current research study.

Documents

Management Department- Graduate Program Assessment …