Evaluating Individual Contribution Toward

Group Software Engineering Projects

J. Huffman Hayes,

T. Lethbridge,

D. Port, USC

Int’l Conf. on Software Engineering 2003

Portland, Oregon 5/8/03

Outline

Group Projects– Characteristics– Benefits

Related Work Grading individual effort

– Criteria– Techniques– Bad things

Best Practices Conclusions and Acknowledgments

Group Projects - Characteristics

Staple of S/W eng. courses Teams of 3 or more Perform s/w development lifecycle One or more semesters May have real customer Develop artifacts

Group Projects - Benefits

Learn to work in groups Learn from each other Learn to schedule Learn to rely on others Allows more complex systems

The Challenge

Grading individual effort Handling “drop outs”

Related Work

Speck [7] – group projects as part of “cooperative learning”

Johnson et al [3] – essential factors for effective group learning– Positive interdependence– Effective interpersonal skills– Accountable

Related Work (continued)

Schultz [6] found motivating slackers a concern Gates et al [2] – motivate students to contribute

equally McKinney [5] suggests combination of:

– Division of Labor report – Group grade – Assessment of dossier– Peer ratings

Grading Individual Effort - Criteria

Fair Consistent Reflect achievement of educational objectives Provide good, understandable feedback Encourage student, avoid discouragement Easy on grader Control grading validity

Grading Individual Effort – Criteria (continued)

No grade inflation/deflation Accurate and unbiased Discourage “risk managing” by students

Techniques/Grading Schemes

1) Group mark for all

2) Division of labor (DOL) report, then separate marks for parts

3) DOL, then grade relative contribution

4) In class pop quizzes on project

5) Cross validate w/results of individual work

Criteria to Grading Technique Map

Table 1. Grading criteria mapped to individual effort grading.

Grading Criterion

All same grade

Each reports own effort

Each evaluates self and others

Each reports in weekly Web log

Each reports in journal

Quiz in class

Project reviews

Individual effort analysis

Fair X X X Consistent X X X Reflect Educ. Objectives

X X X

Provide feedback

X X X X

Encourage students

X X

No grade inflation

X X

Easy on grader

X X X X

Combining Schemes

Fixed % of individual grade based on scheme 2) or 3), remainder on 1)

S1*0.9 + S3*0.1 Group grade of S1 multiplied by a factor from S3

S1 * S3/100 Distribute group grade points

– Group grade = 90%, so 90*3 points spread by 3 students

NOTE: Spell out your intent in the syllabus

When Bad Things Happen

Encourage students to work it out Act as intermediary Use Industry model (Chen [8])

Suggested Best Practices

Allow peer evaluation, but monitor Use demos/quizzes to test project knowledge Have students maintain/post individual effort

info Cross validate individual effort evaluation with

group evaluation

Suggested Best Practices (continued)

Use multiple methods for grading individual effort

Evaluate grading scheme using criteria

Conclusions and Acknowledgments

Determine your grading goals Establish a scheme Evaluate scheme

Thanks to:– Hong-Mei Chen - U. of Hawaii– Tony Baxter, Paul Piwowarski, Jerzy Jaromczyk - U.

of Kentucky

References

[2] Gates, A.Q.; Delgado, N.; Mondragon, O., “A structured approach for managing a practical software engineering course”, 30th Frontiers in Education conf, IEEE, 2000, pp. T1C/21 -T1C/26.

[3] Johnson, D. W., Johnson, R. T., and Smith, K. A. “Cooperative learning: Increasing college faculty instructional productivity”, ASHE-ERIC Higher Education Report 20, 4, Graduate School of Education and Human Development, The George Washington University, 1991.

[5] McKinney, K. “Tips for Grading Group Work”, Illinois State University, web page as of Sept. 2002, www.cat.ilstu.edu/teaching_tips/handouts/tipsgroupwork.shtml

[6] Schultz, T.W., “Students assessing teams”, proc. 29th Frontiers in Education conf, IEEE, 1999, pp. 13B2/1 -13B2/3.

[7] Speck, B.W., “Pedagogical Support for Classroom Collaborative Writing Assignments”, ASHE-ERIC Higher Education Report, 28, 6, Jossey-Bass, a Wiley Company, 2002, pp. 1-139

[8] Chen, H-M., web page http://www.cba.hawaii.edu/HMCHEN/home.htm

Back-up slide – NEEDED? Or move in line?

Evaluation of techniques – what has worked at Ottawa, USC, UK