QuickEval

uickEvalS I M P L E P E E R E V A L U A T I O N S


Project TeamLauren Taglieri, Project ManagerAri Rubinstein, Tech LeadNoah Levin, Design Lead

Google InterviewNoah Levin. July 8th, 2010Project Advisor: Professor Larry Heimannhttp://www.quickeval.org



Problem

Paper peer evaluations are tedious

Most project groups have different dynamics that can be difficult to identify



Process

J a n F e b M a r c h a p r i l

14th 21st 28th 4th 11th 18th 25th 4th 11th 18th 21st 1st 8th 15th 22nd 29th

Initial Researchand Planning

Wireframing and database design

Development

InterviewsCompetitive AnalysisUse CasesSkills AnalysisRisk Analysis

Wireframes (8 iterations)User Testing (5 teachers, 5 students)Data ModelingNon-Functional RequirementsLogo and Identity

Front-End Design (xhtml, css)Back-End Development (php / ajax)User Testing (5 rounds)Help and Documentation

3 Weeks 5 Weeks 7 WeeksSp

ring

Bre

ak

A 15 Week Project



Research & Planning





Development





ring

Bre

ak



Why peer evaluate?

1) Teachers: Adjust grades by performance2) Students: Learn about themselves

S

jerks

S

S

awesome

S

S

S

S

S

S S

S S

T

S

depressives

S

slackers

B B

A

C

S

average

commonly sees

but actually...S

S

quiet, but good!

Source on types of group members: “ruining it for the rest of Us.” Felps, Will. This American Life. Chicago Public Radio. 18 Dec. 2008.



Why peer evaluate?

1) Teachers: Adjust grades by performance2) Students: Learn about themselves

Rate the quality of their work from 1-5

How much time did they contribute?

Would you work with him / her again?Q: Q: Q:

Jan 09 Feb 09 Mar 09 Apr 09 Jan 09 Feb 09 Mar 09 Apr 09 Jan 09 Feb 09 Mar 09 Apr 09



Wireframing & Database





Development





ring

Bre

ak

Team 2: Lauren, Steve, Tracy, Ari, Noah

67-373 Software Development Project

Advisor: Professor Heimann

March 2, 2009

28

T E A C H E R W I R E F R A M E S Roster

QuickEvalHome

Evaluations Courses

My Account Contact | Site Map | FAQ

Welcome, Mr. Henderson [logout]

Name

Email

Group(s) Current Eval

1. Anderson, John [email protected]

1 Complete

x delete

2. Applegate, Phillip [email protected]

-- Incomplete

x delete

3. Carson, Scott [email protected]

-- Complete

x delete

4. Davis, Sandra [email protected]

1 Complete

x delete

5. Davis, James [email protected]

-- Incomplete

x delete

6. Dickenson, Emily [email protected] --

Complete x delete

7. Elder, Jared [email protected]

1 Incomplete

x delete

8. Goldstein, Doug [email protected]

-- Incomplete

x delete

9. James, Terrance [email protected] --

Complete x delete

10. Johnson, Mark [email protected]

-- Complete

x delete

Group 1

1. Anderson, John2. Davis, Sandra3. Elder, Jared4. Westfield, Karen

+ ADD A GROUP

+ add student(s)

All Courses >67-373 Spring 2009 [edit] Roster Results

+ send an evaluation

Actually dragging the names into groups from the roster would be ideal, but otherwise just editing the group # inline would

work.

Clicking the group name would allow you to edit it. We may add more to these group windows in the future.

Upon clicking this, the user will began adding students to the group and a new button of the same size will appear in the next row under ‘Group 1’.

Sub tabs that are different than the primary navigation. These may load via ajax if possible.

Team 2: Lauren, Steve, Tracy, Ari, Noah 67-373 Software Development Project Advisor: Professor Heimann March 2, 2009

18

QuickEval

Home Evaluations My Account

Contact | Site Map | FAQWelcome, Sarah Wilkins [logout]

S T U D E N T W I R E F R A M E S

Evaluations Survey Page

All Evaluations > 67-373 Spring 2009 > Evaluation #1

Final remarks? *

Would you work with them again? *

How many hours do you think they contributed? *

Did high quality work

Rate the following assumptions *

Took Initiative

Communicated Effectively

Worked well with others

Highly Contributed

SAVECANCEL

yes no

1 2 3 4 5

Jared Brittany Phil Rachael

You are currently evaluating...

Default images

may be used

if users do not

have a gravitar.

Breadcrumbs will

be at the top to

help the user’s

orientation.

Student interviews

showed us that

required fields were

not marked. We will

use a red asterix to

fix this.

Comment bubbles

allow you to make

additional notes

about a question if

necessary.

Dark bubbles

mean you’ve filled

it out already.



S

S

S

S

S

S

sends messages creates users

f xes bugs

creates students / courses

T A

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

S

SS

S

S

S

S

S

S

S

S

uploads evaluation

creates evaluations

reviews evaluationsPe

erReview

s

project teams

Project Advsior

Flow of Information



Development





Development




3 Weeks 5 Weeks 7 Weeks

Spri

ng B

reak



Our Solution

A web application for students and teachers to quickly and efficiently evaluate peers for group projects with a simple and intuitive user interface.



DemoMeet QuickEval



Launched in Spring 2009!



Launched in Spring 2009!

And subsequent versions relaunched in Fall 2009, Spring 2010, and beyond...

It never ends!!



User Testing & Feedback

Over 60% forgot to fill out their evaluations




Over 60% forgot to fill out their evaluationsProfessor’s were creating bad surveys




Over 60% forgot to fill out their evaluationsProfessor’s were creating bad surveysResults were ignored and hard to parse



DemoMeet QuickEval 2.0



Next Steps

Sell! Convince Carnegie Mellon Faculty and Students that this product is useful

Release! Launch the next iterations to new departments

Discover! Get as much user feedback as possible and solve the problems with new technologies

Thank You!Your time is important, I appreciate it



Competitive Analysis

Peer Evaluation Software

iPeer, CATME Terminology is confusing / user interface is overwhelming

Survey Software

SurveyMonkey, Qualtrics, Google Docs, etc,. Not group oriented!



Technology

Front-End

Custom Framework

Back-End Management



Lessons Learned

Technical Skills

Task Planning

Teamwork

Communication

Importance of Testing



Team 2: Lauren, Steve, Tracy, Ari, Noah 67-373 Software Development Project Advisor: Professor Heimann March 2, 2009 33

T E A C H E R W I R E F R A M E S

Results (Individual)

QuickEval

Home Evaluations Courses My Account

Contact | Site Map | FAQ

Welcome, Mr. Henderson [logout]

View Averages | View Individuals

All Courses > 67-373 Spring 2009 > Results >

Final remarks?

Would you work with them again?

How many hours do you think they contributed?

Did high quality work 3

Rate the following assumptions

Took Initiative 2

Communicated Effectively 1

Worked well with others 4

Highly Contributed 5

12 hrs

Yes No

45% 55%

• I wish he spoke up more...

What did Sarah say about Phillip ? GO

It’s tough to say, but I think Phillip over-

estimated on his timesheet. I am still not

sure exactly how he contributed

SWAP

Evaluation #1 : Group 1

Jared

Rachael

Brittany

Everyone You could also

select what

Sarah said about

‘Everyone’ to get

an idea of how

she generally

rates people.

Example of viewing

a comment if you

click on it

Simple language

might help to

select the desired

view of results

Swap to view what

Phillip said about

Sarah



Team 2: Lauren, Steve, Tracy, Ari, Noah 67-373 Software Development Project Advisor: Professor Heimann March 2, 2009 16

Quick Quotes

Quick Stats

Quick Alerts

QuickEval

Home Evaluations My Account

Contact | Site Map | FAQ

S T U D E N T W I R E F R A M E S

Home Page

Welcome, Sarah Wilkins [logout]

Sarah is a great teamate.

Very hard working. My only

complaint is that she does

not take criticism well.

Would you work with

him / her again?

view more »

view more »

Q:

Jan 09 Feb 09 Mar 09 Apr 09

Y

N

Welcome back, Sarah

edit settings

You have 3 days left to complete IS Project Evaluation #1

Jared submitted HCI: Methods Evalution #2. 42 secs ago

Mr. Smith extended the deadline for the IS Project

Evaluation until February 13th. 5 hrs ago

The results have been posted for the HCI: Methods

Evaluation #1. 2 days ago

We might have

2-3 questions that

each survey MUST

contain in order to

receive continuous

results and learn

more about yourself

as a groupmate.

Grabbed from the

anonymous open

ended responses.

Randomized so

that you can get

a more passive

understanding of

your group dynamic.

Will contain important

information like

pending evaluations,

finished evaluations,

and account inactivity.

Users will have

personal avatars

using Gravitar

Documents

QuickEval