Measuring team performance with Polls

MEASURING TEAM PERFORMANCE

Peter Rainey Delivery Lead | Project Manager

@PeterRaineyAU

Not everything that can be measured should be!

Delivery Leads Melbourne Meetup - 10th March 2016

WELCOME

• Who I am?

• Mobile devices are welcome, please set to silent but keep it handy.

• Poll Anywhere - Interactive

• Why this topic? - Hackathon

• Some questions to get to know our group

• Group exercise

• Close

Measuring Team performance - Not everything that can be measured should be!

2


3

If you would like to participate in tonight’s session, I’ll be using “Poll Everywhere”.

This allows you to respond to polls or vote using the app on a smartphone or tablet.

Respond via … iOS and Android

1) Install the Poll Everywhere App

2) Join presentation…

PollEv.com/dlmelbmeetup676

Respond via … Web browser

PollEv.com/dlmelbmeetup676

OR


4

Q. If your team gave you a box and walked away.

What would be inside?

WHY THIS TOPIC?


5

Well it because of Hackathons…

I recently participated in a couple of hackathons. Which was very satisfying and great fun.

It amazed me how over such a short amount of time, individuals who didn’t know each other

could work together and produce a solution that exceeded the problem owners expectations.

All in 48 hours!

It may be wonder how this was possible. I compared this experience to the teams I’ve worked

with in the past. They were not like this at all.

Performance was not really measured but if I think about it, there was a real time measuring

(monitoring) taking place with all the team members.

Clear common goal – Trust – collaboration – communication – regular feedback – purpose –

accountable – alignment – skill mix


6

Q. Have you participated in a hackathon or innovation day?


7

Stages of

team growth Commonly used framework, developed in the mid-1960s by Bruce W. Tuckman.


8

Q. At what stage is your team on their journey to high performance?


9

What is a team?

A group of people with a full set of complementary skills required to complete a task, job, or

project. A team becomes more than just a collection of people when a strong sense

of mutual commitment creates synergy, thus generating performance greater than

the sum of the performance of its individual members.

What is measurement?

Values made meaningful by quantifying into specific units. Measurements act as labels

which make those values more useful in terms of details. For example, instead of saying that

someone is tall, we can specify a measurement and specify that the individual is 6 feet tall.

What are metrics?

Metrics are the means to monitor how one is performing against one's goals. Metrics are best

when defined based on a clear set of objectives. Metrics are not the goal.

SO IS THAT ALL WE HAVE TO DO


10

Measure something and it will magically happen?

How does measuring weight ensure weight loss?

How does tracking monthly revenue actually bring in the money?

What should you measure and how should you report on it?

Don’t waste your valuable time measuring that provoke no questions and generate no

action.

Perhaps the team is a better position to decide the measures.


11

What makes a good metric?

Meaningful. They should tie back to something your company wants to

achieve. For example, when your goal is highly responsive support, time to

first response is an ideal metric. Resolution time may not matter.

Moveable. You should measure things on which your team can have

impact. In Taylor Morgan’s words, “If there are metrics that aren’t moving,

or we feel like they aren’t important, we just drop them.”

Authentic. Your reports must tell a true story. It’s possible to use real numbers

to send a misleading message. Be honest even when it hurts.

Contextualized. Numbers in isolation can be stripped of meaning, so

provide them in context.

Consistent. The trends over time are usually more important than specific

data, and looking back over a quarter or a year can give you some

fantastic insights and encouragement.


12

Q. What comes to mind when thinking about measuring team performance?

Word Cloud

SPORT IS A WORLD OF MEASUREMENT

• Weight

• Time

• Distance

• Height

• Speed

• Power


13


14

I love my sport. I am a swimmer and triathlete.

When I first started I would study races and memorise the

times of all my competitors. This obsession spread into

studying world ranked swimmers. Their personal bests,

world records and their winning times at various events.

I can spend hours looking at training data and race results

Its worse now as more data is available. Power, Heart Rate,

DPS, stride length, cadence etc.

I feel this can lead to distraction and not being in the

moment especially when training and racing.


15

Question: Who knows what Sabermetrics is?

Answer: SABeRmetrics is the statistical analysis of baseball.

SABR (Society for American Baseball Research)

It began in the middle of the 20th century and turned mainstream

after the bestselling book Moneyball was released in 2004. A movie

was released in 2011 starring Brad Pitt.

It also has it’s own Manifesto.

So what is Moneyball’s message?

Measuring the right

things in the right way


16

Moneyball’s message…

Moneyball describes how the Oakland Athletics baseball team

used carefully chosen statistics to build a winning team with a

limited budget.

Their rigorous statistical analysis demonstrated that on-base

percentage and slugging percentage were better indicators of

success, and these player qualities were cheaper to obtain on

the open market than more historically valued qualities.

So statistics such as stolen bases, runs batted in, and batting

average, typically used to gauge players value, where seen as

subjective, often flawed and relics of the 19th-century.

Measuring the right



17

Moneyball’s message con’t…

By re-evaluating the strategies that produce wins on the field

the team in 2002 won 20 consecutive games and also finished

first in the League.

With only a budget of US$44 million they were competitive with

larger teams such as the New York Yankees, who spent over

US$125 million.

The key business lesson…. businesses continue to use the wrong

statistics.

Measuring the right



Lets take a sport like Aussie Rules Football.

Players attempt to kick goals.

Lets pretend we are a coach selecting a new player for the team.

We are considering a player who kicks an average of 7 goals a game.

Question: Are they … a Good player, a Bad player or a Great player?

Answer: A good coach would know based on this limited information it would be impossible to answer this question with certainty.

Football is more than just scoring…but some fans would disagree.


18


Players can tackle, mark, handball and assist their team mates with scoring….but they can also make mistakes that may benefit other team.

Some additional information that might help….

If I tell you this player averages 7 goals, 20 tackles and 15 marks a game.

Question: Are they… a Good player, a Bad player or a Great player?

Answer: A poor coach might be tricked into answering this question.

A good coach would realise those statistics only mention certain aspects of effectiveness, to truly assess this player you would still need more information.


19


A good coach would want to know how the player made use of their resources.

More information that might help to know…

• How many shots at goal did they make

• How many mistakes did they make

• How many possessions did their team have during game

• How many minutes did they play


20

Example for one player

Ok suppose I gave you all the statistics.

Would you feel more comfortable making a decision on

that player?


What about situational data?

Suppose those 7 goals were accumulated by a tall 25 year old man, against the worst defences, in one-sided games, in a league of 7 year olds!

Question: Do you still feel good about this players statistics?

Not every set of data is sufficient in helping us make good decisions.


21


22

• Performance metrics can actually motivate the behaviours of athletes, employees, students

and business partners.

• The key is to use an appropriate metric. What is appropriate will depend on the team as

every team is different.

• The current stage (Forming, Storming etc) of the team should be taken into account when

deciding on.

Be mindful that the longer particular metric is used, the more likely it can are to be

unconsciously - or for that matter consciously- “gamed”.

Appropriate Use of Metrics


23

Here is a common statement that I came across a number of times.

“We measured the wrong thing. The statistic we relied on to assess our performance was disconnected from our overall objective of profitability. As a

result, our strategic and resource allocation decisions didn’t support that goal.”

Appropriate Use of Metrics


24

Velocity Burndown

charts

Items (stories)

in-progress

Peer code

reviews

Acceleration

Acceleration

Interruptions

per day (limited

time metric)

Task hour

burndown

Feature

validation with

customers/users

Anonymous

surveys

Obstacles

escaping

iterations

Resource

Utilization

Adherence to

doneness/

acceptance

criteria

Usability tests

(i.e., time on

task)

Adherence to

checklists

(checklist item

burnup)

Earned Value

Backlog

prioritization

Estimates vs

actuals

Story point

burnup

Trust level within

the team

Hours Worked

Increasing T-

shaped skills

Value delivered On time

delivery

Learning fast

and frequently

Lines of Code

Common Measures

MEASURE WISELY


25

Suggestions of Good Metric characteristics

(Hartmann & Dymond - 2007)

• Affirms and reinforces Lean and Agile principles.

• Follows trends, not numbers.

• Belongs to a small set of metrics and diagnostics.

• Measures outcome, not output.

• Is easy to collect. • Provides fuel for meaningful conversation.

• Provides feedback on a frequent and regular basis.

• May measure Value (Product) or Process.

• Encourages "good-enough" quality.

Evaluation Checklist for a Metric

PERFORMANCE IN REAL TIME


26

Many measures are historical. It may be better not to assess teams by looking at

lagging indicators.

But instead get a better understanding of the current level of teamwork.

How can you really measure this without relying on subjective statements like “We

work well as a team”?

Some ways to diagnose a team’s potential and performance in real time…

1. Measure Interaction

2. Address Conflict

3. Align Decisions

4. Create Accountability

5. Trust



27

1. Measure Interaction

Teams thrive when their members are courteous to each other, take time to fully

understand where team members are coming from, and expand on ideas from other

team members to create new strategies previously thought impossible. You can

measure those interactions by recording how many positive or encouraging

comments versus negative or degrading comments are made during meetings.

Because negative emotions are much more powerful than positive ones, high-

performing teams consistently show ratios between 3-to-1 and 7-to-1 (positive to

negative). Anything above 8-to-1 signals artificial harmony and avoidance of tackling

tough issues, while ratios below 1-to-1 are signs of failing and dysfunctional teams.



28

2. Address Conflict

For many, conflict is scary and needs to be avoided at all cost. This is true when we

are talking about negative or combative conflict in which team members are

personally attacked, or if some members have a win-at-all-costs, my-way-or-the-

highway attitude. But avoiding conflict entirely means you may not be tackling the

challenges that really need to be addressed. High-performing teams thoroughly

discuss these critical challenges and exchange their experiences and ideas openly to

find the best possible solution. In most cases, this is a solution no one thought of at the

beginning of the meeting.

Record how many team members actively participate in the idea-generation and

selection process. High-performing teams strive for an equal participation of all team

members, while dysfunctional teams have just one or two people who dominate the

discussion.



29

3. Align Decisions

Critical decisions should be made only after all team members have contributed to,

aligned with, and bought into the developed idea. Everyone should be deeply

knowledgeable about the important details and able to defend the decision based

on facts.

Dysfunctional or “OK” teams agree to decisions made by the most vocal person and

show no commitment to follow through with the steps outlined for implementation.



30

4. Create Accountability

Members of high-performing teams hold each other to the same behavioral and

performance standards and enforce these standards without the team leader’s help.

Do team members show up on time, and are they well-prepared for the team

meetings? Are disruptions, such as side conversations, checking e-mails, and playing

with electronic gadgets, kept to a minimum?

When team members don’t enforce standards and rely on the leader to enforce

rules, they’re really saying that they do not care enough for each other, don’t

understand that it is their responsibility to speak up, or fail to get engaged and put in

the necessary work.



31

5. Trust

Trust, similar to employee engagement, is a psychological construct. It needs to be

made demonstrable in order for it to be measured. In other words, what will be the

tangible evidence that trust exists and how will this be captured?

Identify the types of things that are associated with trust. Put these on a continuum

where you see trust identifiers at one end and the opposite at the other. Assign a

rating scale between the continuums.

Curiosity Defensiveness

Active open listening Poor listening & monologues

Direct communications Indirect communication

Transparency & Clarity Hidden & Distortion

Receptive to Change Doubting of Change


32

Q. What should you measure when measuring team performance?

Brainstorm

&Vote

GROUP EXERCISE

10-15 mins

1) Please form 4 – 5 groups

2) Each group should select at least two of the measurements listed in previous poll.

3) Pick the most important one according to your group

4) At the end of 10-15 mins present why it was considered the most important by the group

5) Lets discuss

Note: To extend this further the group could also discuss the measure that should not be used.


33

KEY POINTS

• Something that is measurable is not necessarily meaningful

• Choose the right metrics

• Measure the right things in the right way

• Measurement can drive behaviour

• Don’t be distracted by looking at lagging indicators

• Measure performance in real time

• Avoid ongoing measurement for measurement’s sake.

• Regularly re-evaluate the measures you are using

• Not everything that can be measured should be!


34


35

“Not everything that matters can be measured

and not everything that can be measured matters”

- Albert Einstein

THANK YOU!…

FOR SHARING YOUR EXPERIENCES

Also thanks to our hosts

and Meetup organiser Olga Burakevych &Louise Lonergan .

Measuring Team performance - Not everything that can be measured should be! 36

ONE LAST THING…


37

Tonight's Meetup was “__________________”.

REFERENCES & INSPIRATION

Cannot Measure Productivity by Martin Fowler

Hack: Up or back by Group of MBA Candidates (IMD Lausanne Switzerland)

The Juice in Hackathons is in the Teams, Not the Ideas by Mark Batson Baril

How commitment affects team performance by J W Bishop, Dow Scott

No-Nonsense Guide to Measuring Productivity by W. Bruce Chew

The True Measures of Success by Michael J. Mauboussin

What Gets Measured Gets Done. Or Does It? By Ruth Henderson

An Appropriate Use of Metrics by Patrick Kua

Appropriate Agile Measurement by D. Hartmann and R. Dymond

Guidelines for Measuring Trust by Katie Delahaye Paine

What is a Good Agile Metric? by Mark Levison

Measuring Trust by Third Horizon


38

http://martinfowler.com/bliki/CannotMeasureProductivity.html

http://www.managementexchange.com/hack/up-or-back

http://www.resologics.com/resologics-blog/high-performing-hackathon-teams

https://works.bepress.com/dow_scott/42/

https://hbr.org/1988/01/no-nonsense-guide-to-measuring-productivity



https://hbr.org/2012/10/the-true-measures-of-success&cm_sp=Article-_-Links-_-End of Page Recirculation

http://www.forbes.com/sites/ellevate/2015/06/08/what-gets-measured-gets-done-or-does-it/#698ad48a7860

http://martinfowler.com/articles/useOfMetrics.html

http://www.innovel.net/wp-content/uploads/2007/07/appropriateagilemeasurementagilemetrics.pdf

http://www.instituteforpr.org/wp-content/uploads/2003_MeasuringTrust.pdf

http://www.infoq.com/news/2009/11/good-agile-metrics

http://www.thirdhorizon.com.au/joomla/index.php/about-us/news/136-measuring-trust

Documents

Measuring team performance with Polls