© Fraunhofer IESE

WITH CROWD-RE TO BETTER REQUIREMENTS

Dr. Jörg Dörr

© Fraunhofer IESE

2

BA Distance

Number of people

Obstacles (oceans, mountains)

Flexibility

Travel time

Travel comfort

Price

Vehicle(s)

Origin Destination

© Fraunhofer IESE

3

ScalabilityScalability

© Fraunhofer IESE

4

0

50

100

150

200

250

300

350

Max. Bus Passenger Capacity(Standing and Seating)

Articulated

Bi-articulated

Straight-sided design

You

ng

man

JN

P625

0G

Ikar

us

280

Yel

low

Co

ach

Cro

wn

Sup

erco

ach

© Fraunhofer IESE

5

0

100

200

300

400

500

600

700

800

900

1000

Max. Airplane Passenger Capacity(All-Economy Configuration)

Jet engines

Wide-body planes

Bo

ein

g 7

47-X

Air

bu

s A

380-

X

Siko

rsky

IM /

S-40

Do

ug

las

DC

-X

Wri

gh

t B

roth

ers

© Fraunhofer IESE

6

Vehicles Getting People from A to BVehicles Getting People from A to B

© Fraunhofer IESE

7

There Will Always be Exceptions…There Will Always be Exceptions…

© Fraunhofer IESE

8

BA Availability of requirements

Number of stakeholders

Project challenges

Constraints

Time

Quality

Cost

…

Vehicle(s)

No requirements Requirements elicited, validated, and documented

© Fraunhofer IESE

9

0

5

10

15

20

25

1.1.1970 1.1.1980 1.1.1990 1.1.2000 1.1.2010

Recommended Number of Participants

1970 1980 1990 2000 2010

RE Interpersonal Elicitation “Vehicles”RE Interpersonal Elicitation “Vehicles”

Workshops

Focus groups

Interviews

On

line

focu

s g

rou

ps

© Fraunhofer IESE

10

Involving the Crowd in InnovationInvolving the Crowd in Innovation

EvaluateUSAGE

UnderstandPEOPLE

DesignSOLUTIONS

AnalyzeUSER FEEDBACK

.

What RE is Essentially

About

InvolveSTAKEHOLDERS

GatherUSE DATA

.

.

© Fraunhofer IESE

11

Why Ask a Crowd for Their Opinion before Realizing a “Good” Idea?Why Ask a Crowd for Their Opinion before Realizing a “Good” Idea?

Any product available online (potentially) has a “crowd”

Knowing how to approach customers (i.e., manage the crowd) means companies are able to channel and unleash the power of the crowd and possibly gain market advantage

Sometimes requires dealing with hundreds to millions of people!

Can reduce “blind spots” when taking management decisions (from visionary down to the implementation level)

The crowd holds great potential to bring the knowledge and resources a company needs

“Thecrowd isagroupofcurrentorpotentialstakeholders,largeenoughinsizetodisplaygroupbehavior,withacommoninterestinaparticularservice.”

FraunhoferIESE

© Fraunhofer IESE

12

What the Crowd Produces –A new GenerationWhat the Crowd Produces –A new Generation

The internet is a predominantly text-based medium

Emails, social media posts, app store reviews, bug tracker entries, etc.

Social media in 2015

2.1 billion unique users (27% of the world’s population, 68% of the active internet users)

Per minute: 2.5 million Facebook posts, 300,000 tweets, 220,000 Instagram photos, 72 hours of YouTube video content

Reviews

“Only” about 1 in 1,000 users write an app review

But: about 50,000 downloads from the Apple AppStore per minute

Around 10,000 reviews on Amazon per month, just for electronics

Reviews are on average nearly 600 characters long

[Sources: Jeff Bullas, Minimaxir, ACI, BBC, Cisco]

© Fraunhofer IESE

15

What Motivates the Crowd?What Motivates the Crowd?

Social incentives

Recognition: winning a competition, prestige

Altruistic motives: moral obligation, social obligation

Monetary incentives

Bounty, cash reward, contest draw

Entertainment

Gamification elements, exclusive access (e.g., previews)

Personal gains

Improving one‘s own world (e.g., a better working service)

Distorting the public view (e.g., sabotage)

© Fraunhofer IESE

17

Some Common ProblemsSome Common Problems

Selection bias

Different preferences towards providing feedback or being involved

Some are very concerned about their privacy, others contribute (classic crowdsourcing) or even pay money (crowdfunding)

Over- / underrepresentation of stakeholder (sub)groups

100% of the users‘ true needs will never be obtained no “One Truth”

Sabotage

Intentional bias by (unknown) motivations

Loss of nuance

Often forced in a template/wizard, excludes exceptions

Tacit information, e.g.: “This new feature is awful.”

© Fraunhofer IESE

20

Despite the Problems: there is so much potential benefit! Despite the Problems: there is so much potential benefit! We need to achieve systematic user participation

Feedback analysis: observing individuals

Context awareness: observing communities

For this, there is much power and potential in a crowd

Contagious mass behavior, reciprocal relationships

Benefits

Large (representative) sample size statistical analyses

Involve the uninvolved (based on trust and expectation)

Discussions drafting, prioritization, validation, reconciliation

Identify specific members who stand out selection

Identify groups attend to minorities

© Fraunhofer IESE

21

Crowd-based RECrowd-based RE

Analyzes any kind of feedback in a remote setting

Interaction (crowdsourcing, community management)

Text mining verbalized conscious needs

Usage mining unconscious needs

Text & usage mining subconscious needs

Use presently available data (passive crowd) and stimulate data generation (active / activated crowd)

Focus on a specific analysis purposes

“Crowd‐BasedRequirementsEngineeringisasemi‐automatedrequirementsengineeringapproachforobtainingandanalyzinganykindofuserfeedback fromacrowd,withthegoalofderivingvalidateduserrequirements.”

FraunhoferIESE

© Fraunhofer IESE

22

Traditional RE vs. Crowd-based RETraditional RE vs. Crowd-based RETraditional RE Crowd-based RE

Eli

cita

tio

n

Mode Manual, co-present(analyst(s) & stakeholder(s) in same place & time)

Semi-automated, remote(analyst(s) & stakeholder(s) can be in different place & time)

Techniques Co-present: interviews, work-shops, focus groups, etc.

Log file & feedback analysis, prototyping, video conferencing, etc.

Results Verbalized requirements Derived online statements & patterns

Documentation Manual and/or computer-assisted processing

Semi-automated algorithm-baseddistillation, derivation, drafting, and compilation (ranking, prioritization, correlation, clustering)

Validation & Negotiation

Co-present Crowdsourced, remote

Management Iterative within one project Iterative, repeated at leisure

Performer of RE Requirements Engineer System, Requirements Engineer

Ideal sample size

Depends on budget/complexity

Unlimited (the more stakeholders, the higher the validity)

Total duration Typically several months,in various stages

Recurring analysis of static data,continuous crowd management

© Fraunhofer IESE

23

Crowd-based RE is about Processing Feedback!Crowd-based RE is about Processing Feedback!

Traditional RE asks for requirements in a co-present setting

Crowdsourcing in RE asks for requirements in a remote setting

Crowd-based RE analyzes any kind of feedback in a remote setting

Crowdsourcing

Text mining

Data mining (usage mining)

These approaches can be performed in parallel and (iteratively) in series

© Fraunhofer IESE

24

Crowd-based RE Aspect: CrowdsourcingCrowd-based RE Aspect: Crowdsourcing

Crowdsourcing Crowd-Based RE

Description a form of RE outsourcing semi-automated approach to analyze user feedback from a crowd

Role of the crowd problem solvers informants

Stakeholder involvement

active mostly passive

Contribution provided upon request, always directly

provided continuously, often indirectly

Processing of data performed manually performed semi-automatically

Crowd-based RE can integrate crowdsourcing, but not vice versa

© Fraunhofer IESE

25

Integrating different Information for REIntegrating different Information for RE

Track and Observe User Behavior

Identify Associated Behavioral Pattern

Aggregated Results

Quantitative Feedback

Qualitative Feedback

Event Logging Usage Mining

Motivate Users to Provide Feedback

Discover Problems, Needs, and Ideas

Motivational Instruments Text Mining

Derived Requirements

© Fraunhofer IESE

26

Crowd-based RE Aspect: Text MiningCrowd-based RE Aspect: Text Mining

Collect and analyze both actively generated (verbalized) opinions

Manually processing feedback is tedious

Especially when trying to draw parallels, identify trends and topics

NL analysis of text-based user feedback using language queries (specifications of linguistic expressions)

Identify statements from stakeholders

Positive strengths, key selling points, popular features

Negative problems, unfulfilled expectations

Feature requests demands, needs

Structuring the statements

Prioritization, clustering, consistencies, conflicts, outliers

May reveal pressing issues, market trends, innovative ideas, etc.

The outcomes need to be validated before we can speak of requirements

© Fraunhofer IESE

27

Crowd-based RE Aspect: Usage MiningCrowd-based RE Aspect: Usage Mining

Can uncover needs of which users are not (consciously) aware or fail to verbalize

Behavioral patterns (e.g., shortcuts, preferential workflows)

Bottlenecks (frequently encountered problems of users)

Deviant activities (which may spur innovative ideas)

Detect issues that require addressing, new uses for the product (including new markets), and opportunities for optimization

Used in combination with prototyping (cf. A/B testing), it can help:

discover new requirements (even redesigning user interactions)

validate the results from text analyses

reconcile identified conflicts

© Fraunhofer IESE

28

iRequireiRequire

Users are triggered to provide requirements, are guided through the documentation process

Incentive: improving (changing) one’s own world

© Fraunhofer IESE

30

PRO-OPT Project OverviewPRO-OPT Project Overview

Challenges in Collaborative Analytics of Diagnostic Data

Workshops

Integrated Data

Product quality analysis andimprovement

Specific Reports for different use cases

Production / Engineering

Predictive and preventivemaintenance

Improvement of the diagnostics system

3rd Party

Supply‐Chain

Usage Mining Text Mining

Aspects Involving CrowdRE

© Fraunhofer IESE

32

IN ZUSAMMENARBEIT MIT

GEFÖRDERT VOM

KONTAKT

Crowdsourcing Logistik vom Land fürs Land

LOGISTIK

Pakete werden nicht mehr nur über den Paketdienst transportiert, sondern erhalten eine Mitfahrgelegenheit – sogar in privaten Fahrzeugen.

Dr.-Ing. Mario TrappThemenverantwortlicher Smart Rural Areas Hauptabteilungsleiter Embedded SystemsFraunhofer-Institut für Experimentelles Software Engineering IESEmario.trapp(at)iese.fraunhofer.deTelefon +49 (631) 6800-2272www.iese.fraunhofer.de

HANDELDer stationäre Einzelhandel profitiert von gemeinsamen Logistiklösungen und einem komfortablen Einkaufserlebnis seiner Kunden, die gerne regionale Produkte kaufen.

MOBILITÄTSoftware vernetzt Mobilitäts- und Logistiksysteme, dadurch entstehen Dienste aus unterschiedlichen Bereichen und schaffen für alle einen großen Mehrwert.

DIE VISION

© Fraunhofer IESE

34

SummarySummary

Crowd-based Requirements Engineering

helps scale RE to much larger groups of stakeholders by employing a set of usage & text mining tools, as well as techniques to get the crowd to participate systematically

makes optimal use of the large amounts of available knowledge to prevent or uncover blind spots and over- or underrepresentation of stakeholder groups

extends the portfolio of existing RE elicitation techniques

already plays an important role in several projects of Fraunhofer IESE in various branches (emergency response, automotive, public sector, smart ecosystems)

© Fraunhofer IESE

35

References – Crowdsourcing / Crowd-REReferences – Crowdsourcing / Crowd-REOrigins of Crowdsourcing

Howe, J. (2006). The rise of crowdsourcing. Wired, 14(6).

Malone, T. W., Laubacher, R. & Dellarocas, C. (2009). Harnessing crowds: Mapping the genome of collective intelligence. MIT Sloan Research Paper No. 4732-09.

Principles of crowdsourcing

Brabham, D. C. (2008). Crowdsourcing as a model for problem solving: An introduction and cases. The International Journal of Research into New Media Technologies, 14(1), 75–90.

Surowiecki, J. (2004). The wisdom of crowds. New York: Anchor.

Lanier, J. (2010). You are not a gadget: A manifesto. New York: Alfred A. Knopf.

RE and Crowdsourcing

Sutcliffe A. & Sawyer, P. (2013). Requirements elicitation: Towards the unknown unknowns. Rio de Janeiro, Brasil: RE 2013, Research Track.