Videos vs. use cases Can videos capture more requirements under time pressure? Olesia Brill, Kurt Schneider, and Eric Knauss

Videos vs. use casesCan videos capture more requirements under time

pressure?

Olesia Brill, Kurt Schneider, and Eric Knauss

2

Agenda

About the paperAbout the method PDDRelated literature Questions

09-04-2014Agenda, About the paper, About the method, PDD, Related literature, References, Questions

3

About the paper


Goal: to investigate the effectiveness and efficiency of creating ad-hoc videos under time pressure for validating early requirements compared to use cases.

Method: Goal-question-metric

Conclusion: videos are a better tool to use for the indentification of performance and basic requirements. For excitement requirements there is no difference between the performance of videos and use cases.

4

About the method


Goals

Questions

Metrics

Goals

Answeres

Values

Goal 1 Goal 2 Goal 3 Goal 4

Hypothesis 1 Hypothesis 2 Hypothesis 3

Metric 1 Metric 2 Metric 3 Metric 4 Metric 5

Created by Basili & Weis (1984)

5

PDD (1)


6

PDD (2)


7

PDD (3)


8

PDD (4)


9

PDD (5)

09-04-2014

Goals

Questions

Metrics

Goals

Answeres

Measurements

Agenda, About the paper, About the method, PDD, Related literature, References, Questions

10

Related literature


Solingen & Berkhout (1991)

Basili, Heidrich, Lindvall, Münch, Seanian, Regardie & Trendowicz (2009)

11

References

Basili, V., & Weiss, D. (1984). A methodology for collecting valid software engineering data. Software Engineering, IEEE Transactions on(6), 728-738.

Basili, V., Heidrich, J., Lindvall, M., Münch, J., Seanian, C., Regardie, M., & Trendowicz, A. (2009). Determining the Impact of Business Strategies Using Principles from Goal-oriented Measurement. Proceedings of Wirtschaftsinformatik, Wien, Austria, 545-554. Brill, O., Schneider, K., & Knauss, E. (2010). Videos vs. Use Cases: Can Videos Capture More Requirements under Time Pressure? In R. Wieringa & A. Persson (Eds.), Requirements Engineering: Foundation for Software Quality (pp. 30-44) Essen: Springer.

Solingen, R. van, Berghout, E. (1999). The Goal/Question/Metric Method – A Practical Guide for Quality Improvement of Software Development. Maidenhead, UK: McGraw-Hill.


12

Questions?


13

Example


14

RQGoals

Research question: What’s the difference in effectiveness and efficiency of paper prototyping compared to wireframes.

Goals: 1. Analyze the efficiency and effectiveness of wireframes2. Analyze the efficiency and effectiveness of paper

prototyping3. Compare wireframes and paper prototyping with

respect to their efficiency and effectiveness4. Analyze the subjective preference of wireframes and

paper prototypes


15

Facet classification


Goal Purpose Concerning aspect Of object In context From perspective

1 Analyze Efficiency and effectiveness Wireframes

Selecting the best technique to use

Usability expert

2 Analyze Efficiency and effectiveness Paper prototyping Usability expert

3 Compare Efficiency and effectiveness

Wireframes and paper prototyping Usability expert

4 Analyze Preferences Wireframes and paper prototyping Usability expert

16

GoalsHypotheses

Goals: 1. Analyze the efficiency and effectiveness of wireframes2. Analyze the efficiency and effectiveness of paper prototyping3. Compare wireframes and paper prototyping with respect to their

efficiency and effectiveness4. Analyze the subjective preference of wireframes and paper prototypes

Hypotheses: 5. For the efficiency aspect of goal 1,2 and 3: usability experts will be able to

complete a wireframe faster than a paper prototype 6. For the effectiveness aspect of goal 1,2 and 3: usability expert will yield

more feedback points when using wireframes instead of paper prototyping

7. To answer goal 4: usability experts will have a preference for making wireframes instead of paper prototypes


17

Hypotheses Metrics

Hypotheses: 1. For the efficiency aspect of goal 1,2 and 3: usability experts will be able

to complete a wireframe faster than a paper prototype 2. For the effectiveness aspect of goal 1,2 and 3: usability expert will yield

more feedback points when using wireframes instead of paper prototyping

3. To answer goal 4: usability experts will have a preference for making wireframes instead of paper prototypes

Metrics: 4. Time to complete wireframe5. Time to complete paper prototype6. Number of feedbackpoints when using a wire frame7. Number of feedbackpoint when using a paper prototype8. Subjective evaluation based on a 5-point likert scale


18

Activity table

09-04-2014Agenda, About the paper, About the method, PDD, References, Questions

Activity table

Activity Sub activity Description RoleGoal Identification

Identify goals From the formulated RESEARCH QUESTION, multiple GOALs can be identified. These GOALs will therefore all support the same cause; finding an answer to this RESEARCH QUESTION.

Researchers

Include facets For each GOAL multiple FACETs must be specified. These FACETs make sure the GOALs are focused and helps avoid ambiguities. See Table 1 for an example of this FACET classification.

Researchers

Hypothesis identification

Operationalize goals The GOALs should be operationalized, in order for them to transform in HYPOTHESES. This operationalization is necessary to avoid HYPOTHESES that are deemed too abstract.

Researchers

Identify hypotheses Based on the (operationalized) GOALs, HYPOTHESES can be identified. This can be done by a pre-study, other research and/or assumptions. For each GOAL, at least one HYPOTHESIS is identified in order to reach that GOAL.

Researchers

Add estimates For each HYPOTHESIS at least one ESTIMATE is added. Including ESTIMATEs makes sure that the researchers, after collecting data, do not think "I knew this before"(Brill et al., 2010).

Researchers

Metric identification

Identify metrics Based on the explicit HYPOTHESIS, METRICs can be defined. Each HYPOTHESIS should have one or several METRICs. These METRICs are the measurable units of the HYPOTHESIS and therefore make sure an answer to a HYPOTHESIS is yielded when measuring a METRIC.

Researchers

Form experimental setup Based on the METRIC LIST, which is a list consisting of all identified METRICs in the previous sub-activity, the EXPERIMENTAL SETUP can be designed. This is done by combining all METRICS in certain stages/phases of an experiment.

Researchers

19

Concept table


Concept tableConcept DescriptionRESEARCH QUESTION A RESEARCH QUESTION is a clear, focused, concise, complex and arguable question around which someone

can center its research (The Wrinting Center, 2012). In the GQM method as used in the paper of Brill et al. (2010) this RESEARCH QUESTION is already present before starting the method.

GOAL GOALs in the GQM may be defined for any object, for a variety of reasons, with respect to various models of quality, from various points of view, relative to a particular environment (Basili, Caldiera, & Rombach, 1994). To makes this clear FACETs are added to the GOALs.

FACET FACETS are template parameters that in GQM include purpose (what object and why), perspective (what aspect and who) and the environmental characteristics (where) (V. Basili, 1993).

HYPOTHESIS A HYPOTHESIS is a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation. In the GQM method the HYPOTHESIS is extracted from one or more GOALs, has one or more ESTIMATEs and METRICs (Oxford Dictonaries, 2014).

ESTIMATE An ESTIMATE gives a HYPOTHESIS a quantitative idea of the effect expected by the researcher (Brill et al., 2010). In the GQM method ESTIMATES are used as references to the expectations of the researchers.

METRIC METRICs are measures needed to collect answers to specific HYPOTHESIS. METRICs can be divided between OBJECTIVE METRICs or an SUBJECTIVE METRICs (V. Basili, 1993).

OBJECTIVE METRIC An OBJECTIVE METRIC is an absolute measure taken on the product or process. For example: time for development, number of lines of code etc.(V. Basili, 1993).

SUBJECTIVE METRIC A SUBJECTIVE METRIC is an estimate of extent or degree in the application of some technique or a classification or qualification of problem or experience. They are used in situations where there is no exact measurement, usually on a relative scale. For example: the experience of programmers, the experience of fun when using a program (V. Basili, 1993).

METRIC LIST The word METRIC LIST is a composition of two words. The first word METRIC is already defined. A LIST is a number of connected items or names written or printed consecutively, typically one below the other (Oxford Dictionaries, 2014). Together these two words result in METRIC LIST, which is a LIST that includes every identified METRIC.

EXPERIMENTAL SETUP The EXPERIMENTAL SETUP in the GQM method used in the paper of Brill et al. (2010) is based on the METRIC LIST. The EXPERIMENTAL SETUP is designed in a way that all METRICs in the METRIC LIST are measured.

Documents

Videos vs. use cases Can videos capture more requirements under time pressure? Olesia Brill, Kurt Schneider, and Eric Knauss