The design of adaptive acquisition of users feedback an empirical study (rcis'14)

www.bournemouth.ac.uk

The Design of Adaptive Acquisition of Users

Feedback: an Empirical Study

Malik Al Maliki, Cornelius Ncube and Raian Ali

Faculty of Science and TechnologyBournemouth University, UK

RCIS, Marrakesh, Morocco, 28-30 May, 2014

malmaliki@bournemouth.ac.uk

www.bournemouth.ac.uk 2

Introduction

malmaliki@bournemouth.ac.uk REFSQ, Essen, Germany, 7-10 April, 2014

RCIS, Marrakesh, Morocco, 28-30 May, 2014

Self-Adaptive Software

“Self-adaptive software evaluates its own behavior and changes behavior when the evaluation indicates that it is not accomplishing what the software is intended to do, or when better functionality or performance is possible.”

(Laddaga 1997)

• Why do we need it?• Adapt to internal and external changes.• Maximize operation in different conditions.• Reduce the cost and time of manual adaptation.• Keep the satisfaction of requirements.

malmaliki@bournemouth.ac.uk RCIS, Marrakesh, Morocco, 28-30 May, 2014

Towards Social Adaptation

• Self-adaptivity is highly dependent on feedback that is provided by the system itself.• Feedback is on current state and its environment.

• System feedback is the driver for triggering and taking autonomous adaptation actions.

• There is no emphasis on the role of users in the adaptation process.

Social Adaptation

“the system autonomous ability to analyse users’ feedback and choose upon an alternative behaviour which is collectively shown to be the best for meeting

requirements in a context” (Ali et al. 2012)

• Users’ judgments on the quality and validity of the different behaviours of a system drive adaptation

• Socially-driven adaptation • Giving users’ a voice in tailoring adaptation actions• Maximize users’ satisfactions.

Social Adaptation Loop (Ali et al. 2012)

Problem

Feedback Quality and Users’ Behavior

• The quality of collected feedback is highly affected by users’ behaviour.

• Then, how do users behave to feedback acquisition? What do they like? What type of

feedback? which method? How is their behaviour socially affected? etc.

Aim of the Study

Aim of the study

• The aim of this study was to :

Empirically understand users’ different perspectives and behavioural aspects to feedback acquisition for socially-adaptive

software.

Study Design

Study design

• Mixed Method approach (sequential-exploratory). • First phase (qualitative):

•Interviews. •7 participants•Served as a foundation for the second phase.

• Second phase (quantitative):•Questionnaires. •100 participants (BU and overseas participants).•Good response rate (100 out of 180).•The survey script contained 31 questions discussing and investigating the results of the first phase.

•Improved the quality and generalizability of the first phase results.

• First phase participants:

• Second phase participants:

Participants’ Characteristics

Participants Age Gender Education Level Home Country

P1P2P3P4P5P6P7

19292419232826

MaleMaleFemaleFemaleMaleMaleFemale

UndergraduatePostgraduatePostgraduateUndergraduateUndergraduateUndergraduatePostgraduate

UKNigeriaNigeriaKSAUKUSAKSA

Age Range Gender

18-25 26-34 35-54 55-64 Total Male Female Total

level of education

High school 3 0 0 0 3 3 0 3Bachelor’s degree 9 3 6 0 18 13 5 18Master’s degree 6 36 10 3 55 30 25 55

Professional degree 0 0 1 1 2 1 1 2

Doctorate degree 3 11 5 0 19 10 9 19Others 0 3 0 0 3 2 1 3Total 21 53 22 4 100 59 41 100

Qualitative Findings(Phase 1)

Overview of interviews analysis

Qualitative Findings

Theme1: Explicit Feedback Advantages[1.1] Evident channel for delivering users’ voice and raising developers' awareness[1.2] Better for ethical reasons

Theme3: Feedback Acquisition Methods[3.1] Email is preferable: o More personalized o More preferable for qualitative feedback o More time space and less interruption o Reasonable number of feedback requests[3.2] Passive feedback forms are preferable[3.3] Quantitative feedback request is preferable[3.4] Combination of qualitative and quantitative (not only quantitative)

Theme2: Motivation for Accepting/Ignoring FeedbackRequests[2.1] Visibility of feedback effect on the system[2.2] Usability and simplicity: o Language used[2.3] Disagreement of existing feedback [2.4] Reasonable number of feedback requests[2.5] The exciting nature of feedback subject [2.6] Positive experience[2.7] Negative experience and needs for improvement[2.8] Less interruption and distraction[2.9] Device used[2.10] Raising public awareness[2.11] Being forced by the software: o Low quality feedback

Theme4: Pause of Feedback Requests[4.1] Same feedback is given[4.2] Lack of interest [4.3] Passive feedback is preferable

Theme5: Timing for Feedback Requests[5.1] Enough time before requesting feedback[5.2] On recent service or product[5.3] Reminder is needed[5.4] Avoid work time or hours

Theme6: Feedback Visibility[6.1] Ability to see what others said[6.2] The trend of current feedback

Quantitative Findings(phase2)

Overview of the Questionnaire

Introduction to the survey

A snapshot of the questionnaire

Quantitative Findings

• Typical users of popular software applications• The majority of the participants represent a typical set of software

users. • Their feedback reflects their experience with popularly used software

applications.

Applications used by users

• Do users like to be asked for feedback by software applications?• Not really!, this reflects• the need for novel mechanisms to increase users’ engagement as

evaluators of software applications.

Users’ likeness of feedback request

• Feedback Acquisition Methods and Feedback Types• Feedback Types

•Explicit feedback• Qualitative feedback • Quantitative feedback • A combination of qualitative and quantitative •Implicit feedback ( 19%)

Explicit Feedback Types

• Feedback Acquisition Methods•Passive feedback acquisition (51%)•Offline feedback acquisition (33%)•Online feedback acquisition (54%)•Using Hints or tips (31%)

Passive Method

Offline Method

Online Method

Hint/tip Method

20.00%

40.00%

60.00%

Acquisition Methods: Users' Pref-erences

• Motivations for Accepting/Ignoring Feedback Requests• Users’ Experience

•Simplicity of feedback requests (64%)•Timing for feedback requests (75%)•Awareness of the usage and impact of the feedback on the

system (54% )•Privacy (31%)•Familiarity (42%)

plicity

0%30%60%

Users’ Motivations: Users’ Experience factors

• Interface Design•Language used (52%)•Graphical design (31%)•Simplicity and complexity (74%)•Fitness of the design and content (81%)•Information provided (24%)

• Social Factors•Visibility and similarity of others feedback (47%)•Volume of already given feedback (52%)•Social recognition (57%)•Feedback acquisition as a social activity (63%)

• Volume and Frequency of Feedback Requests

Effect of High Feedback Requests Volume on UsersPercenta

ge of Users

It is fine with me, I like to give feedback often 3%It is fine with me as long as I am not forced to give answers 13%I tend to respond to some of them 14%I tend to give less focused or less truthful feedback 10%It leads me to give a negative feedback as the requests make me feel annoyed 7%

I tend to ignore all of them and I tend to consider it as a spam 53%

I tend to stop using the software sending me these requests 21%

Users Clusters

• Initial clusters of users’ behaviour to feedback acquisition:

Clusters N Likeness to be asked

Method

Explicit/

Implicit

Reminder

Visibility-Willingne

ss increases

Social Activit

y-interes

Social recognitio

n-willingnes

s increases-

impact

Feedback

Volume

Feedback

Similarity

Cluster 1 (feedback antagonists)

38 No Online No No No No No No

Cluster 2 (passive and stingy

people)

27 No Passive No No No No No No

Cluster 3 (privacy fanatic and

generous people)

21 Yes Offline Very Explicit

Yes Yes_ If able to see others

feedback first

No Yes Few-increase

Cluster 4 (privacy tolerant

and socially ostentatious

people)

14 No Hint+ Online

Implicit is also

No Yes_ If able to see others

feedback first

Yes Yes Large-increase

Similar-increase

Conceptual Framework for Adaptive Feedback

Acquisition

Conceptual Framework for Adaptive Feedback Acquisition

Conceptual Framework for an adaptive acquisition of users’ feedback.

Conclusion and future work

• An empirical mixed method study to investigate users’ behaviour to feedback acquisition in software applications was reported.

• A conceptual framework for adaptive feedback acquisition was proposed.

• Users’ were studied first qualitatively and then quantitatively to enhance our results and allow for more generalization.

• Users’ behaviour with regard to feedback acquisition highly varies and is influenced by a number of behavioural factors.

• Systematic approaches and novel mechanisms to conduct an adaptive feedback acquisition are needed.

• Availability of such systematic approaches can greatly improve the quality of users’ feedback, users’ satisfaction and the quality of socially-adaptive software.

Acknowledgement

• We would like to thank:• Participants who took part in our study for their valuable input.• Dr. Hamid Bouchachia and Dr. Emilio Balaguer for insights on conducting clusters

analysis.• The anonymous reviewers for their valuable feedback.

• The research is supported by: • Bournemouth University through the Software System Research Centre (SSRC) fund. • Saudi Ministry of Higher Education through the 4th scholarship program.• FP7 Marie Curie Grant (the SOCIAD project) • Bournemouth University Fusion Investment Fund (BBB, VolaComp and BUUU projects)

Feedback?

The design of adaptive acquisition of users feedback an empirical study (rcis'14)

Software

Global Travel Time Data Set From Adaptive Empirical ... · Global Travel Time Data Set From Adaptive Empirical Wavelet Construction Hongyu Lai1, Edward J. Garnero1, Stephen P. Grand2,

Zurich to acquire Rural Community Insurance Services (RCIS

An Empirical Investigation of the Stratified Adaptive Computerized

Intel Corporation RCIS: поддержка инноваций и предпринимательства

Activity Theory as a Framework for Building Adaptive E-learning SystemsA Case to Provide Empirical Evidence

RCIS Nr 23 Din 2008-Violenta Fam-Del Juv

Intel Corporation RCIS: поддержка инноваций и предпринимательства Евгений Закаблуковский parus@intel.com

RCIS 2016 conference paper: Variable Interactions in Risk Factors for Dementia

Adaptive Adaptive Adaptive Indexing

th Challenges in Information Science - IEEE RCIS RCIS 2014 Program... · Challenges in Information Science ... Christina Manteli, ... Sundaram, Engaging a Class of 2200 Digital Natives:

Adaptive Analysis of Diastolic Murmurs for Coronary Artery ......Chapter 4 Adaptive Analysis of Diastolic Murmurs for Coronary Artery Disease Based on Empirical Mode Decomposition

Empirical Evaluation of HTTP Adaptive Streaming under

Adaptive Traffic and Dynamical Networks: From Plants to People Methods: agents (i.e. decision-making particles) + networks (i.e. connectivity) empirical

A Review on Bidimensional Empirical Mode Decomposition€¦ · Abstract: The Bidimensional Empirical Mode Decomposition (BEMD) is an adaptive decomposition technique for the decomposition

Adaptive Behavior of Impatient Customers in Tele-Queues: …ie.technion.ac.il/serveng/References/zohar.pdf · Adaptive Behavior of Impatient Customers in Tele-Queues: Theory and Empirical

POD@RCIS · 2019. 3. 7. · POD@RCIS is published by the foundation 'Podarcis', dedicated to the publication and distribution of herpetological information (Chamber of Commerce Rotterdam

RCIS 2013 DelegateGuide

Rcis Dumitru Stan

Congenital Cardiology Today€¦ · Blossom” Technique By Carolyn Wilhelm, MD; Jason Swinning, RCIS; Matt Sisk, RCIS; Ralf Holzer, MD, MSc, FSCAI ~Page 1 The 25th Annual Conference

Farmacovigilancia. La Gestión de RCIS a través de las herramientas del UMC