Ivano Malavolta Gran Sasso Science Institute Stefano Ruberto Gran Sasso Science Institute

Tommaso Soru University of Leipzig

Valerio Terragni Hong Kong University of Science and Technology

End Users’ Perception of Hybrid Mobile Apps in the Google Play Store

New York, 28th June 2015

FRAGMENTATION à a native mobile app is written from scratch for each platform

Mobile platforms today

Objective -C

code

Swift code

XCode

Java code

C++ code

Eclipse

C# code

C++ code

Visual Studio

JS code

A possible solution to mobile platforms fragmentation

Recurrent architecture:

–  apps are developed using standard web technologies –  on top of a hybrid development framework

•  providing a native wrapper and a generic JavaScript API that bridges all the service requests to the corresponding platform API

Web-based hybrid mobile apps

Single code base

Pros •  cross-platform portability •  reuse of existing knowledge of

web developers •  simpler and less expensive

development processes

Pros and cons

Cons •  restricted access to hardware

features •  decrease in performance •  variations on user experience

As of today, limited empirical investigations have been performed on hybrid mobile apps

Strong debate about benefits and drawbacks

Research goal

What is the difference between hybrid and native mobile apps as perceived by end users?

Perceived value

Perceived performance

Perceived bugginess

Initial download overhead

Developer End users

creates download

& use

App

Previous work[1]

FOCUS OF THIS

PAPER

RQ1

RQ2

RQ3

RQ4

We analysed hybrid mobile apps

•  in their actual context of use

•  with a reproducible empirical strategy –  well-defined empirical protocol –  dataset comprising 11,917 real apps

and 3,041,315 user reviews* –  dedicated analysis process and tool**

Design of the study

* complete replication package: http://cs.gssi.infn.it/ms_2015 ** analysis tool from [1]: http://github.com/GabMar/ApkCategoryChecker

Data extraction Classified apps

(hybrid vs native)

Hybrid apps classifier*

Reviews analyzer

top-500 most popular free apps for each category of the Google Play Store

~11k app binaries

50 pages (~255) of reviews for each app

~3M userreviews

apps scores

Apps and reviews mining

perceived value: 0.5users sentiment: 0.6#reviews: 243performance: 0.6bugginess: 0.1size: 3,456 kb

* analysis tool from [1]: http://github.com/GabMar/ApkCategoryChecker

Reviews analysis Stopwords

removal

manually performed by 2 domain experts

Single review

Single reviewscore

polaritypos: 0.8 performancepos: 0.6polarityneg: 0.1 performanceneg: 0.05bugginess: 0.2

300 random reviews

Keywords extraction

Relevant keywords

Lemmatization

Tf-idf based vectors similarity computation

Results

Data-intensive mobile apps[2]

Apps with closer interaction with the Android platform

Winners, in line with informal claims[3,4,5]

Results – value (RQ1) Average of the ratings as provided by end users

3.35 3.75

Rating = real number in [1, 5] Certain balance, with neglectable differences

Results – value (RQ1) Polarity of sentiment of end users

where posa = #reviews with positive sentiment nega = #reviews with negative sentiment

Balance between hybrid and native apps, with some exceptions

Non data-intensive or requiring multimedia capabilities

Results – value (RQ1) Average review count

whre Ra ∈ ℕ

Native apps have been reviewed in average 6.5 times more than hybrid mobile apps

Possible interpretation: hybrid mobile apps are neither perceived as too satisfying nor dissatisfying w.r.t. native ones [6]

Results – performance (RQ2)

where posa = #reviews with positive sentiment w.r.t. performance of the app nega = #reviews with negative sentiment w.r.t. performance of the app

Balance between hybrid and native apps, with some exceptions

Results –bugginess (RQ3)

where buga = #reviews signalling the presence of bugs or failures reviewsa = total number of reviews of the app

The highest unbalance between the two development strategies in our study

bugginessa = buga / reviewsa

Possible interpretation: absence of full-fledged testing frameworks for hybrid apps, such as those provided by native apps IDEs like Eclipse and Android Studio

Results – initial download size (RQ4)

6,586 kb 4,625 kb

In line with the average size of Android apps [7]

sizea = file size in kilobytes of the app APK file

A possible solution to mobile platforms fragmentation

Recurrent architecture:

–  apps are developed using standard web technologies

–  on top of a hybrid development framework

•  providing a native wrapper and a generic JavaScript API that bridges all the

service requests to the corresponding platform API

Hybrid mobile apps

Single code base

Conclusions

Data extraction Classified apps

(hybrid vs native)

Hybrid apps

classifiers

Reviews analyzer

top-500 most popular free apps for

each category of the Google Play Store

~11k app binaries

50 pages (~255) of

reviews for each app

~3M app

reviews

apps scores

Apps and reviews mining

perceived value: 0.5

users sentiment: 0.6

#reviews: 243

performance: 0.6

bugginess: 0.1

size: 3,456 kb

End users value hybrid and native apps similarly Hybrid may be good for data-intensive apps, whereas it performs poorly when dealing with low-level, platform-specific features In some categories, native apps are perceived as better with respect to performance and bugginess

Reviews analysis

Stopwords

removal

manually performed

by 2 domain experts

Single review

Single reviewscore

polaritypos: 0.8

performancepos: 0

.6

polarityneg: 0.1

performanceneg: 0

.05

bugginess: 0.2

300 random

reviews

Keywords

extraction

Relevant keywords

Lemmatization

Tf-idf based

vectors similarity

computation

References

[1] Ivano Malavolta, Stefano Ruberto, Valerio Terragni, Tommaso Soru, Hybrid Mobile Apps in the Google Play Store: an Exploratory Investigation. International Conference on Mobile Software Engineering and Systems (MOBILESoft), ACM, 2015.

[2] Mirco Franzago, Henry Muccini, and Ivano Malavolta. Towards a collaborative framework for the design and development of data-intensive mobile applications. International Conference on Mobile Software Engineering and Systems (MOBILESoft), pages, 58-61, ACM, 2014.

[3] Emiliano Masi, Giovanni Cantone, Manuel Mastrofini, Giuseppe Calavaro, and Paolo Subiaco. Mobile apps development: A framework for technology decision making. In Mobile Computing, Applications, and Services, pages 64–79. Springer, 2013.

[4] Julian Ohrt and Volker Turau. Cross-platform development tools for smartphone applications. Computer, (9):72–79, 2012.

[5] Luis Corral, Alberto Sillitti, and Giancarlo Succi. Mobile multiplatform development: An experiment for performance analysis. Procedia Computer Science, 10:736–743, 2012.

[6] Nan Hu, Jie Zhang, and Paul A Pavlou. Overcoming the j-shaped distribution of product reviews. Communications of the ACM, 52(10):144–147, 2009.

[7] Aapo Markkanen. Findings from Mobile Application File-size Research, 2012. ABI Research market report. Code: IN- 1014787.

Contact Ivano Malavolta |

Post-doc researcher Gran Sasso Science Institute

iivanoo

ivano.malavolta@gssi.infn.it

www.ivanomalavolta.com