CLOUD TESTING MODEL – BENEFITS, LIMITATIONS AND CHALLENGES

7/21/2019 CLOUD TESTING MODEL – BENEFITS, LIMITATIONS AND CHALLENGES

http://slidepdf.com/reader/full/cloud-testing-model-benefits-limitations-and-challenges 1/5

ISSN: 2312-7694Mukesh et al, / International Journal of Computer and Communication System Engineering (IJCCSE), Vol. 2 (5), 2015, 636-640

636 | P a g e

© IJCCSE All Rights Reserved Vol. 02 No.05 Oct 2015 www.ijccse.com

CLOUD TESTING MODEL –

BENEFITS, LIMITATIONS AND CHALLENGES

Dr. Mukesh Chandra Negi

Project Manager, TechMahindra Ltd.PhD, MCA, M.Sc., APGDCA, PGDCA, CIC

[email protected], [email protected]

Abstract — Software testing has always become a

challenging part in the field of IT, especially when related with

enterprise applications in comparison with standalone

applications, where lots of integrations and complexities are

involved including other related challenges, like security, costs,

infrastructure, scalability and growth in future. The role of

software testing is very significant in the area of enterprise

applications world. Criticality of testing is less but the role is

very high. In a typical On-Premise IT environment, you have aninternal infrastructure for testing in almost every organization,

which includes hardware’s, software’s, networking, security and

all related infrastructure and facility managements like all other

development, QA, prod etc. environments. Most of time this

testing environment sits idle and required only when you are

working on scalability of your applications or doing some

functional or nonfunctional testing’s. With time and for

scalability, it’s required to increase or upgrade the testing

infrastructure with time to time which incurs extra cost. Cloud

testing is an era in the field of testing where you can save your

cost as well perform different testing’s more efficiently and

dynamically according to your current requirements and in

future as well. Despite the benefits, there are certain limitations

and challenges also associated. My article will focus on different

limitations and challenges associated with the cloud testing

model.

I. INTRODUCTION

Traditionally, testing environment always seen as an evil

since most of the time it sits idle and in conjunction itrequired lots of infrastructure and resources. You need

everything, a space, non IT infra setup, electricity, backup,

cooling, servers, networking, security, IT manpower etc. for

your testing environment. Your business also grows with

time and in parallel you have to upgrade as well increase IT

resources of you testing environments which incur extra costtime to time. As your requirements start growing with time,

you have to spend more on testing’s and it’s infra to mak e it

sustain as per the current updates and requirements. Cloud

testing is a model where your complete testing environment

is outsourced to a third party service provider as On-Demand

model, and it eliminate your upfront capital expenditures ontesting infra setup. It gives you great benefits like lower your

costs, on demand flexibility, scalability, availability,

maintainability, freedom from infra management, pay-as-use

model etc. but in parallel, Cloud testing has limitations as

well, and it introduces lots of challenges as well like lack of

standards, data integrity and security, layer testing challengesetc.

If you see one of the reports published by Fujitsu in 2010,

almost more than 55% of applications and infrastructure over

public cloud was occupied by the Testing and Development

environments.

Source: Fujitsu 2010

II.

TaaS (Testing as a Service)

TaaS concept in testing is similar like other cloud models

SaaS; PaaS & IaaS where you can completely outsources you

testing IT infrastructure and resources and just use and pay

On-Demand basis whenever required. It’s a completely

outsourcing model where you get the Testing services fromsome third party Testing Service Provider. You just have to

give your exact infrastructure, applications and resources

requirements and time for how long you want the same, and




637 | P a g e


based on that you have to pay only for required resources and

timing only. It’s a great benefit in terms of cost and

scalability where you don’t need to setup and internal on

premises testing environment and not need to worry about

management and scalability of same in future.Below is the graphical representation of same. There is a

TaaS service provider at the end, which runs and provides

services to you 24*7*365 and you as an end user or client can

access it from the cloud web interface anytime from

anywhere.

III. BENEFIT OF CLOUD TESTING

Below are certain benefits of using cloud testing model

A. Scalability (Future Growth)

Like cloud other delivery models, one of the great benefit

of cloud based testing is scalability. You can easily up anddown your requirements anytime as per your requirements.

Especially useful where business demands keep alteredfrequently.

B. Less Cost

Cost is another major benefit of cloud based testing. You

don’t need to invest heavy on any on premises based

infrastructure and applications licensing. Just take the

services as per your requirements whenever you want and

pay for that only.

C. Live replica & Availability

This is another major benefit of cloud testing. In a generic on

premises testing environments, you don’t have an exactreplica of your production environments, which creates lots

of issues during test environment testing’s and live rolloutdue to this simulation problems. With cloud testing. you have

flexibility to choose and perform testing on exact same

environment and resources as you have on your live

production environment, which help in great way to catch

and fix lots of issues, which you can’t able to handle in on

premises less resources testing scenarios.

D.

Easy Customization

This is another great advantage. Testing involves lots of

permutation and combinations scenarios where you need totest your applications on different software’s, infrastructures,

browsers etc. which is very tough in traditional testing

environments due to cost and lots of different factors, but

with cloud testing option you can easily achieve this.

E.

Better Time Management

You time management will be better as you don’t need to

worry and waste your time on and testing related procurement, management, resourcing, scalability,

availability etc. and you can contribute that time and efforts

to your actual business.

F. Agile Development & Testing

During the agile development, you have different teams from

different locations are involved, and during testing,

irrespective of the team, project & location you need the

contribution of all. With cloud testing, you all can work

together from anywhere at same time.

G. Assured Performance and Peak Load Testing

Cloud testing can give you an actual picture of

performance resources your application actually needed at

peak time, which is not possible in your traditional on

premises IT infrastructure testing environment due to

limitations of resources. On cloud testing model, you have

option to identify peak load of your application by taking

resources from the big pool of resources.

IV. DIFFERENT FORMS OF CLOUD

TESTINGS




638 | P a g e


Generally a cloud testing can be categorized in two

different categories as below –

Functional Testing

Non Functional Testing

Functional testing

Functional testing is a verification of the business

requirement functionalities which you have developed as per

the client requirements, if it’s functioning as per the

expectations and giving you the desired results.

Broadly, you can further categorized it into three subcategories as below –

System Verification Testing – During this phase, you

have to make sure each and every module you have

developed are working and giving you the expected output.

System Integration or Interoperability Testing – During this testing you have to check integration testing’s of

modules you have verified during system verification testing.It’s actually a integration testing where complete end to end

flow tested.

User Acceptance Testing – During this phase you have

to carry out different functioning cases testing with the actual

end users to make sure all are working fine, and in case of

any gap identified you have to fix, deploy and test the again.

Non Functionality Testing

It’s all apart from functional testing’s. Here you don’t need to

check any user or client desired functionalities. You have to

concentrate on all rest parts to make sure your designed

system and solution is able to sustain the client requirements

as per the user expectations, governance, risk and

compliance. Broadly, you can divide it into below sub

testing’s –

Availability Testing – Here cloud vendor has to make

sure services are available all the time 24*7.

Scalability Testing – Here you have to make sure the

solution you have designed is capable to scale up and down

the system as per your need at any time in near future.

Disaster Recovery Testing – Here you have to test

failover environment of your live environment to make sureyour business with continue in case of any disaster at main

live location.

Security Testing – Here you have to make sure there are

no security gaps in your system and solution, and it’s

designed to work as proper governance, risks and

compliance.

Performance, Load and Stress Testing – Here youhave to test if you designed testing solution is actually able to

sustain the load and demand of client especially at peak loadtime.

Multi Tenancy Testing – Since you cloud applications if

accessed and used by different users from different locations,

departments and organizations, so you have to make sure

sufficient security, access, authorization and authentications

are in place.

Overall structure you can represent as below –

V.

CLOUD TESTING LIMITATIONSAND CHALLENGES

As I mentioned earlier, Despite lots of benefits, Cloud testing

has some challenges and limitations as well which need to be

studied further to make it an integrated solution which any

organization can adopt without any fear of risks & threats.

Below are few existing challenges and limitations as of now.

A. Lack of Standards




639 | P a g e


Lack of standards is still a big issue in the field of cloud

computing and testing since there is no universal standards

exist as of now. All public cloud providers have their own

process and mechanism in terms of infrastructure,

architecture, cost, integrations etc. and there is no universal

standard which governs the integration of your on premises

private cloud with the public cloud.

B. Security

Security is still another big challenge in the field of cloud

computing, especially in public cloud. You don’t have

control over your data as it’s completely resides in the service

provider network and premises. Data integrity, account

hackings, data loss insecurity, shared technology, insecure

API etc. are same common security factors in public cloud.

C. Dependency on internet

This is also one of the major challenge and dependency on

cloud computing and testing where to perform any kind of

work or testing on your environments you need a good speed

internet connection.

D. Understanding of the Cloud

This is another challenge before adoption of cloud testing.

You have to make sure you have enough resources who know

everything about cloud testing and able to perform work on

same or you have to provide good hands on training to your

staff before adoption and implementation of cloud testing

model.

E.

Selection of a Service Provider

Selection of a good public cloud service provider is also a big

challenge. There are uncountable no of public cloud service

providers in the market and to identify one out of them

according to your need and requirements is still a big

challenge.

F. Infrastructure Requirements

Some cloud service providers in the market provide limited

capabilities and functionalities in terms of servers,

configurations, applications, storage, networking, bandwidth

etc., which is also a challenge.

G. Testing Strategy & Usage

Preparing your strategy of testing over cloud testing and the

identification of actual demand of your testing requirements

is also a big challenge in cloud testing model.

H. Service Level Agreements with Service

Provider

Due to lack of standards, there still lack of good service level

agreements almost with all public cloud service provides.

You have to discuss with all providers in terms of their

service level agreements with respect to availability,

scalability etc. as per your testing needs to filter most suited

provider.

I. Performance issues in Public Cloud

Since public cloud is a multitenant model where all if the

resources are shared by different cloud service provider

clients, so performance demand in public cloud is also a big

challenge since sometime it happened that due to some glitch

in another client applications it may impact your applications

performance as well due to shared resources model.

J. Layer Testing

Since the complete control of infrastructure is completely

owned and controlled by the service provider, so layer testing

is still a big challenge in public cloud testing as during layer

testing’s, you need the details of different configuration and

parameters which you may further need to tune as per your

requirements related with networking, security, database etc,

but you don’t have any control over that.




640 | P a g e


VI. CONCLUSION

Cloud testing model emerged as a great innovation in the

field of IT testing, but since the complexities of the demand

and future is changing, there has to be some way to make

cloud testing process more agile and flexible. There may be

some standards are required to govern the cloud testing

model, as well to challenge the current limitations and

challenges. TaaS has to be accepted as one of the cloud

delivery models apart from SaaS, IaaS and PaaS, and all the

similar limitations and challenges has to be addressed and

fixed in parallel with all cloud delivery model.

REFERENCES

[1] A. Vanitha Katherine & Dr.K. Alagarsamy (2012),

“Conventional Software Testing Vs. Cloud Testing”,

International Journal of Scientific & Engineering Research,

Vol. 3, No. 9.

[2] AppLabs,“Testing the Cloud,”white paper,Internet:

www.applabs.com/html/Testing the Cloud 786 .html

[3] “Confidence In Cloud Grows, Paving Way For New

Levels Of Business Efficiency,” Fujitsu, 2010.

[4] Cloud Based Testing: Need of Testing in Cloud Platforms

by Vinit B. Mohata1, Dhananjay M.Dakhane2, Ravindra

L.Pardhi3 : International Journal of Application or Innovation

in Engineering & Management (IJAIEM): Volume 2, Issue 3,

March 2013

[5] Eljona Proko & Ilia Ninka (2012), “Analysis and Strategy

for the Performance Testing in Cloud Computing”, Global

Journal of Computer Science and Technology Cloud &

Distributed, Vol. 12, No. 10, Version 1.0, USA.

[6] Jerry Gao, Xiaoying Bai, and Wei-Tek Tsai, “Cloud

Testing- Issues, Challenges, Needs and Practice” An

International Journal (SEIJ), Vol. 1, No. 1, SEPTEMBER

2011.

[7] M. Oriol and F. Ullah, “YETI on the cloud,” in Third

International Conference on Software Testing, Verification,

and Validation Workshops, 2010, pp. 434 – 437.

[8[Nancy Gohring, “The Cloud: A Threat to Incumbents,

Opportunity for Startups,” IDG News, Sept. 28, 2010.

[9] Leah Muthoni Riungu, Ossi Taipale, Kari Smolander,

“Software Testing as an Online Service: Observations from

Practice,” In Third International Conference on Software

Testing, Verification, and Validation Workshops (ICSTW),418-423, 2010.

[10] Paul Ashwood, “Why Your IT Organization Should

Move from Traditional application Testing to Testing-as-a-

Service (TaaS)”, HP Enterprises.

[11] P. Joshi, H. S. Gunawi, and K. Sen, “PreFail: a

Programmable Tool for Multiple-Failure Injection,” in

Proceedings of the 2011 ACM international conference on

Object oriented programming systems languages and

applications, 2011, pp. 171 – 188.

[12] Software Testing in Cloud Platform: A Survey by

A.Vanitha, Katherine, K. Alagarsamy : International Journal

of Computer Applications (0975 – 8887)Volume 46 – No.6,

May 2012

[13] Thomas J. Bittman, “Test and Quality Management: The

First Frontier for Private Cloud,” Gartner, Inc., Sept. 3, 2010.

[14] Xiaoying Bai, Muyang Li, Bin Chen, Wei-Tek Tsai,

Jerry Gao. Cloud Testing Tools. Proceedings of The 6th

IEEE International Symposium on Service Oriented System

Engineering (SOSE 2011).

[15] Abbas, Waseem, Nasim Abbas, and Uzma Majeed.

"PERFORMANCE ENHANCEMENT OF END-TO-END

QUALITY OF SERVICE IN WCDMA WIRELESS

NETWORKS." Science International 26.2 (2014).

Documents

CLOUD TESTING MODEL – BENEFITS, LIMITATIONS AND CHALLENGES