Presenter: David A. Watt Copyright Protected Material David A Watt, 2010

Presenter: David A. Watt

Copyright Protected MaterialDavid A Watt, 2010

DatabaseServer

LAN

PeripheralSystems

Client B

Client A

OtherComponents

ApplicationServer

Infrastructure Integration Testing:The Oft Ignored Testing Type

Copyright Protected MaterialDavid A Watt, 2010 2

Traditional Development MethodTraditional Development Method



Systems are Getting More ComplexSystems are Getting More Complex

From a Blog by Lori MacVittie on DevCentral

Load

Bal

ance

rs

Stor

age

Fabr

ic N

etw

ork



Enhanced Development MethodologiesEnhanced Development Methodologies

C

T

A

D

Spiral Model

V-ModelREQMTS

CODE

ACCEPTANCE

TEST

SYSTEM

TEST

INTEGRATION

TEST

UNIT TEST

DEVELOP

DESIGN

Development

Test



A B A B A B A BC C C

D DE

F

It 0 It 1 It 2 It 3 It 4Agile

(Agile Testing; Crispin, Gregory)

IT Systems for Mid to Large Sized CorporationsIT Systems for Mid to Large Sized Corporations

User PopulationUser Population

Training

EmplPortal

Procurement

Security MRP

End User

Support

BillingPoint

of Sale

HRFinanc

e



Common Infrastructure ElementsCommon Infrastructure ElementsDesktop / Laptop Configuration Variations

User Authentication and Digital Rights Management

Database Services

Web Services – Presentation, Delivery and Authentication

Load Balancing

Computer Hosting Platforms and Operating Systems (OS)

Storage Infrastructure

Network Infrastructure

Security Policies and Operations

People, Processes, and User Support

Operational Management and Oversight



Statement of the Problem:Statement of the Problem:Traditional system testing techniques teach and illustrate testing the system as a “complete” but stand-alone system.For some deliverable systems this is ok. But for IT solutions deployed into a medium to large scale corporate environment, this is naïve and insufficient.After all traditional system integration testing has been completed, another layer of testing is required.Testing in this layer verifies that the solution integrates with the rest of the IT infrastructure elements and is coined “Infrastructure Integration Testing”.

INFRASTRUCTURE

INTEGRATION

TEST

REQMTS

CODE

ACCEPTANCE

TEST

SYSTEM

TEST

INTEGRATION

TEST

UNIT TEST

DEVELOP

DESIGN

Development

Test



IT Infrastructure ElementsIT Infrastructure Elements

HVAC Facility Monitoring Operations PersonnelPower Floor Space Telecommunications

Gateways LAN / WAN VPN WirelessFirewalls Routers Switches IDS

DNS, DHCP Proxy Svrs VOIPExtranet Encryption Network Mgmt System QOS

Terminal Services / Citrix Virtual Machine Svcs SOA ArchitectureInternational Remote Access Systems

Windows Forest & AD LDAP Siteminder Svcs F5 Load BalancersFederated AD

Tiered Storage Solutions Message Store Disaster RecoveryStorage Management Systems

Disk Array Fabric Controllers On-line BackupStorage Area Net Network Attached Tape Silos

Operating Systems: Multi-vendor Resource Management File System: NFS, DFS, CIFS

Mainframe VM-Based Svr Open Sys PeripheralsMP Servers Intel Svrs Blade Svrs Backup Sys

Build Docs Help Desk, & TrainingStandard Operating Instrs Application & End User Performance

Configuration Mgmt Sys Patch / Upgrade Mgmt Sys Provisioning & Health MonSoftware Distribution Sys

Database Mgmt Systems Web Presentation ServicesWeb Delivery Services Development Svcs

E-Mail Svcs Document File & Share Incident MgmtBlackberry Office Communications Svcs Collaboration

Compliance OS SettingsSecurity Policy EPO PGP / PKI Enhanced Sec Vulnerability Mgmt

Business Reporting Svcs Department Applications Standard Products & ArchEnterprise Applications

Enterprise Applications Suites Data Access Rights MgmtApplication Authentication Application Security

VPN AccessDesktops Terminals Virus MgmtLaptops Mobile & Wireless Devices

PortalsOperating Systems Personal FirewallOffice Apps End User Apps Disability Support Tools

Encryption

FacilityInfrastr

ClientServices

OpsProcesses

Middleware

Infra Apps

Security

Authenti-cation Svcs

StorageInfrastr

ComputingInfrastr

NetworkInfrastr

ApplicationServices





Questions To Be AnsweredQuestions To Be AnsweredDo your solution’s parameters integrate with the parameters from other solutions? How does your solution impact performance of the rest of the environment?Does your solution comply with corporate security settings?Does your solution integrate with load balance, firewall, and gateway systems?Does your solution fit into production security certificate authority? Are there enough user certificates available? Is the version and configuration of your solution compatible with that of shared infrastructure? Are there adequate installation, operations, and support documents available?Has the help desk and operations support staff become acquainted with your solution?Does your solution require anything to be loaded on the client system? Is it compatible with all versions and configurations? Is your solution compatible with health and performance monitoring tools?Do the backup and disaster recovery requirements of your system integrate with the existing solutions?



Types of Testing to be PerformedTypes of Testing to be PerformedFunctional regression testing of your solution, infrastructure elements, and other solutionsCapacity, load, and performance testing of your solutionPerformance regression testing of the remaining infrastructure and other solutionsPerformance baseline testingFunctional testing of operational monitoring tools and processesRegression testing of your solution and others when changes are made to the infrastructure or other solutionsValidation testing of operational SOIs, training materials, help desk documentation, and other user support processesAd Hoc testing and use by support staffVulnerability scanning and remediationProblem isolation and remediationOn-going capacity modeling



The Challenge of “IIT” The Challenge of “IIT”

Infrastructure Integration Testing (IIT) may be a great idea but ……

Not having good answers to this question often leads to one of two deadly sins:

1. Perform Infrastructure Integration Testing in the production environment

OR

2. Test by putting a solution into production and letting the users find the

“bugs”

Q: How and where do you perform IIT?



Alternate ApproachAlternate Approach

Build an Enterprise Test Bed:

“An accumulation of separate and distinct test environments into an integrated and well managed test bed, which is isolated from the production environment and where functionality, interoperability, and performance testing can be performed on many solutions simultaneously”.



Network InfrastructureComputing Infrastructure

Enterprise Test BedEnterprise Test Bed

Storage InfrastructureAuthentication Services

Scale model of production solutionsPhysically and logically isolated from production environmentAccessible from anywhere on the corporate intranetReciprocal testing: Impact of

System Under Test against the rest of the test bedTest bed against System Under Test

Security & ComplianceInfrastructure Applications

MiddlewareOperational ProcessesApplication Services

Client Services

TestArticle

The Realities of an Enterprise Test BedThe Realities of an Enterprise Test Bed

Logistical

Political

Financial

Operations Management



Test Bed ManagementTest Bed ManagementExperience has shown that test beds must be managed using developed and enforced processes in order for the value of the investment to be maximized.

Consistency in following processes is the key to success.



Network InfrastructureComputing Infrastructure

Agile and the Enterprise Test Bed Agile and the Enterprise Test Bed

Storage InfrastructureAuthentication ServicesSecurity & Compliance

Infrastructure ApplicationsMiddleware

Operational ProcessesApplication Services

Client Services

Regression Tests – on both sprint code as well as infrastructure

Results of the sprint iterations are tested and pronounced “Ready for Production use” in the ETB and held there until a “bundle” of changes are ready for insertion into production use. New code is then implemented into the production environment and enabled for use.(ETB is used as staging area)

ConclusionsConclusionsTreating new solutions as “stand alone” systems needs to change

For IT solutions, “Infrastructure Integration Testing” needs to be performed

IIT verifies that the new solution integrates successfully the rest of the computing environment

IIT testing involves such testing as compatibility, functional, performance, and process validation.

Existing test environments can be consolidated into a centrally managed Enterprise Test Bed.

Test Bed Management techniques must be employed.



QuestionsQuestions



Documents

Presenter: David A. Watt Copyright Protected Material David A Watt, 2010