© Copyright 2008, SoftWell Performance AB 1 Performance Testing Distributed Systems Concepts and Terminology v0.6.1

© Copyright 2008, SoftWell Performance AB

1

Performance Testing Distributed Systems

Concepts and Terminologyv0.6.1


2

Performance Testing - Concepts and Terminology

The document structure

The document describes performance testing in two parts:

General performance concepts

Performance testing concepts


3


The document structure

MTS – Performance Testing Distributed Systems1 – Concepts and Terminology

Generalperformance

concepts 7. Abstract Performance metrics

6. Measurement data objectives and attributes

5. Measured objects

4. Performance characteristics

8. Performance data processing

9. General performance test concepts

10. Performance test environment

11. Performance test specification concepts

12. Workload concepts

Performancetesting

concepts


4

Part 1General performance concepts


5


Performance characteristics


4. Performance characteristics

Categorizing performance characteristics

Powerfulness characteristics

Reliability characteristics

Efficiency characteristics


6


Measured objects


5. Measured objects

Measured services

Measured components

Service concepts

Service characteristics

Service interfaces


7


Measurement data objectives and attributes


6. Measurement data objectives and attributes

Performance metric objectives

Measurement data attribute sets

Identification attributes

Unit attributes

Conditional attributes

Processing attributes

Metric types


8


Abstract performance metrics


7. Abstract Performance metrics

Abstract powerfulness metrics

Abstract reliability metrics

Abstract efficiency metrics


9


Performance data processing


8. Performance data processing

Steps in performance data processing

Collection and storage of raw performance data

Condensation and normalization of raw performance data

Performance data computations

Evaluation of performance data

Presentation of performance data


10

Part 2 Performance testing concepts


11


General performance test concepts


9. General performance test concepts

Performance test concepts

Performance test phases

Performance test objectives

Performance objectives and performance requirements

Performance measurement conditions

Performance targets

Performance measurement standards

Some performance measurement characteristics


12


Performance test environment


10. Performance test environment

Test environment concepts

System Under Test concepts

Test System concepts


13


Performance test specifications


11. Performance test specifications

Elements of performance test specifications

Test objectives

Test conditions

Test configurations

Test data specifications

Test evaluation specifications


14


Workload concepts


12. Workload concepts

Workload set or Traffic set

Workload content

Workload volume

Load concepts

Workload time distribution


15

What is next?


16

Performance Testing – Next step

Proposal

The next document should describe

• Methods in all phases of performance tests

• How methods are applied

The purpose is to

• Describe methodology of performance testing in a generalized manner

• Describe how methods are applied •

• Start the groundwork on a framework for performance testing based on a common view of abstract performance metrics and characteristics

MTS – Performance Testing Distributed Systems2 – Methodology and Realization


17

MTS objectivesfor

Performance Testing


18

MTS objectives for Performance Testing

The MTS objectives should be to:

• Develop a formalized view of performance • Develop a formalized view of performance test methods • Develop a formalized view of performance tests

That can be used as:

• A framework for applied performance tests and test suites• A guideline for design of performance tests tools• A guideline for performance test standards


19

Some long term goalson

Performance andPerformance Testing


20

Long term goals for Performance Testing

Performance is an increasingly important aspect of a design or a system.Performance will be an increasingly important aspect of a standards. A validated function works under most circumstances in production, validated performance is not a bulletproof guarantee for production.

A formalized view of performance and performance testing enables:

• A complete specification of a system • Modeling and design of a system’s performance• Simple monitoring of performance in design and production• Far better ways for predicting performance in production• Intelligent performance test tools that can be built into the system• A continuously improved understanding of performance• A safer investment in software development

Some examples


21

“To measure is to know”

- Lord Kelvin

Documents

© Copyright 2008, SoftWell Performance AB 1 Performance Testing Distributed Systems Concepts and Terminology v0.6.1