1 © NOKIA Nokia Research Center / 18.05.04 Performance Data Collection: Hybrid Approach Edu Metz, Raimondas Lencevicius Software Performance Architecture

1 © NOKIA Nokia Research Center / 18.05.04

Performance Data Collection: Hybrid Performance Data Collection: Hybrid ApproachApproach

Edu Metz, Raimondas Lencevicius

Software Performance Architecture Group

Nokia Research Center

5 Wayside Road, Burlington, MA 01803, USA

E-mail: [email protected], [email protected]


MotivationMotivation• Performance profiling of complex embedded software is important

• Profiling = monitoring software execution + obtained data analysis

• 2 ways to collect data• Event tracing • Sampling

• Both have drawbacks and advantages

• What to use?


PositionPosition

Data collection should be performed using hybrid approach that combines completeness of event tracing

with low cost of sampling


OverviewOverview

• Event tracing

• Sampling

• Hybrid data collection with example

• Why it will fail

• Why it will succeed


Event TracingEvent Tracing• Record events occurring during system execution

• Component entry and exit• Function calls• Execution states• Message communication• Resource usage

• Complete record of information• If event was recorded, it did occur; if it was not recorded, it did not occur

• Allows complete analysis of event relationships in time

• Event tracing comes at a cost• Trace instrumentation takes time• Changes behavior of software system• Could violate real-time constraints and timing requirements in real-time

systems• Disproportional overhead for different software components• Large data volume


SamplingSampling• Intermittent recording of software system’s state

• Program counter• Call stack• Scheduled and blocked tasks

• Can be done strictly periodically or with certain randomness

• Overhead may be orders of magnitude less than event tracing

• Has its own downsides• Granularity of information determined by sampling frequency• Needs “reasonable” data collection interval to provide accurate

data• Loss of information completeness• May complicate application-level information extraction


Hybrid ApproachHybrid Approach

• Often a way to minimize drawbacks of two techniques is to use them both!

• Trace a subset of events• Sample the rest

• Provide low overhead with adequate information completeness

• But… how not to inherit drawbacks of two techniques?• Worst case: high overhead and not enough information for

causality determination• Intelligent selection of what should be traced and what should be

sampled is needed


Hybrid Approach Example:Hybrid Approach Example: Profiling with Tasks Profiling with Tasks

• System contains a number of communicating tasks

• Performance engineer needs to understand what parts of software consume most of the processor time and how this influences task messaging and scheduling

• Using only event tracing would lead to large overhead

• Using sampling would not provide complete information about task messaging

• Hybrid approach solves the problem• Low overhead due to sampling of inside task

activities• Complete information of task messaging

through event tracing - low overhead due to infrequent tracing

Task A Task A

Task B

t (s)

m1 m2

t4t3t2t1


Hybrid Approach HeuristicsHybrid Approach Heuristics

• What should be traced?• Infrequent events

• Task switches• Peripheral activations and deactivations

• Events that have causality relationships• Message sends and receives• Interrupts (maybe)

• Events for which relative time to other event is necessary• Video frame display

• Events for which the absolute time is necessary• Events connected with real world: alarm clock

• What should be sampled?• Frequent events

• Function calls (maybe)• Bytecode/CPU op execution

• Events for which causality is not important• Heap size (maybe)


Related WorkRelated Work

• We do not know any hybrid profilers

• A lot of sampling profilers

• Intel Vtune – either sample or trace

• Hollingsworth et al. – collect all and then sample the collection buffers periodically


Why it will FailWhy it will Fail

• Intelligent selection of what should be traced and what should be sampled is difficult

• Bad tracing/sampling selection may lead to large overhead and missing information

• Trace instrumentation is still required with all its drawbacks

• Yields two sets of profiling data that need to be reconciled


Why it will SucceedWhy it will Succeed

• Solves the issue of obtaining complete causality information with low overhead

• Allows collecting infrequent events even in short program runs

• Limits profiling data volume


Questions ?Questions ?

Documents

1 © NOKIA Nokia Research Center / 18.05.04 Performance Data Collection: Hybrid Approach Edu Metz, Raimondas Lencevicius Software Performance Architecture