Spark the future.

May 4 – 8, 2015Chicago, IL

Testing and Monitoring CDNs for Performance Anton KucerGrahan Cooley

BRK2703

Fundamentals

CDN is a necessity

Core CDN Elements

Customer Origin Server

CDN Edge Network

HTTP Client

Core CDN Traffic Flow

Customer Origin Server

CDN Edge Network

HTTP Client

First Request

Subsequent Requests

Values of using a CDN

Speed Reliability Efficiency

• In almost every case, content utilizing a CDN will be much closer to the end-user and that will result in faster delivery and a better user experience

• The CDN spreads out content, replicating that content across the globe, resulting in reduced demand on any given server

• The CDN optimizes for delivery, allowing the customer to focus on the quality of their content, not on the delivery of that content

CDN Performance Generally

Major Themes

Technical Performance Parity

Generic Performance often

has little value

Latency, Latency, Latency

Latency, Latency Latency

People HATE to wait

100msThe limit for having the user feel that a system is reacting instantaneously.

1,000msThe limit for the user's flow of thought to stay uninterrupted.

10,000msThe limit for keeping the user's attention focused on the dialogue

The Ping command

Does not replicate a

representative CDN scenario

Does not transmit TCP-based data

Uses ICMP, which is for

diagnosis, not testing

Routers may give ICMP

messages lower priority

Routers may be configured to block ICMP messages entirely

Substantive issue

TCP Connect

DNS Lookup

Content Download

Time to first byte

Determine the geography of both the origin and intended audience

Have a good understanding about how latency was tested

Determine the timing of the latency testing Have a deep understanding of what the

product is, what it is attempting to accomplish and how it is leveraging a CDN

How to deal with latency concerns

Additional Metrics

Throughput

Round-Trip-Time (RTT)

Cache/Hit Ratio

the rate of successful message delivery over a channelthe time required for a signal packet to travel from a specific source to a specific destination and back againthe ratio of the number of cache hits to the number of misses

Azure CDN Performance

Azure CDN performance

Performance Evidence

General approach to performance testing

1Set your metrics and establish your baseline performance

2Use publicly available reports like those available from a RUM testing provider like Cedexis to determine what major CDN provider performance should look like, all things being equal

3Run an actual test of how those CDNs perform in your situation and compare the numbers, either through your own testing or through a custom testing provider like Gomez or Catchpoint

4If your results (1) show improvement over your baseline and (2) are in line with the performances provided by generic testing, your scenario is probably close enough to the generic testing to depend on it for choosing your provider

5

If your result (1) show improvement over your baseline and (2) are NOT in line with the generic testing, you will want to drill into the details and determine what the variance is. This is usually one of the following or (a combination): file type, file size, time of day or geography. Then you will want to run tests specifically to see which provider performs best in the conditions you identify.

6

If your results do NOT show improvement over your baseline, you might be in a rare situation where a CDN is not going to help your performance and you may want to consider some alternate ways to address your goals than using the CDN

Third-party Monitoring: RUM v Synthetic

RUM stands for “Real User Measurements”. It depends not on using “virtual” end users or “synthetic" agents but on measurements taken from actual website and mobile application users, using real PCs, laptops and mobiles, to capture a real user’s experience.

Synthetic monitoring utilizes a web browser emulation or scripted

recordings of web transactions to test

performance. Custom paths can be designed to test specific behaviors then

repeated for a set number or duration to gather a

meaningful dataset.

vs

Self-measurement

Identify ContentUpload to

Origin

Download via non-CDN URL

Download via CDN URL

Download via CDN URL

Point to Origin

Monitoring Tools

Dive deep: data over time

AnalyticsCore Analytics: • # of hits• Bandwidth• Data transferred• Cache/hit ratio

Advanced Analytics:• Real Time reports• Custom real-time alerts• Top country/city• Top files• Top user agents• Error report analytics

Core Analytics: # of hits

Core Analytics: Bandwidth

Core Analytics: Data Transferred

Core Analytics: Cache Hit Ratio

Core Analytics: Cache Hit Ratio

Advanced Analytics: Real Time Analytics

Advanced Analytics: Real Time Alerts

Advanced Analytics: Top Countries

Advanced Analytics: Top 404 Errors

Visit Myignite at http://myignite.microsoft.com or download and use the Ignite Mobile App with the QR code above.

Please evaluate this sessionYour feedback is important to us!

© 2015 Microsoft Corporation. All rights reserved.