Sr. Director, Product Management

February 21, 2013

#lspe: Dynamic ScalingShock Absorbers and APIs

Steve Shah

Disclaimer

• I’m going to talk about a product.Iot’s kind of necessary in order to make this talk useful.

ᵒBut a lot of you have this product or know someone that does!

ᵒThe product is pretty cool…

Iot can also sing and dance.

ᵒMaking coffee is on the roadmap.

• Sorry. ᵒYes, I am marketing scum.

ᵒNo, I will not to do a hard sell.

• My CompetitionᵒGoogle it. No really… It’s not hard to find them.

ᵒTheir product has various approaches too. I encourage you to ask them.

Performan

ce Offload SecurityAvailability

What is NetScaler?

NetScaler powers some of the world’s largest infrastructures.

1998 to 2012: From Load Balancing to Virtual

Networking

1998

L4 SLB

1999

L7 SLBGSLBMUX

2002

SSLCMPDNS

2005

AppFWSIPAAA-TM

2003

SSLVPNRHI

2006

ICAIPv6

2008

XMLnCore

2009

VPXEdgeSight

2011

SDXAppFlowDataStream

Secret Decoder Ring:SLB = Server Load BalancingGSLB = Global Server Load BalancingMUX = HTTP MultiplexingSSL = SSL AccelerationCMP = HTTP CompressionDNS = DNS Load Balancing / Proxy

RHI = Route Health InjectionICA = App Proxy for ICAIPv6 = IPv6 Routing, Switching, LBXML = XML Security, RoutingVPX = Virtual NetScalernCore = multi-core scalingSDX = Multi-tenant NetScaler

Agenda

• Things That Impact Scalability

• Shock Absorbers

• Out Scaling

• Your ADC has an API!

Things That Impact ScalabilityTouching on a bit of theory…

Load is Not Linear

• There are startup costs for enabling features in an ADC (memory and CPU)

• However, each incremental request takes a small fraction of resources

• As load increases, some global functions can take resources as wellᵒE.g., flushing unused IP fragments, running timers, management overhead, etc.

Data Structures and Big O

• I/O, Data structures, and String processing are big factors

• The two that get you are data structures and stringᵒACLs, VLANs, connection table, connection state, persistence table, etc.

ᵒHTTP request processing and policy execution

• Know your Big O – understand their impactᵒBig O notation is how programmers describe efficiency of algorithms

ᵒE.g., O(n) vs. O(log n) vs. O(1)

Shock AbsorbersCoping with Load

Launching v8: The Role of Data Structures

• Story time… launching a major service and what we learned

• Major new roll-out – expected to double the number of servers to handle

• Early testing revealed that large numbers of slow connections are meh

• Invest in your data structures! Clean up on several core structures

• Average connection lookup time driven to near constant time: O(1)

• Stir in a team that dreams in assembly language and can see cache

misalignment by glancing at code and shave another 20% off connection

lookup times (absolute times)

• Lesson: drive your apps to good data structures. Drive your vendors to do

better.

MaxConns and SurgeQ

Typical server performance curve

Peak perf – we want to

stay there

Incoming load

MaxConns and SurgeQ

Server stays operating at maximum throughput

Set max conns here

Queue incoming requests

in the ADC

Story time:

When 4 Hurricanes Hit

Out-Scaling

The SR-71 Approach: Go Faster

• Single Systemcoonfigured and managed as a

single logical system

• Scalablesocales with number of devices

(distributes work)

• Fault TolerantᵒHandles device failure, addition…

• Dynamic

Treat a collection of NS devices like a grand unified “big” device

The Sheet-metal Test

Steps:

• Take a cluster of NS, and an L2 switch.

• Configure the devices to your liking.

• Wrap the whole thing with sheet-metal, such

that only the network ports remain exposed.

Test:

Must be able to configure and use this contraption as

if it were just another NS box.

• connect wires into any visible port(s), create

LAGs at will, enable L2 mode, MBF …

• point GUI to Cluster’s IP and configure away

Clustering

• Create a single system image out of a collection of instancesIonstances = virtual machines, physical instances, or instances on multi-tenant boxes

• True shared management + data plane (the sheet metal test)

• Shared state for key data structures (persistence, health check, etc.)

• Linear scale by adding instances (up to 32)

• Ability to manage faults with proportional degradation

Real-timeAnalytics

Bandwidth

Connections

Top ‘N Requests

Response Time

Frequency

Policy Based Traffic Selection

Policy BasedActions

DecisionFeedback loop

Compress

Cache

Log

Drop

Respond

Scaling Globally

Global Server Load Balancing(GSLB)

Route Health Injection(RHI)

NetScaler uses DNS to send users to the closest site based on administrator defined metrics (geography, topology,

site performance, availability)

NetScaler dynamically updates routing tables to direct clients to the active site based on real-time health

monitoring of backend infrastructure.

Active

SiteMirror

Site

Your ADC Has an API!

API in a Nutshell: Your ADC Has This

API

Interfaces Client Toolkits Policy Statistics

SOAP RESTfulScripting

Perl/PHP/Python/PowerShell

OOPJava/C#/ASP/

.NET based

Reverse Call-Out

JSON/XML Bulk

ReportingGranular Reporting

More RESTful - HTTP Status Code

REQUEST RESPONSE

Citrix Confidential - Do Not Distribute

Success Case:GET http://<nsip>/nitro/v1/config/lbvserver/lbv1

Failure Case:POST http://<nsip>/nitro/v1/config/lbvserverContent-Type:application/vnd.com.citrix.netscaler.lbvserver+json

{"lbvserver":{"name":"lbv111", "servicetype":"HTTP"}

}

Success Case

HTTP 200 OK

Failure Case:

HTTP/1.0 409 Conflict

{"errorcode": 273, "message": "Resource already exists", "severity": "ERROR"

}

Indicate we want “rollback on failure” in this session

Prepare 3 lbvservers to be added in one bulk operation

Print results

Example: Using Java

Output

No attempt to add

“lb3” because of

Rollback behavior

AutoSense and AutoScale

CloudStack

Internet

NetScaler is auto-provisioned by CloudStack

M

M

M

M

NetScaler monitors servers for CPU, Memory, Latency, Throughput …NetScaler monitoring engine auto-detects abnormal behavior with servers

M

M

NetScaler triggers AutoScale capability in CloudStackCloudStack “auto-provisions” new server instances based on AutoScale policyOn successful AutoScale, CloudStack provides new service descriptionsNetScaler automatically adds new service resources and does bindingsTraffic is automatically scaled to the newly added services on NetScaler

M

M

Work better. Live better.