Benchmarking with NGINX

Introduced by Owen Garrett

Presented by Rick Nelson

Nginx, Inc.

About this webinar

When you want to know how many resources to allocate for your NGINX

servers or what the capacity of your current NGINX servers is, you need

to be able to perform proper benchmark testing, but this can be

complicated. In this webinar, you'll learn the considerations that need to

go into planning, configuring and running benchmarks.

Agenda

• Introduction

• Common Pitfalls

• Tips and Techniques

• Demonstration

• Questions

INTRODUCTION

What is NGINX?

Internet

N

Web ServerServe content from disk

Application ServerFastCGI, uWSGI, Passenger…

ProxyCaching, Load Balancing… HTTP traffic

Application Acceleration

SSL and SPDY termination

Performance Monitoring

High Availability

Advanced Features: Bandwidth Management

Content-based Routing

Request Manipulation

Response Rewriting

Authentication

Video Delivery

Mail Proxy

GeoLocation

Why do Benchmarking?

• Stress test

• Capacity planning

• Comparison testing (bake off)

Benchmarking Considerations

• It’s complicated

• What is the goal?

• What kind of test environment do you have?

• What testing tools do you have?

• How well can you simulate production traffic?

What areas are you testing?

• Web server

• Application Server

• Reverse Proxy

• All of the above

What are you testing?

• Can you simulate production traffic?• If not, what are you concerned about:

– Connections– Request rate– Bandwidth– SSL– All of the Above

What are you testing?

• Can you do a full production test?

• If not, do a smaller scale test and extrapolate

• Know your traffic

– Get vs. Post, request/response sizes, etc.

• What you need to test vs. what you are actually testing

What are you testing?

• You may want to test a single variable

– Connections

– Request rate

– Bandwidth

– New SSL handshakes

Test Environment

• You are always testing the whole environment

– Testing Tools

• Load generators

• Web Servers

– Systems under Test (SUT)

• Reverse proxies

• Web servers

Test Environment

• Good rule of thumb for cores needed:

– Load Generators: 2N

– Reverse Proxies: N

– Web Servers: 2N

COMMON PITFALLS

Not Knowing What You Are Testing

• Know your testing tools

• What question does a test answer?

– For example:

• How many requests per second can a SUT handle?

• Can a SUT handle a certain number of requests per second

Real Clients versus Synthetic Clients

Real Clients Synthetic Clients

Latency Low-High Low

Packet Loss Low-High Low

Bandwidth Low-High High

Time between requests Long Short

Idle connections Yes No

Unrealistic Synthetic Clients

• Misleading results– System looks good during benchmark– System has problems with real clients

• Why is this? – Synthetic clients are ideal for the server

• Low latency, low packet loss, busy connections

– Real clients are not• High latency, packet loss, idle connections

Synthetic Clients

• Unrealistic synthetic clients– Apache Bench (ab)

• More realistic synthetic clients– Open Source

• Siege, wrk

– Commercial• Spirent, Ixia, Cloudtest, other cloud services

Misconfiguration

• Many configuration settings can impact tests– Some Linux kernel settings may be too low for heavy loads

– Keepalives

– SSL key sizes make a big difference

– Compression

– Benchmark clients or servers

Tips and Techniques

• Use multiple approaches– Real world simulations for real world results

– Simple tests for baselines and debugging

Tips and Techniques

• If you have found the real limit of a SUT then:

– At least one system resource should be exhausted

• CPU

• Memory

• Bandwidth

• Disk I/O

– If not, then a bottleneck exists elsewhere

Tips and Techniques

• Start with the NGINX defaults

• NGINX directives can impact performance

– accept_mutex, worker_processes, worker_connections, keepalive_timeout, lingering_close, sendfile, keepalive, aio, open_file_cache

Tips and Techniques

• Some 10G drivers don’t use cores effectively

– Some cores at 100%, others have low usage

– Solution: driver dependent

• Scripting

• New driver version

• New Card

Tips and Techniques

• Monitor error files and errors from testing tools

– Errors can return faster

– You may be hitting system limits or errors

• Don’t have load generation on a SUT

• Double check the result figures

Tips and Techniques

• Run all tests multiple times

• If using virtualization, be aware of other loads on the host

• If you don’t understand the results - simplify

Demonstration

Load Generator

Load Generator

Apache NGINX

ab siege

Datacenter

Questions and Answers

Closing thoughts

• 38% of the busiest websites use NGINX

• Check out the previous webinar on tuning at nginx.com

• Future webinars: nginx.com/webinars

Try NGINX F/OSS (nginx.org) or NGINX Plus (nginx.com)