25
1 IOPS Requirements for the NetWorker Server Technical Notes P/N 300-012-592 REV A02 June 2011 This technical note identifies the overall input/output operations per second (IOPS), and provides sizing guidelines for NetWorker server requirements based on specific test setups. The IOPS values were measured by using different loads on both Windows, and Linux NetWorker servers. The tests measured the impact of IOPS during IO intensive operations such as bootstrap backup, purge operations and during catalogue management tasks such as nsrim and nsrinfo. The EMC NetWorker Performance Optimization Planning Guide provides more details on IOPS requirements for the NetWorker server. This technical notes document contains information on these topics: Executive summary ................................................................................... 2 Test scenarios .............................................................................................. 3 Data analysis ............................................................................................... 4 General IOPS recommendations ............................................................ 16 Test setup .................................................................................................. 21 Test configuration .................................................................................... 22

IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

Embed Size (px)

Citation preview

Page 1: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

1

IOPS Requirements for the NetWorker Server

Technical Notes P/N 300-012-592

REV A02 June 2011

This technical note identifies the overall input/output operations per second (IOPS), and provides sizing guidelines for NetWorker server requirements based on specific test setups.

The IOPS values were measured by using different loads on both Windows, and Linux NetWorker servers. The tests measured the impact of IOPS during IO intensive operations such as bootstrap backup, purge operations and during catalogue management tasks such as nsrim and nsrinfo. The EMC NetWorker Performance Optimization Planning Guide provides more details on IOPS requirements for the NetWorker server.

This technical notes document contains information on these topics:

Executive summary ................................................................................... 2 Test scenarios .............................................................................................. 3 Data analysis ............................................................................................... 4 General IOPS recommendations ............................................................ 16 Test setup .................................................................................................. 21 Test configuration .................................................................................... 22

Page 2: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

2

Executive summary

IOPS Requirements for the NetWorker Server Technical Notes

Executive summary

The 4 major components for system performance are memory (speed and capacity), processing power, network bandwidth, system bus and disk (IOPs and capacity). Every system is always limited in one of these areas at any given time. Ensuring that the system is not hitting the current bottleneck is a very complex task. Performance tuning is simply the art moving the bottlenecks back to optimize performance.

Originally, memory was the primary bottleneck in the storage industry. Improvements in new systems are the basis for a shift in storage performance bottlenecks to CPU and IOPS on disk.

With respect to measuring overall performance on a backup application, IOPS is the most common metric.

This technical note provides introductory information to determine the calculations for required IOPS, and recommendations for stable NetWorker operations. This technical note also describes how individual storage components affect overall IOPS capabilities on the NetWorker server.

Page 3: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

3

Test scenarios

IOPS Requirements for the NetWorker Server Technical Notes

Test scenarios

This table lists the scenarios used in the IOPS testing.

Table 1. IOPS test scenarios Component Setup Testing factor Tests

IOPS calculation for the NetWorker server

NetWorker server - 1 NetWorker storage nodes and clients : 5

1 session 1 saveset of 100GB (single savegroup).

5 sessions 5 savesets each from one client (single group).

10 sessions 10 savesets with each client configured with 2 savesets (single group).

100 sessions 100 savesets with each client configured with 20 savesets (single group).

1000 sessions 1000 savesets with each client configured with 200 savesets (single group)

5000 sessions 5000 savesets with each client configured with 1000 savesets (single group).

NetWorker startup operation

Measure the IOPS during NetWorker startup.

NetWorker jobsDB purge operation

Measure the IOPS on NetWorker server during Jobs DB purge.

NetWorker maintenance tasks

Measure the IOPS on NetWorker server by running manually nsrim, mminfo and nsrinfo commands.

NetWorker bootstrap backup

Measure the IOPS on NetWorker server during bootstrap backup. Identify the IOPS impact during bootstrap backup.

Page 4: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

4

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

Data analysis

This section describes the results of the test scenarios.

IOPS measurements for the startup and duration of backups

This section describes the IOPS measured during the startup and duration of the backup operations.

The following graphs illustrate the IOPS 4500 save sessions that start at the same time. Windows uses less IOPS compared to Linux when starting N number of saves in parallel where N is at least 1000.

Figure 1. IOPS for the startup and duration of backup jobs

Analysis and recommendations

IO spikes are generated on the NetWorker server when a large number of jobs start at same time. The IOPS requirement increases exponentially and remains increased briefly when a large number of jobs start simultaneously. NetWorker requires 1 IOPS for each session that starts in parallel.

It is recommended not to start more than 40 clients per group by using default client parallelism. The practice of starting a large number of clients simultaneously can lead to I/O starvation.

Page 5: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

5

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

This table illustrates the IOPS on nsr with different RAID configurations.

Table 2. RAID configurations on nsr Windows_ num _sessions

RAID3_ max_IOPS_with_ bootstrap

RAID3_ max_IOPS_without_ bootstrap

RAID3_ average _IOPS

RAID5_ max_IOPS_with _bootstrap

RAID5_ max_IOPS_without_bootstrap

RAID5_ average _IOPS

1 161 91 5 541 119 6

5 220 220 9 136 136 10

10 169 114 10 183 183 9

100 556 556 39 510 419 45

1000 905 523 97 1084 734 95

4500 1593 523 108 3604 639 140

Page 6: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

6

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

Figure 2. Average iOPS on Windows during backup

Figure 3. Average IOPS on Windows

Page 7: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

7

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

Table 3. RAID configurations on Linux Windows_ num_ sessions

RAID3_max_IOPS_with_bootstrap

RAID3_max_IOPS _without _bootstrap

RAID3_average_IOPS

RAID5_ max_IOPS_with_ bootstrap

RAID5_max_IOPS _without _bootstrap

RAID5_average_IOPS

1 119 37 0.6 69 66 9

5 159 159 3 133 133 11

10 171 171 14 257 257 11

100 207 207 16 715 715 41

1000 1098 1098 98 1516 1516 117

4500 1276 1276 125 930 930 129

Page 8: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

8

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

Figure 4. Average IOPS on a Linux server during backup

Figure 5. Average IOPS on a Linux server

Page 9: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

9

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

Analysis and recommendations

The following describes the analysis for the IOPs testing:

Average IOPS for Windows and Linux during backup operation remains almost same. Depending on load, NetWorker server requires (maximum concurrent number of sessions reached on server / 3) IOPS. Maximum parallelism of NetWorker server is 512, so highest possible load is 170 IOPS in general.

The IOPS on RAID-5 LUN with same number of disks is high compared to RAID-3; this is due to implementation of RAID and penalty of RAID-5. It is recommended to have a greater number of disks (based on the IO load) for parallel IO in RAID groups which host nsr on the NetWorker server.

IOPS requirements increase linearly with an increasing NetWorker load (number of sessions).

The IOPS during a bootstrap backup is very high compared to a regular backup. Faster disks (storage array cache) speed up the bootstrap backup. Ensure average IOPS are met for bootstrap backup. The impact of bootstrap backup on NetWorker server IO operations is mentioned in below section of this document. The IOPS requirement is very high for bootstrap backup on Windows compared to Linux however bootstrap backup depends on speed and filesystem performance of local disk mounted on NetWorker server.

IOPS requirements for a large number of volume operations The following is displayed during a request/mount/save operation:

70896 3/23/2011 7:46:48 PM nsrd vanila:Q:\vanila-dataset-6\5MB-dataset-435 saving to pool 'Default' (cherry.026) 71659 3/23/2011 7:46:48 PM nsrd vanila:Q:\vanila-dataset-6\5MB-dataset-435 done saving to pool 'Default' (cherry.026) 5013 KB 42506 3/23/2011 7:46:55 PM nsrd write completion notice: Writing to volume cherry.016 completed 70896 3/23/2011 7:46:56 PM nsrd vanila:Q:\vanila-dataset-6\5MB-dataset-436 saving to pool 'Default' (cherry.018) 71659 3/23/2011 7:46:57 PM nsrd vanila:Q:\vanila-dataset-6\5MB-dataset-436 done saving to pool 'Default' (cherry.018) 10 MB 70896 3/23/2011 7:46:57 PM nsrd vanila:Q:\vanila-dataset-6\5MB-dataset-437 saving to pool 'Default' (cherry.018) ………. ……….. ………. 70896 3/23/2011 7:48:38 PM nsrd vanila:Q:\vanila-dataset-6\5MB-dataset-451 saving to pool 'Default' (cherry.026) 71659 3/23/2011 7:48:39 PM nsrd vanila:Q:\vanila-dataset-6\5MB-dataset-451 done saving to pool 'Default' (cherry.026) 5013 KB

Page 10: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

10

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

70896 3/23/2011 7:48:40 PM nsrd vanila:Q:\vanila-dataset-6\5MB-dataset-452 saving to pool 'Default' (cherry.026) 71659 3/23/2011 7:48:40 PM nsrd vanila:Q:\vanila-dataset-6\5MB-dataset-452 done saving to pool 'Default' (cherry.026) 5013 KB

Figure 6. IOPS during volume operations

Analysis and recommendations

Frequent, large IO spikes occur on the NetWorker server (on both Windows and Linux) during a large number of volume operations. When NetWorker completes a write operation to a volume there is a spike in IOPS on the NetWorker server for a brief time.

For environments running a small number of volumes the effect of IO spikes is minimal, but for environments with frequent mount requests, a significant load is added to the server. It is recommended to have a minimum of 100 IOPS for significant activity (greater than 50 mount requests per hour). To avoid the load, use a smaller number of larger volumes.

IOPS requirement during the NetWorker startup operation

The IOPS observed during NetWorker server startup which includes bringing all the NetWorker processes, automatic media and index consistency check, mounting all volumes (excluding purge) is shown in below graphs for Windows and Linux platform. The IOPS noticed during jobsdb purge is covered as separate in this document.

Windows: 9513 3/9/2011 11:58:42 AM nsrindexd Running nsrck to check client file indices0 3/9/2011 11:58:42 AM nsrindexd NetWorker0 3/9/2011 11:58:42 AM nsrindexd 7.6.2.Build.5970 3/9/2011 11:58:42 AM nsrindexd 5970 3/9/2011 11:58:42 AM nsrindexd Thu Feb 17 17:58:19 20110 3/9/2011 11:58:42 AM nsrindexd Build arch.: ntx640 3/9/2011 11:58:42 AM nsrindexd DBG=0,OPT=42506 3/9/2011 11:58:42 AM nsrd index notice: checking index for 'radds153'

Page 11: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

11

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

42506 3/9/2011 11:58:42 AM nsrd index notice: M:\Legato\nsr\index\radds153 contains 0 records occupying 0 KB

42506 3/9/2011 11:58:42 AM nsrd index notice: checking index for 'alums'

83310 3/9/2011 11:58:58 AM nsrd <<< NSR server host 'cherry' using Client id '95c0af3f-00000004-4d6b30de-4d6b30dd-00011800-7223438b' >>>

42506 3/9/2011 11:58:58 AM nsrd index notice: checking index for 'vanila'

39078 3/9/2011 11:59:28 AM nsrd RPC error: Failed to authenticate with server cherry during RAP bind operation: Timed out

42506 3/9/2011 11:59:31 AM nsrd index notice: M:\Legato\nsr\index\vanila contains 22334 records occupying 11 MB

42506 3/9/2011 11:59:31 AM nsrd index notice: checking index for 'badam'

42506 3/9/2011 11:59:31 AM nsrd index notice: M:\Legato\nsr\index\badam contains 72500 records occupying 95 MB

42506 3/9/2011 11:59:31 AM nsrd index notice: checking index for 'cherry'

42506 3/9/2011 11:59:31 AM nsrd index notice: M:\Legato\nsr\index\cherry contains 74955 records occupying 99 MB

42506 3/9/2011 11:59:31 AM nsrd index notice: checking index for 'radds186'

38752 3/9/2011 11:59:38 AM nsrd rd=radds186:/radds186_50GB_RD0_CX960/aftd_dev3/_AF_readonly Mount operation in progress

38752 3/9/2011 11:59:40 AM nsrd rd=radds186:/radds186_50GB_RD0_CX960/aftd_dev4/_AF_readonly Mount operation in progress

38752 3/9/2011 11:59:40 AM nsrd rd=radds153:/50GB_RD0_CX960/aftd_dev1/_AF_readonly Mount operation in progress

38752 3/9/2011 11:59:41 AM nsrd rd=vanila:P:\aftd_dev2\_AF_readonly Mount operation in progress

Linux: 42506 02/27/2011 09:12:12 PM nsrd index notice: checking index for 'radds186'

42506 02/27/2011 09:12:12 PM nsrd index notice: /RAID_3_5_Disks/nsr/index/radds186 contains 69127 records occupying 93 MB

42506 02/27/2011 09:12:12 PM nsrd index notice: checking index for 'badam'

Page 12: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

12

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

42506 02/27/2011 09:12:12 PM nsrd index notice: /RAID_3_5_Disks/nsr/index/badam contains 66720 records occupying 85 MB

42506 02/27/2011 09:12:12 PM nsrd index notice: checking index for 'alums'

42506 02/27/2011 09:12:12 PM nsrd index notice: /RAID_3_5_Disks/nsr/index/alums contains 68604 records occupying 85 MB

42506 02/27/2011 09:12:12 PM nsrd index notice: checking index for 'vanila'

42506 02/27/2011 09:12:12 PM nsrd index notice: /RAID_3_5_Disks/nsr/index/vanila contains 27202 records occupying 16 MB

42506 02/27/2011 09:12:12 PM nsrd index notice: Completed checking 7 client(s)

83310 02/27/2011 09:12:31 PM nsrd <<< NSR server host 'raddd230' using Client id '41d471fa-00000004-4d641d03-4d641d02-00011800-5c23438b' >>>

33526 02/27/2011 09:12:45 PM nsrmmd Start nsrmmd #13, with PID 29649, at HOST raddd230

33526 02/27/2011 09:12:49 PM nsrmmd Start nsrmmd #17, with PID 29727, at HOST raddd230

38752 02/27/2011 09:12:52 PM nsrd /space/bootstrap_index2/_AF_readonly Mount operation in progress

38752 02/27/2011 09:12:52 PM nsrd rd=radds153:/nwdevice3/aftd4/_AF_readonly Mount operation in progress

38752 02/27/2011 09:12:58 PM nsrd rd=radds153:/nwdevice2/aftd4/_AF_readonly Mount operation in progress

38752 02/27/2011 09:13:01 PM nsrd rd=alums:/data-set/_AF_readonly Mount operation in progress

38752 02/27/2011 09:13:05 PM nsrd rd=radds153:/nwdevice2/aftd3/_AF_readonly Mount operation in progress

33526 02/27/2011 09:13:08 PM nsrmmd Start nsrmmd #36, with PID 30127, at HOST raddd230

38752 02/27/2011 09:13:13 PM nsrd rd=radds153:/nwdevice2/aftd2/_AF_readonly Mount operation in progress

38752 02/27/2011 09:13:13 PM nsrd rd=radds153:/nwdevice2/aftd1/_AF_readonly Mount operation in progress

33526 02/27/2011 09:13:14 PM nsrmmd Start nsrmmd #42, with PID 30251, at HOST raddd230

38752 02/27/2011 09:13:16 PM nsrd rd=radds153:/nwdevice5/aftd3/_AF_readonly Mount operation in progress

38752 02/27/2011 09:13:17 PM nsrd /space/bootstrap_index/_AF_readonly Mount operation in progress

Page 13: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

13

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

Figure 7. IOPS during startup

Analysis and recommendations

Volume mount and index check operations are the highest consumers of IO. There is a sharp spike during NetWorker startup when the index check and volume mount operations start. Windows has a greater number of index records than Linux. Therefore, Windows uses more IOPS than Linux during NetWorker startup.

It is recommended to use a minimum of 30 – 50 IOPS during startup. The more storage layer IOPS provided, the faster the startup operation. Do not start any NetWorker operations until the NetWorker index and media checks, volumes mounts, and purge operations complete.

IOPS requirements during the NetWorker jobsdb purge operation.

The jobsdb purge operation is one of the highest IO and CPU intensive operations in NetWorker. With NetWorker 7.6.2, the purge algorithm runs a light purge once an hour, rather than every 12 hours. If the NetWorker server is busy, purge operations are deferred for 30 minutes. If the server is busy for 9 consecutive times, the full purge is enforced.

Linux 39074 02/26/2011 02:54:51 PM nsrjobd JOBS notice: Starting full purge of jobs database

39074 02/26/2011 02:57:46 PM nsrjobd JOBS notice: Completed full database purge in 2 min 55 sec. Records purged: 28020

Windows 39074 3/9/2011 1:00:36 PM nsrjobd JOBS notice: Starting full purge of jobs database

39074 3/9/2011 1:09:50 PM nsrjobd JOBS notice: Completed full database purge in 9 min 14 sec. Records purged: 69067

Page 14: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

14

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

Figure 8. Jobsdb purge operations

Analysis and recommendations

28020 NetWorker server records are purged on Linux, and 69067 records are purged on a Windows NetWorker server. The IOPS requirement increases exponentially as the number of records increases.

IO load on the NetWorker server increases when the purge operation starts. Failure to meet the IOPS requirements for purge operations has a direct impact on overall performance. Also, if NetWorker operations that generate high IO load on the NetWorker server starts while the purge operation is running, NetWorker server performance is negatively impacted.

Purge operations can take 30 IOPS for small environments with up to 1000 backup jobs per day, 100 IOPS for medium environments, and up to 200 IOPS for large environments with high loads of 50,000 jobs per day.

IOPS requirements for NetWorker maintenance operations

Browsing NetWorker media and index operations generates very high IO operations on the NetWorker server. The nsrinfo program also generates very high IO load on the NetWorker server as compared to the mminfo program.

Page 15: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

15

Data analysis

IOPS Requirements for the NetWorker Server Technical Notes

Figure 9. IOPS during nsrinfo testing

Figure 10. IOPS during mminfo testing

Analysis and recommendations

The nsrinfo program took a maximum of 500 IOPS to browse 10000 savesets on one client. During browsing, IOPS remains very high until the browse operations complete.

To browse 50000 savesets simultaneously using mminfo, the NetWorker server required a maximum of 900 IOPS.

It is advised to run NetWorker browse operations (especially nsrinfo) at off peak times on the NetWorker server to accommodate different load operations.

Page 16: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

16

General IOPS recommendations

IOPS Requirements for the NetWorker Server Technical Notes

IOPS requirements on the NetWorker server for bootstrap backup

Figure 11. Average and bootstrap IOPS

Analysis and recommendations

IOPS requirements increase if NetWorker must perform a large bootstrap backup that is dependent on the size and number of files (/nsr/res/jobsdb) in the bootstrap backup. It is recommended to run the bootstrap backup during off peak times on the NetWorker server, otherwise the NetWorker software can experience slow response times during the bootstrap backup. There are a large number of spikes in IOPS during the bootstrap backup on both Linux and Windows. However, the bootstrap backup is often slightly faster on Linux. However, the speed and type of local disk on the NetWorker server can impact the improved Linux performance.

Note: IOPS on the NetWorker server increase only if the bootstrap is runs concurrent to regular backups. However, if the bootstrap backup is configured to run during an idle time, IOPS requirements are not increased.

General IOPS recommendations

This section describes general IOPS recommendations.

Requirements for monitoring the NetWorker datazone

Storage must provide a minimum of 30 IOPS to the NetWorker server. This value increases depending on the NetWorker server load.

Page 17: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

17

General IOPS recommendations

IOPS Requirements for the NetWorker Server Technical Notes

Requirements for backup job startup and duration

The startup and duration requirement for backup jobs is the largest portion of the NetWorker workload:

Depending on load, add (maximum concurrent number of sessions reached on the server, and divide it by 3) to IOPS requirements.

Maximum parallelism of the NetWorker server is 512, so the highest possible load is 170 IOPS.

IOPS requirements further increase if NetWorker is required to perform index and bootstrap backups at the same time.

For simultaneous index, and bootstrap backups, add 50 IOPS for small servers (up to 100 clients), 150 for medium (up to 400 clients) and 400 for large servers. Add this only if the bootstrap is running concurrent to regular backups. However, if the bootstrap backup is configured to run during idle time, IOPS requirements are not increased.

IOPS requirements further increase if NetWorker is configured to start a large number of jobs at the same time.

To accommodate for such spikes in load, add 1 IOPS for each session that is started in parallel.

Recommendation is not to start more than 40 clients per group using the default client parallelism of 4. This results in 160 IOPS during group startup.

Starting a large number of clients simultaneously can lead to system I/O depravation.

Each volume request results in a short I/O spike of approximately 200 IOPS for few seconds. For environments with a small number of volumes, this effect is minimal. However, for environments with frequent mount requests, an additional 100 IOPS is required (greater than 50 mount requests per hour). To avoid excessive load, use a smaller number of larger volumes.

NDMP backups add additional load due to index post-processing.

For environments with large NDMP backups (more than 10 million files), add an additional 120 IOPS.

Page 18: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

18

General IOPS recommendations

IOPS Requirements for the NetWorker Server Technical Notes

Requirements for successful completion of internal maintenance tasks

The following lists additional required IOPS for internal maintenance tasks:

Daily index and media database consistency checks add 40 IOPS for small environments and up to 200 IOPS for large environments with more than 1,000 configured clients.

Environments with very long backup and retention times (1 year or longer) will experience large internal database growth resulting in additional requirements of 100, and up to 200 IOPS.

Purge operations can take 30 IOPS for small environments with up to 1000 backup jobs per day. 100 IOPS for medium environments, and up to 200 IOPS for large environments with high loads of approximately 50,000 jobs per day.

Requirements for reporting tasks

Monitoring tools, such as the NMC server, DPA, or custom reporting or monitoring scripts add an additional load to the NetWorker server:

For each NMC server, add an additional 100 IOPS

For DPA reporting, add an additional 250 IOPS

Customer reporting or monitoring scripts can add an additional load. For example, continuous reporting on the NetWorker index and media databases can add up to 500 IOPS.

Requirements for manual tasks

Each recover session required to enumerate objects on the backup server will add additional load to the NetWorker server.

For example, to fully enumerate 10,000 backup jobs, the NetWorker server can require up to 500 IOPS.

The total number of IOPS should increase by 30% to allow for spikes, and unrelated operating system workloads.

Table 4. Required IOPS for NetWorker server operations Type of operation Small

NetWorker environment

Medium NetWorker environment

Large NetWorker environment

Concurrent backups 30 80 170

Bootstrap backups 50 150 400 Backup group startup 50 150 250 Volume management 0 0 100

Page 19: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

19

General IOPS recommendations

IOPS Requirements for the NetWorker Server Technical Notes

Large NDMP backups 100 100 200 Standard daily maintenance tasks 40 75 100

Maintenance of large internal databases 0 100 200 Purge operations 30 100 200 NMC reporting 50 75 100 DPA reporting 50 100 250 Recovery 30 200 500

The following lists the definitions for small, medium and large NetWorker environments:

A small NetWorker environment is one with less than 100 clients, or 100 concurrent backup sessions

A medium NetWorker environment is one with a maximum of 400 clients, or 250 concurrent backup sessions and

A large NetWorker environment is with more than 400 clients, or 500 concurrent backup sessions.

NetWorker does not limit the maximum number of clients per datazone, however the recommendation is to have a maximum of 1000 clients. This limit is due to the complexity of managing large datazones, and the increased hardware requirements on the NetWorker server.

Considerations

As the I/O load on the NetWorker server increases, the storage layer service times increase as well. If service times exceed the required values, there is a direct impact on NetWorker server performance and reliability. Information on Requirements for maximum service times is available in the NetWorker Performance Optimization Planning Guide.

If the NetWorker server is performing the actual data movement (the backup device is on the same system and not on the NetWorker storage node) performance of the backup server is directly impacted.

The same physical storage sub-system can deliver different performance depending on configuration. For example, splitting a single NetWorker mount point (nsr) into multiple ones can significantly increase performance due to the parallelism of the underlying filesystem handler in the operating system. More information on storage optimization is available in the NetWorker Performance Optimization Planning Guide.

Single disk performance is often insufficient for large NetWorker servers. A comparison of single disk performance is available in Table 4. To achieve higher IOPS numbers, combine multiple disks for parallel access.

Page 20: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

20

General IOPS recommendations

IOPS Requirements for the NetWorker Server Technical Notes

Best performance for standard disks is achieved with RAID 0+1, but modern storage arrays are commonly optimized for RAID5 access for random workloads that are commonly seen on NetWorker server.

NetWorker uses committed I/O writes, so write caching on the operating system has little or no impact, as each atomic write is flushed. However, hardware-based write-back cache can significantly improve NetWorker server performance. Table 6 provides details on hardware requirements.

The following are examples based on the IOPS testing:

Small to medium NetWorker datazones: Optimized: 200 clients running with parallelism of 100 jobs.

Up to 1,000 backup jobs spanning the entire day, with no external reporting and no overlapping maintenance tasks.

Minimum required IOPS: 200. Recommended IOPS: 400. Non-optimized: same workload with most jobs starting at

the same time. Overlapping production backups, bootstrap and maintenance jobs with additional reporting.

Minimum required IOPS: 800. Recommended IOPS 1000. Large NetWorker datazones:

Optimized: 1000 clients running with parallelism of 500 jobs. Up to 50,000 backup jobs spanning the entire day, using B2D or large tape volumes, with no external reporting and no overlapping maintenance tasks.

Minimum required IOPS: 800. Recommended IOPS: 1000. Non-optimized: workloads with the majority of jobs starting

at the same time. Using a large number of small volumes, overlapping production backups, bootstrap and maintenance jobs with additional reporting present.

Minimum required IOPS: 2000. Recommended IOPS: 2500. Different NetWorker configurations can result in up to 250% additional load on the NetWorker server. Also, the impact on sizing is that well-optimized large environments perform better than non-optimized medium environments.

Page 21: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

21

Test setup

IOPS Requirements for the NetWorker Server Technical Notes

Test setup

This section explains the setup used for the IOPS testing used, and on which the conclusions and recommendations are based.

Figure 12 represents the setup used in the IOPS testing.

LAN 1 Gbps

CLARiiON CX960

SAN

NW Server(Windows/

Linux) NW SN1/Client1NW SN2/Client2

NW SN3/Client3

NW SN4/Client4

NW SN5/Client5

Figure 12. IOPS test setup

Page 22: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

22

Test configuration

IOPS Requirements for the NetWorker Server Technical Notes

Test configuration

This section describes the configurations used for the NetWorker server IOPS performance testing.

NetWorker software

The following lists the NetWorker software details used for the IOPS testing:

• NetWorker Release 7.6 Service Pack 2 on the NetWorker server, storage node and clients

• AFTD device

• NetWorker server parallelism was set to 512 (maximum)

Storage

The following lists the storage details used for the IOPS testing:

• CLARiiON CX960 with dedicated SP

• Default settings were used on the CLARiiON

NetWorker server operating systems

The following lists the NetWorker software operating system details used for the IOPS testing:

Windows 2008 R2

Linux RHEL 5.5 x64

Approach

The following is the approach used to calculate the NetWorker server IOPS values during testing:

Operating system Windows & Linux

NetWorker server (nsr) IOPS

Number of sessions 1, 5, 10, 100, 1000, 4500 Dataset type 100 GB RAID type 3 and 5 Number of disks 5

Page 23: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

23

Test configuration

IOPS Requirements for the NetWorker Server Technical Notes

Measurement tools

The following tools were used to measure the NetWorker server IOPS:

On Windows, the perfmon counter was used to measure the transfer per second for average and maximum IOPS.

On Linux, the iostat command was used to measure the total IO requests per second.

Dataset details

This table lists the details of the dataset used in the IOPS testing.

Table 5. Dataset details in IOPS testing Num_of_sessions Dataset size Details

1 100 GB 1 client configured with a LUN-1 approximately 100 GB with one 100 GB file for backup.

5 100 GB 5 clients configured with LUN-2s approximately 25 GB with one 20 GB file. Five 20 GB savesets for backup.

10 100 GB 5 clients configured with LUN-3s approximately25 GB with two 10 GB files.

100 100 GB 5 clients configured with LUN-4s approximately 25 GB with twenty 1GB files.

1000 100 GB 5 clients with each client configured with LUN-5s approximately 25 GB with two hundred files 0.1GB files.

4500 100 GB 5 clients configured with LUN-6 approximately 25 GB with one thousand 0.02 files.

Page 24: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

24

Test configuration

IOPS Requirements for the NetWorker Server Technical Notes

Hardware configuration

This table describes the hardware configurations used in the IOPS testing.

Table 6. Hardware configuration in IOPS testing Server #

Role of the machines

Operating system

System model

Processors System type

Memory (RAM)

1 Networker Server/storage node

Windows 2008R2

PowerEdge 2950

2 dual core @ 3 Ghz

x64-based PC

12 GB

2 Networker Server/storage node

Linux RHEL 5.5

PowerEdge 2950

2 dual core @ 3 Ghz

x64-based PC

12 GB

3 Networker storage node/client1

Solaris10( SUNT5140)

SUNT5140 8 physical and 128 vCPU @ 1.2Ghz

SPARC 16 GB

4 Networker storage node/client2

Solaris10 Sun Fire V245

2 CPUS @ 1.5 GHz

64-bit, SPARC V9

4 GB

5 Networker storage node/client3

Windows 2008

PowerEdge 2950

2 dual core @ 3 Ghz

x64-based PC

4 GB

7 Networker storage node/client4

Linux PowerEdge 2950

2 dual core @ 2.3 GHz

x64-based PC

8 GB

8 Networker storage node/client 5

HPUX 11.31 HP-UX server rx2620

2 CPUs @1.6Ghz

ia64 (itanium)

2GB

Page 25: IOPS Requirements for the NetWorker Server Executive summary IOPS Requirements for the NetWorker Server Technical Notes Executive summary The 4 major components for system performance

25

Test configuration

IOPS Requirements for the NetWorker Server Technical Notes

Copyright © 2010 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

All other trademarks used herein are the property of their respective owners.