HelgeKlein.com Home of
SetACL
A scientific approach toXenApp farm sizing
Helge Klein
HelgeKlein.com Home of
SetACL
Who isHelge Klein?
CTP, MVPAuthor of SetACL and Delprof2Independent consultant and developerArchitect of what later became Citrix Profile Management
HelgeKlein.com Home of
SetACL
talking about?What is he
Scientifically sound farm sizing methodologyHow to calculate farm capacity
HelgeKlein.com Home of
SetACL
Methodology
1. Determine capacity of existing farm2. Measure load and identify bottlenecks3. Calculate capacity of new farm
HelgeKlein.com Home of
SetACL
ExampleDescription Farm capacity
old (total)Farm capacity old (per user)
Factor Farm capacity new (per user)
Farm capacity new (total)
RAM for user sessions [MB] 630,000 315 2.0 630 1,575,000
CPU [normalized cores] 1,250 0.63 0.25 0.16 391
Hard disk [write IOPS] 7,300 3.65 1.5 5.48 13,688
Network [MB/s] 24,000 12.00 0.1 1.20 3,000
HelgeKlein.com Home of
SetACL
Where to get the
numbers?1. Data collection2. Observation3. Measurements4. Calculation
HelgeKlein.com Home of
SetACL
Data collection
HelgeKlein.com Home of
SetACL
for /f %i in (AllFarmServers.txt) do wmic /node:%i cpu get name, maxclockspeed, systemname, description, manufacturer, revision /format:csv >> CPUs.txt
Collect CPU dataCreate AllFarmServers.txt with qfarm
Use resulting list to determine server model
HelgeKlein.com Home of
SetACL
Srv001,x86 Family 15 Model 4 Stepping 10,GenuineIntel,3400,Intel(R) Xeon(TM) CPU 3.40GHz,1034,Srv001Srv001,x86 Family 15 Model 4 Stepping 10,GenuineIntel,3400,Intel(R) Xeon(TM) CPU 3.40GHz,1034,Srv001Srv001,x86 Family 15 Model 4 Stepping 10,GenuineIntel,3400,Intel(R) Xeon(TM) CPU 3.40GHz,1034,Srv001Srv001,x86 Family 15 Model 4 Stepping 10,GenuineIntel,3400,Intel(R) Xeon(TM) CPU 3.40GHz,1034,Srv001Srv002,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv002Srv002,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv002Srv002,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv002Srv002,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv002Srv002,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv002Srv002,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv002Srv002,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv002Srv002,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv002Srv003,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv003Srv003,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv003Srv003,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv003Srv003,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv003Srv003,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv003Srv003,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv003Srv003,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv003Srv003,x86 Family 6 Model 44 Stepping 2,GenuineIntel,2666,Intel(R) Xeon(R) CPU E5640 @ 2.67GHz,11266,Srv003Srv004,x86 Family 6 Model 26 Stepping 5,GenuineIntel,2666,Intel(R) Pentium(R) III Xeon-Prozessor,6661,Srv004Srv004,x86 Family 6 Model 26 Stepping 5,GenuineIntel,2666,Intel(R) Pentium(R) III Xeon-Prozessor,6661,Srv004Srv004,x86 Family 6 Model 26 Stepping 5,GenuineIntel,2666,Intel(R) Pentium(R) III Xeon-Prozessor,6661,Srv004...
The result
HelgeKlein.com Home of
SetACL
RAM, NICshard disks
Could be determined via WMI, tooOften knowing the server model is sufficientComponents per model often identical
HelgeKlein.com Home of
SetACL
Component HP DL360 G4
CPU 2x Xeon dual core 3,0 GHz, no Hyper Threading4 Cores, CPUID: x86 Family 15 Model 4 Stepping 1
RAM 4 GB
NICs 2x Gigabit, 1 connected
RAID Smart Array 6i
Hard disks 2x 70 GB configured as RAID-1, Compaq BF0728A4CB, 3,5“, 15.000 R/Min, Ultra320 SCSI
Spec sheet
HelgeKlein.com Home of
SetACL
Resulting dataServer model NumberDL360 G7 26DL360 G6 52DL360 G5 70DL360 G4p 60DL360 G4 35
HelgeKlein.com Home of
SetACL
Observation
HelgeKlein.com Home of
SetACL
How many users are logged on?Load of CPU, RAM, NICs
Individual processes with a lot of RAM or CPU?
HelgeKlein.com Home of
SetACL
Physical Disk\% Disk Time
Physical Disk\Avg. Disk Queue Length
„Time“ is equivalent to flickering of hard disk LEDDisk queue length: Number of waiting IOs
HelgeKlein.com Home of
SetACL
Hypothesis: farm is
memory limited
Limiting factor will differ between farms
HelgeKlein.com Home of
SetACL
Measurements
Tool: PerfmonNext slides: relevant counters
HelgeKlein.com Home of
SetACL
Terminal Services\Active SessionsTerminal Services\Inactive Sessions
Terminal Services\Total SessionsSystem\Processes
General system information
HelgeKlein.com Home of
SetACL
PhysicalDisk(_Total)\% Disk TimePhysicalDisk(_Total)\Avg. Disk Queue Length
PhysicalDisk(_Total)\Disk Reads/secPhysicalDisk(_Total)\Disk Writes/sec
PhysicalDisk(_Total)\Avg. Disk sec/Transfer
Hard disk activityLoad, queue length, operations per second, latency
HelgeKlein.com Home of
SetACL
Processor(_Total)\% Processor TimeMemory\Available MBytes
Network Interface(*)\Bytes Total/sec
CPU, RAM and networkRAM: total amount must be known!
HelgeKlein.com Home of
SetACL
logman create counter TSPerf -f csv -cf C:\PerfLogs\Counters.txt -o C:\PerfLogs\Server13.csv -si 60 -rf 24:00:00
AutomationCreate and start data collector set. Format CSV, performance counters are
read from C:\PerfLogs\Counters.txt, output file is C:\PerfLogs\Server013.csv, 60 second sampling interval, duration 24 hours.
HelgeKlein.com Home of
SetACL
logman create counter TSPerf -f csv -cf C:\PerfLogs\Counters.txt –o C:\PerfLogs\Server13.csv -si 60 -rf 24:00:00-s Server13
Execution on remote computer Server13
HelgeKlein.com Home of
SetACL
for /f %i in (Servers.txt) do logman create counter TSPerf -f csv -cf C:\PerfLogs\Counters.txt -o C:\PerfLogs\%i.csv -si 60 -rf 24:00:00 -s %i
Many serversOne computer name per line in Servers.txt
HelgeKlein.com Home of
SetACL
Analyzingthe measured data
HelgeKlein.com Home of
SetACL
CPU and sessions
Server 37
Moderate load during logon phase,
afterwards even less
A lot of overcapacity
HelgeKlein.com Home of
SetACL
CPU and sessions
Server 89
HelgeKlein.com Home of
SetACL
Overlaying the CPU load of
many servers
Easily verify the analyis
HelgeKlein.com Home of
SetACL
HDD and sessions
Server 37
Moderate load, peaks during logon phase
Full load at aprox. 200
HelgeKlein.com Home of
SetACL
HDD and sessions
Server 89
HelgeKlein.com Home of
SetACL
Overlaying the HDD load of
many servers
Easily verify the analyis
HelgeKlein.com Home of
SetACL
RAM and sessions
Server 37
Continually increasing load, maximum in the
afternoon
Available RAM must not go near zero
(because of disk cache)
High load
HelgeKlein.com Home of
SetACL
RAM and sessions
Server 89
HelgeKlein.com Home of
SetACL
Overlaying the memory load
of many servers
Easily verify the analyis
HelgeKlein.com Home of
SetACL
Network and sessions
Server 37
200 = 2 MB/s
Average rate < 200 KB/s
Very low load, a lot of overcapacity
HelgeKlein.com Home of
SetACL
Network and sessions
Server 89
HelgeKlein.com Home of
SetACL
Overlaying the network load
of many servers
Easily verify the analyis
HelgeKlein.com Home of
SetACL
Hypothesis confirmed: farm is limited by
available memoryCPU load: low, network: negligiblehard disk: moderate
HelgeKlein.com Home of
SetACL
Calculatingfarm capacity
Description Farm capacity old (total)
Farm capacity old (per user)
Factor Farm capacity new (per user)
Farm capacity new (total)
RAM for user sessions [MB] 630,000 315 2.0 630 1,575,000
CPU [normalized cores] 1,250 0.63 0.25 0.16 391
Hard disk [write IOPS] 7,300 3.65 1.5 5.48 13,688
Network [MB/s] 24,000 12.00 0.1 1.20 3,000
HelgeKlein.com Home of
SetACL
How to compare performance of different CPUs?Benchmarking is difficultBetter: Moore’s law (doubling of performance every 18-24 months)Surprisingly accurate (amongst other things because it is a self-fulfilling prophecy)
NormalizingCPU performance
HelgeKlein.com Home of
SetACL
Performance after time (in months)Assumed performance doubling every 21 monthsOldest CPU in farm = 1,0
𝐿(𝑡 )=𝑒0,033𝑡
HelgeKlein.com Home of
SetACL
CPU Cores First sold Performance index Performance index per core
Xeon 7100 „Tulsa“ 3,0 GHz
2 08/2006 1,0 1,0
Xeon 7100 „Tulsa“ 3,4 / 3,6 GHz
2 03/2007 1,26 1,26
Xeon E5430 / E5440
4 11/2007 1,64 0,82
Xeon X5550 4 03/2009 2,78 1,39
Xeon E5640 4 03/2010 4,13 2,06
HelgeKlein.com Home of
SetACL
CPU Number of CPUs
Number of cores
Number of normalized CPUs
Number of normalized cores
Xeon 3 GHz 70 140 70 140
Xeon 3,4/3,6 GHz
119 238 150 300
Xeon E5430/E5440
70 280 115 230
Xeon X5550 67 268 186 370
Xeon E5640 26 104 107 210
Total 352 1030 628 1250
HelgeKlein.com Home of
SetACL
Farm capacity:
1250 normalized CPUs
0.63 CPUs / user
HelgeKlein.com Home of
SetACL
Hard disk performance =
IOPSWith many concurrent accesses transfer rate is mostly irrelevantMore important: IOPS (operations per second)Exact number depends on measurement method do not believe vendors
HelgeKlein.com Home of
SetACL
PhysicalDisk(_Total)\Disk Reads/secPhysicalDisk(_Total)\Disk Writes/sec
IOPS measurement with PerfmonRead and write IOPS may be very different
HelgeKlein.com Home of
SetACL
Read and write IOPS
Average: ~15
Including spikes: 30
HelgeKlein.com Home of
SetACL
Farm capacity:
7300 IOPS
3,7 IOPS / user
HelgeKlein.com Home of
SetACL
RAM – how muchdo we have?
We need total RAM that is available for user sessions
HelgeKlein.com Home of
SetACL
Server model RAM / server [GB] #Servers RAM / server total [GB]
DL360 G4 3,5 35 122,5
DL360 G4p 3,5 60 210
DL360 G5 3,25 70 227,5
DL360 G6 3,49 52 181,48
DL360 G7 3,49 26 90,74
Total 243 832
HelgeKlein.com Home of
SetACL
Description Memory [MB]Session 0 (OS + Presentation Server + antivirus) 325
Kernel (pooled + non-pooled) 175File system cache 400Total 900
Calculate RAM used by the OS (Server 2003)
HelgeKlein.com Home of
SetACL
Farm capacity:
630 GB
315 MB / user
25% Overhead because 900 MB used by OS
HelgeKlein.com Home of
SetACL
Network – too much for HDX?
1 Gigabit/second per server~ 100 MB/s
HelgeKlein.com Home of
SetACL
Farm capacity:
24 GB/s
12 MB/s / user
Extreme overcapacity
HelgeKlein.com Home of
SetACL
Futurecapacity
Description Farm capacity old (total)
Farm capacity old (per user)
Factor Farm capacity new (per user)
Farm capacity new (total)
RAM for user sessions [MB] 630,000 315 2.0 630 1,575,000
CPU [normalized cores] 1,250 0.63 0.25 0.16 391
Hard disk [write IOPS] 7,300 3.65 1.5 5.48 13,688
Network [MB/s] 24,000 12.00 0.1 1.20 3,000
HelgeKlein.com Home of
SetACL
Parameters. Calculation includes:Changed user count, different memory requirements of new OS,server replacement, capacity increase for disaster recovery
Parameter ValueFarm users old (CCU) 2,000
Farm users new (CCU) 2,500
RAM for OS new [MB] 1,300
CPU performance new [normalized cores]
1.9
IOPS per disk new 300
Reserve for disaster recovery 1.5
HelgeKlein.com Home of
SetACL
Servers are memory limited after setting the amount of RAM,the other parameters can be calculated
Virtual server
RAM [MB] 32.768
User sessions 49.9
CPU cores (physical) 4.1
IOPS 273
NICs 0.6
HelgeKlein.com Home of
SetACL
After setting the number of VMs per host the other parameterscan be calculated.Excel sheet available upon request.
Physical serversVMs per host 3 #Virtual
servers50
RAM [MB] 101,376 #Hosts 25
CPU cores 12 Users / host
150
Hard disks 3
NICs 2
HelgeKlein.com Home of
SetACL
Thank you!Helge Klein