Oracle E-Business Suite PC Client Performance and … E-Business Suite 11i and 12 PC Client Performance Page 4 Comparing ... consider which factors you need to include in your testing

Oracle E-Business Suite 11i and 12 PC Client Performance

An Oracle White Paper October 2007

Oracle E-Business Suite 11i and 12 PC Client Performance

Executive Overview .......................................................................................... 5 Why Not Just Skip to the Results? ............................................................. 5

Introduction ....................................................................................................... 6 The Importance of the Holistic Approach ............................................... 6 Oracle Client Recommendations ................................................................ 7

The Testing Methodology ................................................................................ 8 Creating a List of Test Candidate Transactions........................................ 8 The Test Environment and Testing Issues ............................................... 9 Testing Accuracy .........................................................................................11 PC Clients Used in these Tests .................................................................11

Windows Memory Definitions and Measurement Tools ..........................12 Different Types of Memory ......................................................................12

Choosing a Tool to Measure Memory.................................................13 Choosing Which Memory to Measure ................................................16

Process Trimming .......................................................................................19 Rules of Thumb – Private Bytes and Working Set ................................20

Understanding the Browser Memory Offset and Scaling Results ............21 How to Map Integrated Browser Components......................................22

Adobe Acrobat Reader, Skype, and the Google Toolbar.................22 Microsoft Phishing Option ...................................................................24

Factors That Affect the Browser Memory Footprint............................24 Deciding How To Scale Results ...............................................................25

Baseline the Browser Memory Offset .................................................25 Decide Which Scaled Approach to Use ..............................................25

Oracle E-Business Suite 12 – The Results ...................................................27 Comparing Browsers and Oracle E-Business Suite Components .......28 Average Memory by Oracle E-Business Suite Component Type........31 Reviewing the Ratio of Working Set and Private Bytes ........................31 Comparing Menu Memory Utilization.....................................................33 Comparing Memory Profiles across a Range of Clients........................34 Comparing Load Times for across the Range of Clients ......................35

Comparing Load Times across Fast Clients .......................................36 Comparing Reload Times......................................................................37 Comparing Load Times and Jar File Volumes...................................38

Network Traffic for Oracle E-Business Suite 12 with JRE..................39 Comparing Oracle E-Business Suite 11i and 12 .........................................42

The Effect of Business Flows in Oracle E-Business Suite 11i and 12 42 Comparing Working Set for Oracle E-Business Suite 11i and 12 .......43

Oracle E-Business Suite 11i and 12 PC Client Performance Page 3


Comparing Oracle JInitiator 1.3.1.29 with Sun JRE 1.5.0_12..............44 Recommendations For Low Specification Clients .....................................45

Is a Low Specification Client Usable? ......................................................46 Controlling the User Environment ..........................................................46 The Low-Specification Client Checklist ..................................................46

Memory Optimizers ...............................................................................49 So, How Much Memory Does my Application Need? .........................49 More Testing Tips.......................................................................................49

Conclusion........................................................................................................50 Appendix A: OAF/HTML Screens and Forms Used in the Tests .....52 Appendix B: Tuning the Client .................................................................55 Appendix C: Windows Memory Internals...............................................57

EXECUTIVE OVERVIEW It often makes financial sense to extend the operational life of IT equipment. This paper provides guidelines on how to improve performance and thereby possibly extend the useful life of slow or low-memory PC clients. It identifies specific scenarios where the difference between low and high specification clients makes very little difference to end-user performance, and answers the important question of whether you need to upgrade memory, CPU, or both. This paper compares Oracle E-Business Suite 11i and 12 using certified browsers, Oracle JInitiator 1.3.1.29 (Oracle E-Business Suite 11i only) with the Sun Java2 Standard Edition Java Runtime Engine 1.5.0_12 on a range of PC client configurations using Windows XP.

Extending the useful life of older PC clients can often be cost-effective, even if

they have been fully depreciated.

Upgrading to the latest and fastest machines helps provide the best performance, but the latest technology tends to command a price premium. The costs involved may be disproportionate to the business gain and the performance may be substantially more than strictly necessary to support the business. Budgetary constraints and asset life dictate that the price/performance goal is to balance the return on investment while providing reasonable longevity. Although the cost of hardware continues to diminish, the time needed to upgrade a range of machines can be prohibitive, especially when multiple geographic locations are involved.

The information in this paper will help you make informed decisions to establish the minimum requirements for a specific price-performance point, and identify a specific configuration that will achieve the throughput necessary to support your business. The approach, tools, and observations provide a quantifiable basis for desktop upgrade decisions that enable you to determine a suitable specification for a PC client. This can be benchmarked with your particular set of applications in a range of heterogeneous environments to ensure its suitability and performance.

Extending the concepts presented in the The Holistic Approach to Performance Tuning Oracle E-Business Suite whitepaper, this paper applies a holistic approach and considers all the salient facts that may affect performance. This provides a foundation for a cost-benefit analysis between zero-cost options, upgrading the hardware as a one-off cost, and the on-going costs incurred from improvements to the network infrastructure.

The holistic approach provides a foundation for a cost-benefit analysis and

a quantifiable basis for desktop upgrade decisions.

Why Not Just Skip to the Results? You can find a plethora of definitions for every aspect of Windows terminology across the Internet, some right, some incomplete, and some wrong. The definitions used in this paper are very precise, and this necessitates some complexity and discussion around Windows memory management. A basic understanding of what is being measured is essential for a meaningful review of the comparisons and charts.

A basic understanding of what is being measured is essential for a meaningful review of the comparisons and charts.


INTRODUCTION While this paper focuses on low specification clients, some customers will already have sufficient hardware in place, but will be concerned about their ability to support the business at some point in the future. Even if you have what would normally be considered a high-specification machine, you may still encounter memory problems when running several applications simultaneously, so the ideas and concepts presented here are still relevant.

As this paper cannot cover all the permutations and combinations of applications that you could be running, it provides a comprehensive approach and describes the tools that you can use to profile the application and Windows environment.

Microsoft state that adding memory makes a significant difference to Windows performance and this is also true for some Oracle E-Business Suite components. This statement does not mention CPU speed, and therefore it appears that they consider this less significant. Consequently, this paper includes data points from a range of PC clients that will help validate whether or not additional memory alone is likely to improve performance sufficiently.

This paper does not discuss Windows Vista or Sun Java2 Standard Edition Java Runtime Engine 1.6. Certification with the Oracle E-Business Suite is planned and details will be included in a later revision of the paper.

The Importance of the Holistic Approach Understanding where and how the machines will be deployed is just a starting point and adopting a holistic approach is essential. In addition to considering factors that affect your tests directly, such as the other applications that will be used on the client, you also need to consider key transactions that are necessary to support the business, simulation and modeling of the network, and whether some performance issues are a result of other factors such as application tier or database performance.

The holistic approach provides a foundation for a cost-benefit analysis and

a quantifiable basis for desktop upgrade decisions.

It is very unlikely that this paper could match the exact combination of Oracle E-Business Suite components with the specific combination of software that you use on the same specification client, in addition to modeling specific network scenarios. Therefore, the rationale of this paper is to provide examples of Oracle E-Business Suite components and an approach that you can use in your own environment.

In addition to providing benchmarks for the range of machines that you would expect to deploy, this paper also includes a very low-specification client (below the recommendation for the Oracle E-Business Suite) as a checkpoint and control. The memory on this client could not be upgraded, but the information may be useful, for example, for rugged machines typically used in hostile environments that may be exceptionally difficult or expensive to upgrade.

Including a client below the minimum recommendation provides insights to the

performance of specialized clients that are difficult or expensive to upgrade.


Oracle Client Recommendations Oracle has traditionally taken the same approach as Microsoft by specifying the minimum desktop client recommendations. While the official specifications offer a useful starting point, as shown in Table 1, the ultimate decision of what constitutes acceptable performance for a given scenario in your environment is driven by your business requirements.

Oracle Desktop Client Recommendations Required Recommended

Processor 233MHz 1.4GHz + or higher11i

MetaLink Note 232833.1

RAM 256MB Or higher1

Processor 350MHz 1.4GHz + or higherR12 MetaLink Note 405293.1

RAM 256MB Or higher1

Table 1: Oracle Client Requirements and Recommendations 1You will experience better performance with additional memory

The Oracle client recommendations state that when evaluating CPU speed and amount of memory, consider Microsoft's minimum requirements for the operating system, other software that runs concurrently, and external factors such as network characteristics. You should check that the desktop configuration is sufficient to achieve the throughput necessary to sustain your business model; your requirements may be higher than the minimum specification. Even if you have what would normally be considered a high-specification machine, you may encounter performance problems when running several applications simultaneously.

A minimum specification client only provides a minimal performance that may

not be sufficient to support your business. It is important to note that minimum specifications are exactly that, and will only provide minimum performance. A minimum specification client may be adequate for casual home use, but would probably not be considered sufficient when running several applications together as would typically be the case in a business setting. For example, displaying thousands of tasks and shifts that span several weeks on a Gantt chart in Advanced Supply Chain Planning will cause severe paging on minimum specification machines, and the PC client will appear unresponsive for a considerable time.


THE TESTING METHODOLOGY You will need to create a list of candidate transactions, benchmark a range of configurations to determine performance within your specific environment, and then consider which factors you need to include in your testing. This section provides guidelines on selecting a set of transactions to test, and provides a provisional set of guidelines based on the experience gained during the testing that was conducted for this paper.

Creating a List of Test Candidate Transactions One of the primary components of a benchmark is a set of key transactions that are usually best defined by the business and user managers. Users typically cite form or screen opening times and inter-field navigation as performance concerns. As data and inter-field operations generally have a high dependency on the data volumes, system setup, profile options, and other variables, these factors have been excluded as the test environment is based on a single machine using a default Oracle E-Business Suite install and the Vision database.

Create a list of critical business transactions that you

need to benchmark.

If you wish to expand the tests in this paper, it is important not to confuse issues directly relating to client performance with other factors such as navigating between fields on a form or querying data. These are more likely to be influenced by the relative performance of the database, application tiers, and associated servers.

The tests in this paper focus on the time to open a form or screen as the primary indicator of client performance. While some tests include querying data, this is generally restricted to situations where this is required to enable other forms to open. Their definitions are listed in Appendix A. Some screens and forms can be accessed through multiple responsibilities or users, but each path may produce different results. For this reason, the user details and menu navigation path are included to ensure that any comparisons are valid and to assist with comparative testing in the event that you are unfamiliar with the full range of products.

The tests in this paper focus on the time to open a form or screen as the primary

indicator of client performance.

Your initial list of forms and screens is collated by surveying your departments or business management units. The composite list may be very large and you may need to create a subset of representative test candidates. If your background is technical, you may consider mapping the forms and their respective transactions against high-volume or high-transaction tables. However, this will need further review from a functional perspective and understanding of the business flow that includes which forms can be kept open simultaneously, and (and more importantly) the frequency of switching between forms, screens, and responsibilities.

You may also benefit by using an approach where a business process can be broken into a set of discrete functions that need to be completed within a set time. For example, a customer service agent who takes an order over a phone will typically need to perform the following tasks:

• Enter or locate customer and address information • Enter an order for the goods and check their availability • Enter payment details • Create a picking list for the warehouse • Print an invoice and shipping label


While this may be a simplification of your business flow, it is easy to identify the first three processes as critical business functions, i.e. those that need to be completed in a timely manner. If the order entry clerk needs to switch between screens, the system will need to react quickly enough so that the customer does not become frustrated with the delay and possibly even terminate their order. The final two tasks may be classified as non-critical as they can be deferred or run as concurrent processes; in any event, they are not relevant in this context, as the process code does not interact with the PC client.

Once you have a list of forms and screens, you need to decide and agree performance targets for each component that is essential to sustain the critical business functions, as this will decide the amount of client tuning and minimum hardware requirements. The following set of guidelines provides the basis for a test specification:

Ensure that your simulation models have all salient characteristics including the

network and other applications that will be running simultaneously.

• Specify which forms and screens need to be open at the same time.

• Specify other applications that will be used at the same time including for example, firewalls, antivirus, chat, email, spreadsheets, word processors, diary systems, and other internal systems and applications. Importantly, specify if these are usually minimized when not in use.

• Specify the network characteristics that need to be modeled including latency, bandwidth limitations, acceleration, and saturation levels. Other less tangible characteristics such as packet loss, or fragmentation (assembly and disassembly), can be a major overhead in certain geographies, or on particular types of network.

Specify any factors that the desktop policy allows that would degrade client performance. For example, allowing a high-resolution photo as a wallpaper and/or Windows animation effects may degrade the form display time by as much as 30% on low specification clients. The exact list of items that needs to be considered or included increases as the machine specification decreases. A set of high impact changes for low-memory clients is included in Appendix B: Tuning the Client.

The Test Environment and Testing Issues The following web browsers are certified by Oracle for use with Windows XP clients in Oracle E-Business Suite 11i and 12 environments:

• Internet Explorer 6 • Internet Explorer 7 • Firefox 2

Each browser is an application in its own right, and each has different characteristics with a different memory footprint and behavior. The amount of memory used by a browser when it is started with a blank “Home Page” is known as the browser’s memory footprint and referred to as the browser offset. This needs to be investigated and taken into consideration when analyzing results and making recommendations.

The initial amount of memory used by a browser is referred to as an offset and needs to be considered when

analyzing the results.


All the tests in this paper were run against an Oracle E-Business instance with the Vision Demo Database. The machine was a single 2.13GHz Intel Core Duo server with 4GB RAM running Red Hat Linux 4 using the ext3 file system on a logical volume group that spanned two disks. As this machine is exceptionally small when compared to a typical production system, all the tests were run once prior to recording results to ensure that the performance of this machine did not adversely affect the tests. This is similar to most production systems that utilize a number of caches that maintain the code and data that is common use. While this does not affect client memory, you should expect to achieve better response times on your system.

You should expect to achieve better response times on your system.

The benchmarks compare Oracle JInitiator 1.3.1.29 and the Sun Java2 Standard Edition Java Runtime Engine (JRE) 1.5.0_12, which were the latest certified versions at the time of writing. No patches were applied to the Oracle E-Business Suite 11i or 12 environments, even though there are several patches or family packs that you would normally apply to any environment for functionality or performance. This approach means that you can quickly replicate this environment. It may be pertinent to compare a subset of tests in your environment to establish if there is a significant difference or factor that can be applied to scale the results in this paper.

To save time, use a subset of tests to establish a scaling factor that can be

applied to the results in this paper.

When a form or screen starts, it will frequently access system, product, user, and other information. This initial communication between the client and server is affected by latency, available bandwidth, and possibly other constraints related to the network infrastructure (such as packet prioritization). Therefore, if you are planning to deploy clients in remote high-latency locations, you will need to simulate the salient conditions in your test environment. This is especially important as a form accessed over a LAN may only take a few seconds to open, whereas it may take a minute or even longer when deployed over a high latency network link. If it transpires that latency is the primary performance bottleneck, then you should consider traffic compression technologies and caching. Network optimization will have a positive effect on overall client performance.

You need to encompass all salient client and network conditions in your test

environment.

The following key points were identified during testing:

• The nature of Windows memory management means that you need to restart the browser in-between each of the tests when determining the memory profile of individual forms and screens. This does not affect combinations of screens used in a standard business flow.

• If you repeatedly open and close a browser you may notice that the memory measurements change, especially on very low-specification clients. This may be the result of paging, and should stabilize after a short time. It is imperative to ensure that the initial browser memory allocation does not change between tests, as this would skew the results.

• You need to ensure the working environment is representative of the user experience. In this paper, all the applications were opened in advance of the tests. Equally important is that they are in the same state each time. In the tests the windows overlaid each other; minimizing any of them at any time would have radically skewed the results. You will need to bear this in mind when following a set of actions that represent a standard business flow.


• Unless shown to be contributing to a memory problem, avoid minimizing and restoring the measurement tools as this significantly changes the memory profile on low-specification machines.

• Do not leave a test part way through and then return to it later, as the Windows memory manager may well have reorganized the Windows environment and this may create a substantially different environment.

• Unlike working on the Internet, if you make a mistake when selecting a form and quickly click on another option, you will find that you have an apparently anomalous large memory footprint. This is accounted for by both forms being opened, but only one being visible, as they overlay each other when they are rendered.

• With Internet Explorer 7 and Firefox 2 specifically, some of the memory measurements take some time to settle after a screen or form has opened.

• Several of the clients had a relatively large amount of memory (1-2GB) that should have been more than adequate to test without process trimming or other Windows memory management issues. It was surprising how few applications it took before Windows started to trim the Working Sets, so you should consider this, especially when testing a suite of applications. Refer to the Process Trimming section for further information.

Additional guidelines are provided in the More Testing Tips section.

Testing Accuracy If your simulation and testing methodology is consistent, repeated tests on the same PC client should be within a tolerance of 2.5%. If the accuracy is lower, then you should establish the cause, as it could be contributing to problems with the end-user experience.

Even though measuring Windows memory is an inexact science, you should achieve

an accuracy of 2.5%

PC Clients Used in these Tests The PC clients used in the test are listed in Table 2. All of the machines were running Intel processors with Windows XP SP2 and all the latest patches.

With the exception of the Intel 930 Core Duo PC, all the machines were laptops. The 366MHz/128MB client is below the Oracle recommendation, but the architecture meant that it was neither possible nor practical to change the amount of installed memory. This was included as it provides insights to the performance of specialized clients that are difficult or expensive to upgrade. In theory, as the tests were conducted with the minimum of other applications running, 256MB should be adequate for all except the most demanding of tests.

Virtual machines (VM) were considered as an easy way of changing the system memory, as they appeared to function without a problem. However, the memory profile (specifically Working Set and Private Bytes) within the VM environment did not always follow the same profile as a native Windows environment. At this stage, more research is needed to reconcile disparities in the memory measurements between real and virtual clients, and consequently further testing was not carried out.


Speed Memory Comment

366MHz Pentium II

128MB This is the minimum specification client for Oracle E-Business Suite 12. The memory is just above the Microsoft minimum but below Oracle recommendations.

700MHz Pentium III

256MB Although old technology, this is the minimum memory recommended for Oracle E-Business Suite 11i and 12.

1.6GHz Pentium

1GB This speed appears to be very common, but many users expect to run several large applications simultaneously but with only 256MB or 512MB RAM.

T2400 Mobile Core Duo 1.8GHz

2GB This is alleged to be equivalent to 3.43GHz (according to non-Intel resources). At the time of writing, this is the standard Oracle laptop (with either 1GB or 2GB of RAM).

Intel 930 Core Duo 3GHz

2GB This is one of the fastest clients available at reasonable cost without commanding a substantial price-premium.

Intel 930 Core Duo 3GHz

256MB Again, one of the fastest clients, but this will highlight differences between the 256MB and 2GB memory sizes.

Table 2: PC Clients Used for the Tests

WINDOWS MEMORY DEFINITIONS AND MEASUREMENT TOOLS One of the challenges of Windows internals is that definitions found on the Internet are often incomplete, taken out of context, or simply erroneous. For this reason, the specific meaning of each of these terms needs to be defined and explained. This section introduces the types of memory and how each is managed; further detail and more comprehensive definitions are included in Appendix C: Windows Memory Internals.

An understanding of the terminology is essential for successful interpretation of

the test results.

Different Types of Memory Like most operating systems, Windows uses a virtual memory model, and each 32-bit Windows process can have private and shared memory, which are defined as follows:

• Private memory – This is the exclusive private virtual memory allocated to the process and inaccessible by any other process. It is also referred to as Private Virtual Bytes.

• Shared Memory – Shared memory contains an image or a mapped file such as a dynamic link library (DLL) that can be simultaneously accessed by multiple programs. Once referenced, the object is mapped into each process’s address space even though the data or image file is only loaded once into system memory.

Private memory and shared memory are collectively referred to as the process’s Working Set (WS), which is defined as the combination of physical private and shared memory pages that have been used or recently “touched” by the threads of a process. They can be referenced without incurring a page fault.


The amount of shared memory depends on the combination of applications that you use concurrently, and (to some extent) the functions that you are using within the Oracle E-Business Suite. Figure 1 compares the average amount of shareable and shared memory for each of the browsers across a range of tests (on the 1.83GHz T2400 Core Duo with 2GB RAM). You would not expect many memory pages to be shared in the test environment, as the client only had standard background applications such as a virus checker, and a minimal business set of applications open including Mozilla Thunderbird (email), Pidgin (chat client), Process Explorer and Microsoft Excel.

The amount of shareable and shared memory depends on the other applications

in use at the same time.

Figure 1: Comparing Shareable and Shared Memory

The amount of shareable memory will vary over time, but is not really a concern, as it does not limit the actual amount of shared memory pages. As Internet Explorer is a Microsoft product, you would expect it to share more memory pages with the Windows Kernel and other Microsoft applications, and this is indeed evident in the results. The amount of shared memory used by Internet Explorer increases when also using other Windows applications such as Microsoft Outlook. Firefox is an open source browser, so not surprisingly shares a relatively small amount of memory with Windows components. Even so, the 3.5MB difference between Firefox and IE7 is only 35% and in practical terms, this is a small percentage. Several other factors need to be considered prior to choosing a particular browser.

Choosing a Tool to Measure Memory

Undertaking meaningful measurements of the various types of Windows memory can be complicated, so this section starts by providing an overview of the available tools and then maps them to the different type of memory in a tabular format.

This section focuses on the following three tools:


• Task Manager (taskmgr) – Start by entering taskmgr at a command prompt, from the taskbar context menu, pressing [CTRL] [SHIFT] [ESC], or [CTRL] [ALT] [DEL] then selecting Task Manager. The column names can be misleading. In the default view, it shows the mem usage column, which is a measure of only the physical amount of memory used by a process (Working Set). You also need to display VM Size, which is the process’s virtual private memory usage (and not the virtual memory size). Measurements do not necessarily reflect a reduction in the Java heap belonging to the Oracle client even though you can see a reduction in the Java console and a corresponding reduction in the commit charge (shown in Task Manager). Working Set can be overstated, as Windows does not necessarily trim the physical memory allocated to a process until it has a need to, or there is a period of inactivity. If Windows trims the Working Set, you cannot use this as a reliable measure of the true amount of memory (Private Virtual Bytes) that the program is using.

• Performance Monitor (perfmon) – Start from the command prompt, or navigate to Settings | Control Panel | Administrative Tools | Performance. This very comprehensive tool requires pre-configuring with an appropriate set of counters. It can be cumbersome and scaling graphs can be difficult. In general, it is better suited to tracking changes over time.

• Process Explorer (procexp) – Start from the command prompt, or the installed location. This is a free program available from Microsoft and is the tool that was used for all the measurements in this paper. It is quick and easy to configure, and there are support forums should you encounter issues. Documentation is minimal, but this tool was the easiest to configure, and provided the complete range of metrics throughout this paper.

Process Explorer is shown in Figure 2. The top pane shows detailed information about currently active processes. The lower window can be configured to show either the process handles that a process has opened, or (as in this example) the DLLs and memory-mapped files that the process has loaded.

The Process Explorer, a free tool from Microsoft, provides the Working Set and

Private Bytes used for the measurements throughout this paper.

The following steps were used to configure the tool to provide the measurements in this paper:

• In Options | Configure Highlighting: Uncheck Own Processes.

• Right-click either column set and select the corresponding columns in the Process Memory tab (top pane) and DLL tab (lower pane) sections.


Figure 2: Process Explorer

Process Explorer provides the Working Set (WS) and Private Bytes (PB) used for the measurements throughout this paper. The Peak Working Set specifies the maximum amount of physical memory that a process has been allocated since it started. During testing, the peaks only lasted a short time even on the slowest client, and there was no correlation with any major performance related Window events. The peaks were always within close proximity of the Working Set and can generally be ignored. WS Private shows the amount of memory exclusive to the process. The WS Shareable and WS shared columns break down the process’s non-private pages into what can potentially be shared, and what is actually being shared by at least one other process.

As a point of interest, on a PC with plenty of memory, Working Set is usually larger than Private Bytes. If Private Bytes significantly exceed Working Set for the majority of applications that have never been minimized, you will benefit from running fewer applications simultaneously or additional memory. This introduces further important concepts that are discussed at length in the Process Trimming section.

Working Set is usually larger than Private Bytes on a machine with plenty of memory.

If Private Bytes exceed Working Set for the majority of applications that have never

been minimized, you will benefit from running fewer applications simultaneously

or additional memory. Notice that Figure 2 shows a very small Working Set for the Palm Desktop; this indicates that the application was minimized and the process has been trimmed. While the process only had 5MB of physical memory allocated, it was still using 20MB of private virtual memory (Private Bytes).

Double-click a process (or right-click and select Properties from the context menu) to show additional process information. The Performance Chart in Figure 3 provides a graph of the Private Bytes history, and although not the focus of this paper, the CPU and I/O. The Performance tab in Figure 4 shows detailed information about the process. This information provides significantly more detail. For example, it is possible to calculate the precise amount of physical RAM that is currently backed up in the paging file. This may be interesting if you were trying to size a paging file accurately, instead of simply using the standard approach of using minimum = maximum = 2 x physical memory. The columns displayed on the main screen provide adequate analytical information for the purposes of this paper.


Figure 3: Process Explorer Performance Graph

Figure 4: Process Explorer Performance Tab

Choosing Which Memory to Measure

As you will appreciate, there are several different ways to measure memory. Peak Working Set, Working Set, Private Bytes, WS Private and WS Shared are compared graphically in Figure 5. To be pedantic, the measurements include the initial amount of memory used by the browser, which is a pivotal topic discussed in The Browser Offset Memory section. As such, the chart should only be used to compare the different approaches to memory measurement. In order to make the comparison easier, the terminology is summarized in Table 3.

There are four ways to measure memory.

Figure 5: Comparing Memory Measurements


The least noteworthy is the sum of the amount of physical memory exclusive to the process (WS Private) and the amount shared with at least one other process (WS Shared). Although this is an absolute measure of the physical memory utilization, it is understated: instead of using WS Shared, it should use WS Shareable (as used by Working Set) to map all the pages used by the process.

If you decide to use Working Set, you also have to measure

Private Bytes as a control.

Working Set represents the pages in physical memory that are dedicated to a particular process. From a theoretical point of view, Working Set (effectively the sum of WS Private + WS Shareable) would seem to be the most accurate way to measure the size of a single process. With multiple applications, counting the entire Working Set for each process will over-count memory, as the shared component is included in each. Due to the way that Windows manages memory, you also need to measure Private Bytes to ensure that the Working Set has not been trimmed (which can happen as soon as an application is started).

You could argue that you only need to be concerned with the Peak Working Set as this is the most likely to cause problems. However, as mentioned in the previous section, these were very short duration and did not have a significant affect, even on the smallest of machines. Therefore, this does not represent the actual amount of memory being used. Furthermore, like Working Set, Peak Working Set may also be subject to process trimming on low-memory clients.

Private Bytes represents the private virtual memory that the program has been allocated and is used exclusively by the process; it can reside in physical memory or the page file. It is smaller than Working Set as it does not include the shared memory component, but could be considered more accurate if the shared memory components were already shared by the other active applications.

Private Bytes is more accurate than Working Set on a low-memory client.

At this stage, the difference between Working Set and Private Bytes looks very small, and you might argue that you could use either of these approaches. Each has its problems and both have potential inaccuracies. It appears that you could quote a maximum safe value (Working Set) or more accurate (Private Bytes), and for slightly better accuracy add the amount of shared memory. In order to understand more about the relationship between these two figures you need to understand what happens when the Windows memory manager trims a process.

This paper focuses on both Private Bytes and Working Set.


Process

Explorer

Task

Manager

Performance

Monitor

Description Comments

Virtual Size N/A Virtual Bytes A process’s virtual address space consists of free, reserved, and committed memory pages. Reserving address space costs nothing in terms of memory consumption but allows for future allocations from a contiguous area of memory.

This is far higher than the physical requirements. It does not represent the real amount of memory that the program is using.

Private Bytes

VM Size Process | Private Bytes

This is the private virtual memory that the process has been allocated. It is committed memory that can reside in physical memory or page file and is protected and inaccessible by any other process. The difference between Private Bytes and WS Private is the total that is currently not in memory but is backed by the page file.

This may undercount memory as it omits the shared memory that the process is using. However, it is accurate when shared memory is minimal. It is the best value to use once a process has been minimized, or on a machine with low memory where the process may be immediately trimmed.

Working Set Mem Usage

Process | Working Set

These pages are currently resident in physical memory and dedicated to a specific process. It is effectively the sum of Process Explorer WS Private + WS Shareable. The peak Working Set is the maximum amount of physical memory used since the process was started.

This value can be larger than the minimum number of bytes actually needed by the process, as it includes the shared memory for each application. It may artificially reduce as you open more programs (due to trimming). The accuracy decreases when several processes involved in the total.

WS Private N/A N/A This is the amount of physical mapped memory used by, and only accessible by, that process. Other processes cannot share it. This is the same as Private Bytes except all referenced pages are currently resident in physical memory as opposed to both physical memory and page file.

This cannot be used in isolation, as it does not include the shared memory component.

WS Shareable and WS Shared

N/A N/A WS Shareable is the amount of memory in a process’s Working Set that can be shared with other processes. WS Shared is the subset of the shareable memory that is shared with at least one other process.

Use this to determine the amount of shared memory that is in use/may be used by the process Working Set.

Table 3: Mapping Task Manager, Performance Monitor, and Process Explorer


Process Trimming Understanding process trimming is essential as it explains why the ratio of Private Bytes to Working Set radically changes. A process’s Working Set may be trimmed with only a single Oracle E-Business Suite form open on a PC client with only 1GB RAM; if this happens, it will skew your results, hence the need to monitor Private Bytes.

Working Set may be trimmed with a only single Oracle E-Business Suite form or screen on a client with only 1GB RAM.

If the Working Set of an existing process increases, or you start another application and there is enough physical memory available, it is simply assigned from the available pool of free memory without affecting other processes. However, if insufficient free memory is available, Windows will need to trim (displace) memory pages from other processes. Unlike unmodified memory pages that can be reused immediately, modified pages must to be written to the page file before they can be reused. Paging large numbers of modified memory pages due to insufficient physical memory or too many applications running simultaneously will result in disk thrashing.

Once a process has been trimmed and the Working Set dramatically reduced, it can take several seconds for an application to recover and become responsive again. It would be natural (but incorrect) to think that Oracle E-Business Suite was the cause of the slowdown, when in fact Windows was responsible. To ensure optimal performance at all times, users should never minimize an Oracle E-Business Suite component.

Users should never minimize an Oracle E-Business Suite form or screen.

If you accidentally minimize a form in a test, you will have to restart the test.

The Performance Monitor chart in Figure 6 shows the relationship between Working Set and Private Bytes for the Oracle E-Business Suite 12 Purchase Order form using Internet Explorer 7 on a client with 2GB memory (effectively unrestricted and not subject to immediate trimming). The chart shows the process from starting the browser, logging in, selecting the purchasing menu, and opening the form. Up to this point, Working Set exceeds Private Bytes as expected.

Minimizing the form results in a substantial reduction of the Working Set, but (importantly) Private Bytes remains static. When free memory falls below a threshold, pages are trimmed from the Working Set. While the exact memory figures are not relevant, minimizing all browser windows reduces the Working Set to 14% of the original.

Minimizing an application radically reduces the Working Set, but Private Bytes

remains the same.

Restoring the browser window containing the form and running an open query proves that the application is functioning, but at this point, it is only using 40% of the original allocation. Moving and maximizing all the browser windows increases the Working Set, but this only reached 70% of the original, even when subsequent queries were run. However, minimizing and then later maximizing a form that contains a multi-record block causes a loss of mouse focus within that block. It is therefore advisable not to use the minimize form option until the resolution of Oracle bug 6041461.


Logi

n Sc

reen

Pu

rcha

sing

Men

u

Stra

rt Pu

rcha

se O

rder

Min

imiz

e al

l W

indo

ws

Form is Displayed

Restore the Form

Run an Open Query

Maximize the Form Window

Maximize all Explorer Windows

Minimize the Form

Star

t Bro

wse

r Run further Queries

Private Bytes Remains the Same

Figure 6: Process Trimming -Working Set and Private Bytes Compared

Rules of Thumb – Private Bytes and Working Set As previously mentioned, Windows trimming can be very aggressive. Sometimes you may find it difficult to understand why trimming is taking place when you appear to have ample free memory. The process is discussed in detail in the Process Trimming and the Memory Stacks section of Appendix C.

Once trimmed, a process’s Working Set is understated as it only counts physical memory and does not include memory paged to disk; therefore, you also have to monitor Private Bytes. This leads to the following rules:

Once trimmed, a process’s Working Set is understated, as it does not include

memory paged to disk.

• On a client with plenty of available memory, use Private Bytes to check that Working Set has not been trimmed, as it is not affected by the Windows memory management process.

On a client where the Working Set has been trimmed (perhaps as soon as the application is started), you have to use

Private Bytes, and for better accuracy, add the amount of shared memory.

• On a client with little available memory, Windows will trim the Working Set physical memory for use by other applications, perhaps as soon as it is started. In this case, you have to use Private Bytes instead, and for better accuracy, add the amount of shared memory.


UNDERSTANDING THE BROWSER MEMORY OFFSET AND SCALING RESULTS During the tests, the browser was set to display a blank “Home Page” by default. The browser offset (initial amount of memory) varied between 9MB and 36MB, depending on the browser/client combination. The variation of more than 25MB between the highest and lowest Working Sets needs to be normalized; failing to consider this would skew and invalidate comparisons across browsers.

Failing to normalize the browser offset would skew and invalidate comparisons

across browsers.

Figure 7 shows the measurements for private bytes and working set for each of the three browsers across the five test machines, using JRE with Oracle E-Business Suite 12. The chart is color-coded. Each browser is assigned a different color using two shades: blue for IE6, yellow for IE7, and orange for FF2.

Figure 7: Browser Memory As Measured For Oracle E-Business Suite 12 Using JRE

There is a lot of detail in this chart. Notice that the browser footprint on the first two low-specification machines is very small; this may be due to a more radical approach to memory management, effectively resulting in immediate process trimming. When IE7 is installed over IE6, it integrates all the same browser extensions used by its predecessor, and the larger footprint relates to an increase in functionality.

Firefox has the highest memory footprint on the 1.83GHz Core Duo,

but this includes several plug-ins and helper applications.

A first glance at these results shows that Firefox has the highest memory footprint (on the 1.83GHz Core Duo), but this is misleading as this particular browser includes several plug-ins and helper applications that you may not find in a normal business environment. A more accurate comparison for the basic Firefox browser is the 23.6MB on the Intel 930 Core Duo, which was a fresh install and this is noticeably less than Internet Explorer 7 (30.4MB).


The initial intent was to reduce the browser memory profile to a minimum and ensure consistency by using an identical Windows build. The high impact changes listed in Appendix B were necessary for the smallest clients and for consistency were applied to all environments. Although the changes improved performance, none significantly influenced the browser’s footprint. While this is clearly unrelated to Oracle E-Business Suite, establishing the reasons for the difference in browser behavior was the first mystery that needed to be solved in these tests.

Understanding the browser offset identified additional optimization

opportunities for low-memory clients that would have been missed.

How to Map Integrated Browser Components The Process Explorer shown in Figure 2 lists the DLLs referenced by the browser. Sorting by WS Total will show the highest memory components first. Not only is it sometimes difficult to correlate DLLs to a specific browser component, it can be difficult to differentiate between core browser components and those associated with additional features. Some can be intuitively mapped to a plug-in or helper application, while others can be mapped from within the browser.

Identify the highest memory utilization components using Process Explorer and

then map obscure DLLs from within the browser.

In Firefox enter about:plugins in the URL; in Internet Explorer 6, this information is available in the Tools menu; in Internet Explorer 7 use the Manage Add-ons screen. An example of the Manage Add-ons screen is shown in Figure 8. The display categories include Add-ons that have been used, those currently loaded, those run without requiring permission and those that load when Internet Explorer Starts (This final option was available at the start of the tests, but appears to have been removed, possibly by a Windows update).

Figure 8: Internet Explorer 7 Manage Add-ons Screen

Adobe Acrobat Reader, Skype, and the Google Toolbar

Adobe Acrobat Reader, Skype, and the Google Toolbar were installed in each of the browsers and their memory profiles compared in Figure 9. The Internet Explorer 7 memory utilization increased much more than the other browsers even though the version and size of the utilities was identical.


Figure 9: Comparing Browser Add-On Memory Profiles

Table 4 shows an abbreviated export from Process Explorer for Internet Explorer 7 with Adobe Reader, Skype, and the Google Toolbar. Note that some utilities will not appear in the DLL listing until they have been used at least once.

The largest components are the Google Toolbar and the

Microsoft Phishing Filter Data File.

Process PID Working Set Private Bytes

iexplore.exe 4680 39,900KB 28,612KB

Name Description WS Total WS Private

IEFRAME.dll Internet Explorer 3008KB 2328KB

googletoolbar Google IE Client 2560KB 2328KB

ieapfltr.dat MS Phishing Filter Data File 2392KB 0KB

mshtml.dll MS HTML Viewer 2260KB 40KB

SPhoneParser Skype 1756KB 576KB

SKypelEPlugin Skype 1168KB 740KB

shell32.cll Windows Shell Common 476KB 36KB

Total 24,588KB

Table 4: A Sorted Extract from Process Explorer

In this particular example, the WS Total for the mapped components account for approximately 24MB out of the total 39MB Working Set, which is considerably less than the Working Set Size. A part of the reason is that, for example, Adobe Reader (AcroIEHelper.dll) does not appear in the list unless you have a PDF file open in the browser.


Microsoft Phishing Option

Table 4 identifies the Microsoft Phishing option as one of the largest components. This can be disabled and the results for the Internet Explorer 7 memory footprint are shown in Table 5.

Mapping add-ons and plug-ins can be difficult and instead it may be easier to measure the browser memory footprint

before and after you integrate the features.

Option Working Set Private Bytes

On 39,900KB 28,612KB

Disabled 36,416KB 24,736KB

Difference 3,484KB 3,876KB

Table 5: Exploring the Microsoft Phishing Option

Given that the Phishing data file (ieapfltr.dat) has a WS Total of 2,392KB, disabling this option shows a Working Set reduction of 3,484KB, which indicates that some other associated components were not loaded. While disabling this function on an Intranet may be useful, this highlights a possible requirement to review proposed changes with a wider audience (such as network security in this case).

Disabling every add-on resulted in an unexpectedly small reduction of the browser’s Working Set by approximately 7MB. For IT professionals wanting to take the analysis to the next stage, the browser startup events can be captured using Process Monitor (also free from Microsoft). These can be filtered using the browser Process IDentifier (PID), which is shown in Process Explorer. There are in excess of 30,000 events for Internet Explorer 7, but again, correlation to the DLLs, drivers, and data files is not easy. Trying to normalize the browser footprints was abandoned after extensive analysis.

Disabling all add-ons did not reduce the browser memory as much as expected and

attempting to normalize the browser footprints was abandoned.

Factors That Affect the Browser Memory Footprint Normally the Internet can supply a plethora of information, but there is very little discussion on the amount of memory used by browser components. The following list of components was compiled throughout the investigation; some aspects could be construed as obvious while others are more obscure:

Avoid integrating browser components that are not essential to the business.

• Figures 7 and 9 show that Internet Explorer 6 has the smallest browser footprint, and as will be shown, the smallest average Working Set across the Oracle E-Business Suite tests in this paper.

• The browser footprint may be automatically trimmed if there is little available memory. This can be affected not only by applications that are automatically started with Windows, but also by background services.

• A significant amount of memory can be used by browser themes, extensions, plug-ins and helper applications such as Adobe Acrobat, Adobe Shockwave, Adobe Flash Player, Apple QuickTime, RealNetworks RealPlayer, Skype, and password managers.

• Toolbars can use several megabytes of memory.

• Codecs (to support multimedia content) are generally not required.


• Phishing controls are probably not required on secure Intranets with restricted or no Internet access

• The download history uses memory and should be regularly cleared. This seemingly insignificant issue has been reported as an issue with Firefox.

• Windows components including the number of language packs and multi-lingual support, all use memory. Avoid the installation of unnecessary language packs, even if you use a corporate image.

• Avoiding other integrated components is more difficult, such as MIDI and audio support, the Remote Access connection manager and associated APIs and the Windows Telephony client. The combination depends on the type of client and device drivers, but older machines tend to have more drivers that are less integrated with Windows (and less shareable/shared memory).

Overall, when designing a system for low-memory clients, the goal is judicious removal of high-memory non-essential components that needlessly add to the browser memory footprint.

Deciding How To Scale Results Deciding which method to use to scale

results is critical. As different browsers have different offsets on each client, it is important to find an approach that can be used to scale the results so that they can be compared. Deciding which method to use to scale results was found to be as involved as trying to normalize the browser footprints.

Baseline the Browser Memory Offset

Referring to the discussion around Figure 7, the most reliable measurements for the browser footprints are the fresh installs on the Intel 930 Core Duo PC. Rounding these figures keeps the calculations simple without introducing significant errors. Table 6 shows the resulting offsets that have been used as a basis to scale all the results.

Table 6 shows the offsets that have been used as a basis to scale all the results.

Browser IE6 IE7 FF2

Working Set 20MB 30MB 25MB

Table 6: Standard Memory Offsets

Decide Which Scaled Approach to Use

The approaches considered to scale the results are as follows:

• Difference the Results Using an Average Browser Footprint – Adjust each result by adding or subtracting the difference between the actual browser offset and the datum provided in Table 6. This approach worked remarkably well until it was noticed that the ratio of Private Bytes to Working Set was being skewed for particular tests.

• Exclude the Browser Offset – Deducting the initial browser footprint means that you could simply add your browser offset to scale the results for your environment. However, the results are skewed by the differences shown in Table 6, thereby reducing differences between the browsers.


• Scale Results based on the Static Ratio of Browser Offsets – This approach resulted in a dramatic increase in Private Bytes for both IE7 and FF2 as the ratio of Private Bytes and Working Set is not static and varies by the type of Oracle E-Business Suite component.

• Scaled Results Using a Variable Ratio – This proved to be the most accurate method, but you have to use variable scaling as follows: The variable scaling ratio is the

most accurate.

o Use a scaling factor based on the browser offset from Table 6 to calculate the new Working Set

o Establish the percentage difference for the original Working Set and Private Bytes

o Use the percentage to calculate the new Private Bytes value from the new Working Set

CAUTION! You cannot use this approach on a client running Windows XP SP2 with less than 2GB memory otherwise you will find that the Windows memory manager may skew the results due to aggressive trimming. In this case, you will have to use Private Bytes as the key, but this may be less accurate (depending on the shared memory).

Figure 10 shows a graphic comparison and visual check of all the average memory used by each method for the full range of tests using Oracle E-Business Suite 12 with JRE. Firefox included several additional components and including it shows how well each method worked.

Figure 10: Verifying the Choice of Scaling Method

This chart shows that excluding the offset reduces the differences between the browsers, but this is misleading as this incorrectly indicates that Firefox has the highest footprint. Scaling using a static ratio dramatically changes the ratio of Private Bytes and Working Set for IE7 and Firefox. The scaled results using a variable ratio follow the same pattern as the extrapolated averages in Table 6; the Firefox browser with the additional components has been fully normalized and the ratio of Private Bytes and Working Set for all three browsers is verifiable.


Figure 10 implies that Internet Explorer 6 remains the browser with the smallest initial memory footprint. You might infer that the other browsers are less suitable for low-memory clients and although the difference of a few megabytes is very small, further research shows that this decision is not so clear, as the profile changes depending on the type of application used. This highlights a common problem with mixed benchmarks, and demonstrates that while using a more granular approach is sometimes beneficial, it does not eliminate the need to perform your own tests with your own particular combination of applications.

ORACLE E-BUSINESS SUITE 12 – THE RESULTS The first half of this paper includes a full testing methodology that can be deployed in your environment and describes how to accurately scale the results when working with different browsers across a range of clients.

The key points that need to be understood for successful interpretation of the results includes the types of memory that need to be measured (Working Set and Private Bytes), the ratio between them, why they cannot be used in isolation, and what happens when an application is minimized or the process is trimmed by the Windows memory manager.

As a recap, some of the key points are as follows:

• Working Set and Private Bytes are both important and should not be used in isolation.

o Working Set is a measure of the combination of physical private and shared memory pages that are currently resident in physical memory and dedicated to a specific process. When measuring multiple applications, it can be overstated as it includes the shared memory component.

o Private Bytes is a measure of the virtual private memory allocated to the process. It can reside in physical memory or the page file, but does not include shared memory and so may undercount the amount of memory being used.

• Minimizing or trimming a process results in a substantial reduction of the Working Set, but importantly Private Bytes remains static. Once trimmed, a process’s Working Set is understated as it only counts physical memory and does not include memory paged to disk.

On a client where the Working Set has been trimmed (perhaps as soon as the application is started), you have to use

Private Bytes, and for better accuracy, add the amount of shared memory.

The following rules were established:

• On a client with plenty of available memory, use Private Bytes to check that Working Set has not been trimmed, as it is not affected by the Windows memory management process.

• On a client with little available memory, Windows will trim the Working Set physical memory for use by other applications, perhaps as soon as it is started. In this case, you have to use Private Bytes instead, and for better accuracy, add the amount of shared memory


Unlike Oracle E-Business Suite 11i, Release 12 does not use Oracle JInitiator for its Forms-based content. Instead, Oracle Forms runs directly with the native Sun Java2 Standard Edition Java Runtime Engine 1.5.0_12. For further information, refer to Steven Chan’s blog http://blogs.oracle.com/schan/release12.

All the measurements in this paper assume that you have used the standard menus as installed and not used shortcuts that would inevitably change the memory utilization profile. Recall also that the browser was restarted for technical reasons between each test instead of using the change responsibility function.

Although this paper focuses on low-memory clients, you will find that as Windows takes a very aggressive approach to trimming process memory, using a client with less than 2GB of memory will skew your results. For this reason, the amount of client memory is important and needs to be considered in each set of results. Note that the amount of memory used by Process Explorer and Excel is relatively small, and therefore should not affect the tests.

Comparing Browsers and Oracle E-Business Suite Components Figure 11 shows the Private Bytes and Working Set for all three browsers on the 1.83GHz 2GB RAM client for the full range of tests listed in Appendix A.

Rather than chart the functions in the order that they were tested, they were consolidated into groups. Aggregating the functions in this manner (rather than by product area or business flow) provides a much clearer picture and highlights several points that might have otherwise gone unnoticed. The dashed lines denote each section in the chart. The first section shows the browser and login screen followed by menus, then by OAF/HTML screens, Oracle Forms and finally Gantt Charts. Note that the last form, the Supply Chain Planners Workbench Form is a necessary precursor to its associated Gantt chart. The last two sets of results are the same Gantt chart, without and with data. It is important to bear in mind that the amount of data in the Vision database is limited, but there is enough to show a corresponding variation.

Sequencing charts by type brings highlights anomalies and patterns are

easier to notice.


http://blogs.oracle.com/schan/release12

Figure 11: Working Set and Private Bytes for the Full Range of Tests


While the chart may or may not show the exact screens that you use, generalizations provide guidelines for rough estimates. There are several distinct levels in the chart, which form the basis for the following generalizations:

While it is possible to generalize, the amount of data that is displayed may

dramatically increases the memory profile, especially on large Gantt charts. • Menus: The amount of memory varies depending on the number

of items. It is interesting that Firefox 2 has the poorest footprint when used with large menus. As well as being a reason to keep menus small, this is important enough to warrant more investigation, and is discussed in the Comparing Menus section.

• OAF/HTML Screens: Most of these have the same memory profile peaking at around 40MB Working Set (excluding Dashboards) with Internet Explorer (which also has the highest memory footprint). Although self-evident, if you have memory constrained clients, using OAF/HTML screens is preferable to forms as they have a smaller footprint. This is something that you may want to consider when creating your own custom or bespoke applications.

• Dashboards: Although a part of the OAF/HTML screens group, these differ fundamentally, as they render several graphs and charts that have no associated data. As these graphs and charts are rendered as graphics, the memory footprint is unlikely to change regardless of the amount of data. It is notable that IE7 has a substantially higher footprint.

• Forms: The first four forms have an almost identical footprint, which is not surprising as they all use the same JAR files. The forms between the Contact Centre and Customer Interaction History are also very similar, although they tend to use different sets of JAR files. The two exceptions in this group, which both have significantly higher memory profiles as they are larger, are the Item Costs and Order Entry workbenches.

• Gantt Charts: These have the highest memory footprint, and again, are all very similar. The last two sets of figures show that the amount of data being displayed does make a difference to the amount of memory that they require. While the Vision database does not contain much data, some customers have noticed exceptionally high memory requirements with these charts and therefore you may need to consult users to establish how much data they regularly display on screen. If there is a performance issue, then advise that reducing the amount of data (if possible) will bring temporary respite while you determine the amount of memory that they require.


Average Memory by Oracle E-Business Suite Component Type If you use a range of products, then the generalized benchmark figures such as the browser averages for the full suite of tests shown in Figure 12 may be more useful. If your Oracle E-Business Suite usage can be broadly categorized into OAF/HTML, forms, or Gantt charts, it may be useful to focus specifically on that area (bearing in mind that there may be exceptions). This chart shows the average memory utilization (Private Bytes and Working Set) grouped by type for Oracle E-Business Suite 12 using browsers with JRE across all of the tests on the 1.83GHz 2GB client.

Figure 12: Average Memory Utilization for Oracle E-Business Suite 12 Using JRE

Grouped by Type

To be more precise, you need to consider this chart and the amount of memory that each browser shares with the other applications that users typically run simultaneously. The amounts in this test as shown in Figure 1 are IE7: 9.6MB, IE6: 8MB, and FF2: 6.2MB. These relatively small differences indicate that Internet Explorer 6 still appears to be the best choice for memory-constrained clients. Some of these figures will be affected by the amount of data (such as Gantt charts, customizations, and form folders) and it is important to note that these figures do not scale linearly when using multiple forms or screens at the same time.

If a browser uses more memory, the overall profile may reduce when

considering the amount of shared memory.

Reviewing the Ratio of Working Set and Private Bytes The variation in Private Bytes to Working Set ranges between 5% and 30% for the full set of tests. Examples of a very small difference include the Customer Interaction History and Supply Chain Planners Workbench Form, with Firefox, and the Payables, General Ledger and Project Forms with Internet Explorer 7. The original results are shown in Figure 11. When the tests were repeated, the ratio increased, as did the Working Set (marginally). As there was no change in the other applications being used, you could infer that the frequency of tests resulted in the Windows memory manager aggressively trimming memory.


It follows that if you restrict your measurements to Working Set in isolation, then depending on the other applications that you use on your client, you could have anomalous readings that would otherwise go unnoticed, even on a client with 2GB. As noted in The Test Environment And Testing Issues section, it was surprising how few applications it took to change the memory model. For this reason, you always need to compare Working Set with Private Bytes to provide a check point that will ensure that process memory is not being trimmed without your knowledge.

Looking more closely at Figure 11, a very specific pattern of the ratio of Working Set and Private Bytes emerges, as shown in Figure 13. The percentage difference is substantially larger during the first half of the chart (login and OAF/HTML) as compared to forms, and then there is another smaller step change to the Gantt charts. Both forms and the Gantt charts have a substantially higher memory footprint. This either implies that the Windows Memory Manager was trimming to a greater degree, thus dropping Working Set while Private Bytes stayed the same, or perhaps that the different set of Forms used fewer shared memory objects between them, thereby raising Private Bytes while Working Set stayed the same.

Using different views of the same set of results sometimes helps to validate an

investigation.

Figure 13: The Relationship between Private Bytes and Working Set

If you think it unlikely that Windows was trimming the process, you need to consider other factors that may cause this behavior. For example, the amount of shared memory, or some other memory structure, could vary between the different types of forms and screens. As all browsers exhibit the same step changes, any browser will show this effect. The amount of shared memory is determined by the set of applications that are running simultaneously, and measurements from Internet Explorer 7 are shown in Table 7. This shows that the difference between the functions is minimal, in which case shared memory cannot be the reason.


Function Memory (KB)

Browser 9708

Login Screen 12344

OAF/HTML Screens 12444

Menu 12998

Oracle Forms 13400

Gantt Charts 13588

Table 7: Shared Memory by Product Group

Comparing Menu Memory Utilization The memory utilization and the number of items in each menu are compared in Figure 14. The Main (Choose Responsibility) menu – (110 items) and Payables menus (280 items) have relatively few items and display quickly. The final three menus are substantially larger. Enterprise Asset Manager Menu (1200 items), Manufacturing and Distribution Manager Menu (2650 items) and Order Management Super User Menu (1650 items) take substantially longer and have a larger memory footprint.

Keep menus small especially in low bandwidth situations

(latency is not as important).

Figure 14: Menu Memory Utilization

Trimming a large menu results in a reduction of the memory utilization from 63MB to 27MB; this represents a saving of almost 50%. With menus in particular, bandwidth is the primary network constraint; latency has a much lower affect. For example, for all latencies from 50ms to 300ms, the Manufacturing and Distribution Manager Menu takes almost a minute to display on a network with only 256Kbps available bandwidth, and almost two minutes with 128Kbps available.


Clearly, the memory utilization and bandwidth concerns are sufficient reason to keep menus small whether memory-constrained, or working over a high latency or low-bandwidth connection. This is true even though the memory utilization for multiple forms and screens (as used in a complete business flow) does not scale linearly.

Firefox exhibits a high memory utilization compared to Internet Explorer, which appears to scale more linearly in proportion to the number of items being displayed. However, keeping menus small will reduce this difference between the browsers. Some customers combine menus under a single responsibility, so they can take advantage of being able to keep multiple forms open. If this is not something you have considered, the only complication that you need be concerned with is profile option precedence (user, responsibility, application, site) and ensure that the profile options are set at user level (and not defaulting from the responsibility level).

As the time to display a menu was identified and logged as a result of these tests, you will need to review Oracle bug 6329486 for possible patches. Review Oracle bug 6329486 for possible

patches.

Comparing Memory Profiles across a Range of Clients This test compares performance across four of the PC clients detailed in Table 2 that represent the range of memory being considered (128MB, 256MB, 1GB, and 2GB). Two salient factors are that the CPU and technology inevitably varies across the machines (which is not relevant for this test) and that 128MB is below the minimum Windows and Oracle specification. The software on the three smaller machines was kept to a minimum of essentially Oracle E-Business Suite, Process Explorer, ZoneAlarm, Norton AntiVirus, and the default set of background processes. Tuning the Windows environment did not significantly affect the memory profile. A handful of standard Windows tuning practices were employed to improve performance (as listed in Appendix B), but all of them failed to significantly reduce or stabilize the browser memory footprint.

Figure 15 shows the Working Set for the full range of tests and color-coordinated callouts containing selected sections of Private Bytes, which can be easily mapped back to the original client using the colors and/or machine specification. Including full graphs for Private Bytes made the chart overly complex and hid some of the detail.

The callouts show the Private Bytes (same color as the main chart) and Working Set (shown in red) for each of the four clients

As indicated by the browser offset shown as the first measurement in the chart, the initial browser offset was not compensated for on the two smallest machines with 128MB and 256MB respectively. Both of these small clients have an almost identical memory profile, though it took a very long time for the Gantt charts to be rendered and the stability of the undersized (128MB) client was questionable due to the amount of thrashing observed. The 1GB client starts by following the same coordinates as the 2GB machine, until the memory requirements suddenly increase with forms and Gantt charts. At this point the Working Set on the client with 1GB is less than on the client with 2GB. Recalling that working set is the physical memory pages that are dedicated to a specific process, the somewhat surprising conclusion is that Windows must be aggressively trimming the process on the client with 1GB. Although the pages will most likely be in memory (on the standby or modified list), there is still a finite cost in recovering and making them available to the parent process when referenced again.


Figure 15: Oracle E-Business Suite 12 Memory Profiles across a Range of Clients

The callout on the left shows a sample of the Working Set and Private Bytes for the 1.6GHz 1GB client. The ratio between these measurements appears exceptionally consistent. Although they fluctuate through the remainder of the tests, they follow the expected pattern (Working Set greater than Private Bytes) for all clients, until the final stage of the tests containing the Gantt Charts. At this point, Private Bytes (shown in red) exceed Working Set for all the clients with less than 2GB. Each of the callouts on the right of the chart shows the Private Bytes and Working Set for the WIP Discrete Job Workbench.

This represents a fundamental change in the expected profile, and is typically one area where inadequate client memory results in performance problems and consequently complaints from unhappy users. The worst profile and greatest difference between the sets appears on 700MHz 256MB client, though to be fair, the 128MB client was really struggling for memory and so you would not expect anything sensible from the memory figures. As you would expect, the 366MHz 128MB client also uses old technology, and so paging and other disk activity would be slower than on the other clients.

Windows was trimming processes on a 1GB machine even when all running applications were using less than 160MB. It is worth stressing again that if private bytes exceeds working set for the majority of applications, either the client is running too many applications, or (if all the applications are needed) you need more memory.

If Private Bytes significantly exceed Working Set for the majority of

applications that have never been minimized, run fewer applications

simultaneously or add memory. Comparing Load Times for across the Range of Clients Unlike OAF/HTML screens and menus, E-Business Suite 11i and 12 forms and Gantt Charts will download the necessary JAR files to the local client when a form is accessed the first time. For example, using JRE 1.5.0_12 the Purchase Order form downloads 10.3MB of JAR files, whereas a Gantt chart may require 19.3MB of JAR files (the exact figures vary with the functions in use). Once JAR files have been downloaded, they remain valid for the browser session. They are revalidated if you restart the session, but not if you switch forms or responsibilities.


Comparing Load Times across Fast Clients

The Oracle E-Business Suite load times were compared for the same four clients used in Figure 15, for the full range of tests excluding the measurement for the WIP Discrete Job Workbench with no data. Again, reviewing all the data in one graph is cluttered, and therefore a range of graphs have been used. When reviewing the charts, note that the Y-axis scale has been changed in order to highlight key elements.

Figure 16 compares the initial and subsequent load times for two fastest clients. Remember that JAR files need to be downloaded the first time they are used, and the download time will be influenced by network characteristics such as latency and available bandwidth. In high-latency situations, using a local proxy server will help to alleviate network problems and improve download times. The client specification is listed in Table 2. Although CPU benchmarks are not very representative, the 1.8GHz Mobile Core is allegedly equal to a 3.43GHz Pentium (non Intel sources), though this has not been verified.

Figure 16: Comparing Initial and Subsequent Load Times

Accepting a small margin of error, the times for each of the clients for the initial login and menu are comparable for each of the clients (menus do not use JAR files). The times to load the large menus are relatively long, even though these are the fastest clients in the test and they are being loaded over a low latency LAN.

The difference in client speed is very clear in the forms and Gantt chart sections. As expected, the fastest machine (which usually commands a price premium) provides the best response times. Downloading the JAR files is generally a one-off operation and most of the time you will be working with locally cached (downloaded) JAR files. The relative performance of the two fastest clients is very similar, bearing in mind that both machines are very lightly loaded in terms of the number of applications that are running.


There is only about 2 seconds difference in response times between the two fastest clients even though they are a generation apart in technology terms, and one will cost more than twice as much as the other.

The fastest client that commands (which a price premium) is not significantly faster,

and the evidence suggests that CPU definitely less important than memory.

You can calculate the amount of time spent validating the JAR files by subtracting the validated and first use figures. Notice that the graphs converge at two points in the forms section. The reason is that in order to get to these forms, you have to navigate through other forms, so the previous business flow validated the JAR files.

Comparing Reload Times

Figure 17 compares the load times for forms that were previously used in the same session. This is more representative of normal user operation (with cached and validated JAR files) where users will iteratively cycle through the same set of forms. Notice that the vertical scale has been changed to 100 seconds, which is a very long time to sit waiting for a screen update. The 366MHz actually takes 172 seconds for the final Gantt chart (Supply Chain Planners workbench) with the majority of the time spent paging memory; the machine ended up in an almost unusable state.

Changing the scale of your graphs may highlight features that are otherwise

difficult to notice.

Figure 17: Comparing Load Times for Forms Previously Used in the Same Session

Both of the smaller clients perform very poorly when rendering large menus and the graphics-rich dashboards. While this chart provides an overall picture of what is happening, expanding the Y-axis scale to 30 seconds, as in Figure 18 really highlights the differences.

While technology does not generally scale linearly, the 366MHz is approximately twice as slow as the 700MHz. The 366MHz appears to work almost as well as the rest of the machines for small OAF/HTML forms and there is little difference between the 700MHz and 1.6GHz, and not as much difference as you might expect between the 1.83GHz Mobile Core (with a dual processor) and the 1.6GHz Pentium.

The slow clients almost match the fastest machines for OAF/HTML functions.


Remember that the server used for these tests is very small and is not as fast as you would expect in a production system. For example, if your target time for opening the Purchase Order Workbench is five seconds, then you have some clear information on which you can make comparisons, perhaps as a basis for allocating newer equipment to customer-facing staff.

Figure 18: Comparing Load Times for Forms Previously Used in the Same Session

(expanded)

Although the first use times (without JAR files on the client) have not been included for the 700MHz client, it is reasonable to conclude that they are substantially longer than the 1.6GHz client. When you compare the measurements across Figures 17 and 19, you can see that when the forms have been used once in the session, that the times to reopen the form are similar. One approach to extend the working life of older technology is to have call centre staff start the necessary Oracle E-Business Suite forms prior to starting their work shift, so that the JAR files will be validated before using the forms in earnest when interacting with customers.

Comparing Load Times and Jar File Volumes

As discussed in the previous section, when a form is used for the first time, JAR files are downloaded to the client. If they have previously been downloaded, they are validated. Figure 19 shows the times for each of these events for Oracle E-Business Suite forms using JRE 1.5.0_12 with Internet Explorer 6 with the corresponding JAR file volume (MB). In order to ensure consistency, the local JAR cache was cleared between each test, with the exception of forms that were only accessible through other forms. JAR files are compressed and are transferred over the network before being uncompressed saving around 50% of the bandwidth.

JAR files are stored in compressed format and are transferred over the network

before being uncompressed saving more than 50% of the bandwidth.


Figure 19: Comparing Form Opening Times and JAR File Volumes

This chart only includes the forms and Gantt charts, as these are the only components that require JAR files. Although these components represent a subset of the Oracle E-Business Suite, the sample should be reasonably representative. The three lines representing the downloaded time, validation time, and volume of JAR files (either downloaded or validated) follow a similar pattern. It is evident that the form opening times are roughly proportional to the volume of JAR files that need to be downloaded. As mentioned previously, converging time lines indicate that the JAR files have already been downloaded and validated.

Form opening times are roughly proportional to the volume of JAR files that

need to be downloaded and validated.

The speed of JAR file revalidation depends on a number of factors including the application tier (or proxy server if applicable), network capabilities, and (to a lesser extent) the speed of the client. On the 1.83GHz test machine, revalidating the JAR files occurs at between 1-2 MB per second. As already mentioned, several forms share the same set of JAR files and so the first time usage timings (including the JAR file download or validation) will only apply to whichever form is used first in that particular set. You can easily compare the JAR files for each of the forms by reviewing the contents of the JAR file cache.

When JAR files are shared, the first time use timings only apply to whichever form

is used first in that particular set.

Network Traffic for Oracle E-Business Suite 12 with JRE The next logical step after discussing the JAR file download volumes is to review the effect on the network. In the example shown in this section, the JAR files have not been downloaded to the local client representing the worst-case scenario.

The Wireshark network protocol analyzer was used to capture the network information, which was then analyzed using Application Characterization Environment (ACE). This provides the ability to visualize transactions over and above what can be seen in a raw data capture, so that the characteristics of a transaction, such as including TCP behavior, are visible.


As the definition of an application turn (app turn) varies, in this context an app turn is said to occur when the flow of application messages changes direction, typically in a request-response sequence. High application turns (20+) indicate a chatty application, which will be negatively impacted by latencies occurring on a WAN. ACE also counts each TCP port connection initial handshake as an application turn.

The worst-case performance scenario for any customer occurs when JAR files are downloaded by a client. The amount of data, number of packets, and app turns for the first use are shown in Figure 20. Although JAR files only apply to forms, including the full range of tests enables comparisons between the types of Oracle E-Business Suite component. Restarting the browser and clearing the JAR and browser cache between each test ensured that the figures are correct for each of the forms.

The worst-case performance scenario for any customer occurs when JAR files are

downloaded by a client.

As expected, the Gantt charts have the highest network utilization; this will vary depending on how much data is transferred. The dashboards also have a relatively high memory profile transfer approximately 1MB, as compared to 50KB for the other OAF/HTML screens.

The exact volume of network traffic depends on how you access the forms and how you change responsibility. For example, network usage for the General Ledger Enter Journals form starts at 5MB on first time use with no local JAR files. This reduces to 200KB if you reselect it from the main menu, and reduces further to only 32KB if you switch between forms within the responsibility.

Use the access paths shown in Appendix A to match the results in this paper. Using

alternate routes may produce different results.

This principle does not only apply to forms. If you do not clear the browser cache and restart the session, the login network traffic reduces further from 240KB to 24KB. Note that as menus are not cached, the associated network download does not reduce - another reason for keeping menus small.

This follows the same approximate pattern as the form opening times shown in Figure 19. As you would expect, there is very little network traffic when secondary forms are opened. You can deduce that components with the highest network utilization will be most affected by high-latency and/or low bandwidth networks. As an aside, if network latency is a concern, The Holistic Approach to Performance Tuning Oracle Applications White Paper (although slightly dated and outside the scope of this paper) showed that CPU compensates for latency over a WAN. If latency and bandwidth are of concern, you may want to consider network compression and optimization.

It is important to emphasize that the total amount of network traffic varies significantly with the amount of data and new users running open queries stress every part of the technology in ways that had not been expected.

New users running open queries stress every part of the technology in ways that

had not been expected.


Figure 20: Network Traffic (First Use – worst case)


COMPARING ORACLE E-BUSINESS SUITE 11i AND 12 Customers using Oracle E-Business Suite 11i have a choice of continuing to use JInitiator or changing to the Sun Java2 Standard Edition Java Runtime Engine. If you are using Oracle JInitiator and plan to upgrade your environment the following documents may be useful:

• Oracle MetaLink Note: 124606.1 Upgrading Oracle JInitiator with Oracle Applications 11i.

• Oracle MetaLink Note: 290807.1 Upgrading Sun JRE (Native Plug-in) with Oracle Applications 11i for Windows Clients.

This section reviews the memory differences between Oracle E-Business Suite 11i and 12 and then explores the technology choices available with Oracle JInitiator 1.3.1.29 and Sun JRE 1.5.0_12. At the start of the tests, JInitiator 1.3.1.28 was used; by the end, 1.3.1.29 was certified and the tests were repeated. Although there was no noticeable change in the memory footprint, this may not be the case with earlier versions.

The Effect of Business Flows in Oracle E-Business Suite 11i and 12 As mentioned several times in this paper, the memory used by multiple forms and screens does not scale linearly. It is very unlikely that this paper could match the exact combination of forms and screens, client specification, and the specific combination of other software that you use. Instead, general guidelines have been provided that can be applied to your own environment as needed.

The memory used by multiple forms and screens does not scale linearly.

Attempting to find a formula that enabled the results in this paper to be scaled for multiple forms and screens was abandoned as it became overly complex. Ultimately, it is quicker to run the set of forms for your particular business flow and take the necessary measurements. Figure 21 shows that client memory does not scale linearly. It shows the memory profile of a single form (the Purchase Order Workbench), and compares that to a set of four forms that are simultaneously open. The utilization only increases by a few percent when the additional forms are opened indicating that much of the forms code and mapped files (such as DLLs and JAR files) are shared. The forms used for the test are shown in the chart, and none were minimized. The difference between the Working Set and Private Bytes reduces when several forms are opened, possibly indicating that Windows may be trimming based on the application activity and an increasing number of potential threads.


Figure 21: Comparing Form Combinations

Comparing Working Set for Oracle E-Business Suite 11i and 12 Figure 22 compares the Working Set requirements for Oracle E-Business Suite 11i and 12 using Internet Explorer 6. The comparison is limited, as some measurements would not have been directly comparable due to menu path changes between versions. If you want to perform the full range of tests, then you will need to restructure some of the menus and apply Oracle patch 4469342 to enable connections to the TCF Socket Server when using JRE for some of the functions. Alternatively, you could consider changing Oracle E-Business Suite 11i to use servlet mode.

The largest difference is approximately 2.5%, which could be considered within the bounds of experimental accuracy, especially given that measuring Windows memory is not an exact science.

The client memory is almost identical for E-Business Suite 11i and 12 for this subset

of tests. However, there are some documented differences between Gantt

charts.

Figure 22: Comparing Working Set for Oracle E-Business Suite 11i and 12


Although this chart only represents a subset of forms and screens, it appears the amount of client memory between E-Business Suite 11i and 12 can effectively be classed as identical, and therefore client memory should not be a concern when upgrading between the versions. There are probably some exceptions to this rule, and therefore you should always perform your own tests.

While Figure 22 was restricted to Working Set for clarity, Figure 23 compares the Private Bytes and Working Set for each release; this was included both in the interests of completeness, and to check whether there was a change in profile. Both versions are using JRE, and therefore as the Working Set is the same, you are looking for any anomalies. As with Oracle E-Business Suite 12, there are different levels for each type of product, but these are more difficult to distinguish on this chart. As before, the Customer Interaction History has a small Private Bytes value, which is unexpected and needs further research to find the cause.

Figure 23: Memory Profiles for Oracle E-Business Suite 11i and 12

Comparing Oracle JInitiator 1.3.1.29 with Sun JRE 1.5.0_12 Customers on Oracle E-Business Suite 11i have a choice of technologies. Figures 24a and 24b provide two different views of the same data that compare the Working Set (including the browser offset) for a subset of Forms on the Intel 930 Core Duo.

Figure 24a continues the theme of low-memory clients, and clearly shows that the combination of Internet Explorer 6 and Oracle JInitiator 1.3.1.29 has the smallest footprint. This follows the expected pattern that newer technology has more features and consequently a higher resource requirement. Figure 24b compares technologies by browser, and more clearly shows that JInitiator has a lower footprint than JRE. There are some clues to why this may be the case in the JInitiator \doc\changes.txt file. JInitiator 1.3.1.29 refers to a base of JRE 1.3.1_20, and again this will have smaller resource requirements than the later version of JRE 1.5.0_12.


Figure 24a: Comparing JInitiator 1.3.1.29 and JRE 1.5.0_12 by Technology

Figure 24b: Comparing JInitiator 1.3.1.29 and JRE

1.5.0_12 by Browser

The amount of shareable memory is the same for both of the technologies, which is what you would expect as they have a common base. For completeness, you need to consider the effect of the shared memory component (as shown in Figure 1). You would reduce the browser memory utilization by the following amounts: FF2 6.2 MB, IE7 9.6MB, and IE6 8MB. However, this fails to explain the difference. Bearing in mind that even though the tests span a wide range of Oracle E-Business Suite forms and screens, this is still just a subset of tests. However, all the evidence suggests that the best combination for low memory clients is IE6 with JInitiator for Oracle E-Business Suite 11i and IE6 with JRE for Oracle E-Business Suite 12.

All the evidence suggests that the best combination for low memory clients is

IE6 with JInitiator for Oracle E-Business Suite 11i and JRE with

IE6 for Oracle E-Business Suite 12.

RECOMMENDATIONS FOR LOW SPECIFICATION CLIENTS There are several approaches to help ensure the longevity of any low-specification machines that you have, while maintaining the necessary throughput to sustain your business. The overall approach is to configure low-specification clients using the technologies that provide the best performance and use the least memory for the combination of Oracle E-Business Suite components and other software that you use.


The key is to achieve best performance at zero cost by modifying the environment, or deciding whether an upgrade will provide sufficient performance. This section describes a number of zero-cost solutions. Although each individual change may only return a small benefit, the cumulative tuning effect can be quite large.

Is a Low Specification Client Usable? Looking at the charts throughout this paper, you can see that a low-specification client can provide surprisingly good performance with OAF/HTML screens and with several of Oracle E-Business Suite forms once they have either been opened or previously used in the same session. In order to decide whether a particular client specification is acceptable, you need to relate the performance directly back to business requirements to ensure that acceptable throughput is provided for all the transactions compiled during the Creating a List of Test Candidate Transactions stage.

Low speed clients are almost as fast as more powerful clients for many

E-Business Suite functions. Memory is far more important than CPU speed.

Bearing in mind that the server used for the tests in this paper was very small, you should expect to achieve form and screen opening times that are at least as good as those shown in the charts. In addition to measuring the first usage and subsequent times you also need to consider other effects, such as 100 users at a remote location all opening a form for the first time and therefore downloading a new set of JAR files. The impact is not only the network, but also the associated loss of productivity while the network is saturated.

Controlling the User Environment It is imperative to control the Windows environment on

low-specification clients. The importance of this section cannot be overstressed. Typically, performance engagements involve users complaining about response times. Lack of control over the user’s environment is a frequently a major contributing factor. There are two very different schools of thought.

In the first approach, some customers roll out a basic image and allow their users to make almost any changes. The second approach has customers expending a lot of time and effort creating an image that is extensively controlled by strict policies. Regardless of the approach, the key elements of a standard corporate configuration are connectivity, security, and end-user applications. In general, allowing end users to make a significant number of changes to the client environment usually degrades system performance. Regardless of the machine, you can ensure a particular level of performance by preventing unnecessary changes. Perhaps not surprisingly, the importance of this increases significantly when resources are limited.

You will often find a higher incidence of end-user performance complaints in this type of environment. User-instigated changes have accounted for as much as 30% degradation in the time it takes to render a form. If you do not maintain at least rudimentary control of the end-user’s Windows environment, not only will you have to expend a lot more effort (and budget) on tuning the database, application tiers and network to compensate, you may also need to upgrade the clients prematurely.

The Low-Specification Client Checklist The following items provide a checklist that you will need to customize for your environment and product mix:


• Add more Memory: If possible, add memory to old clients as this considerably improves performance at minimal cost. It is worth repeating: Memory is far more

important than CPU speed, but you also need to consider the Windows

environment. • Tune the Windows Environment: In addition to tuning the browser, it is

essential to tune low specification machines in order to ensure their usefulness and ensure adequate performance. Tuning Windows is well documented and this task is straightforward. Appendix B contains a set of high-impact recommendations that typically make a substantial difference to the performance of the client environment. This set of changes were made to the 366MHz laptop; bear in mind that old laptops had several technical compromises and were generally not as fast as their desktop equivalent. Although each change individually has a relatively small effect, they collectively improved performance by 2-4 seconds when restoring a screen from being minimized to being fully rendered. The following controls are highly recommended:

o Prevent the use of high-resolution wallpapers, including high-resolution corporate logos.

o Set Windows performance options to “Adjust for best performance”

o Reduce the number of screen colors.

• Tune the Browser: Review the Factors Affecting the Browser Memory Footprint section for guidelines on judicious removal of browser themes, toolbars, codec’s, extensions, plug-ins, helper applications, language packs, and other superfluous components.

• Use the Leanest Technologies: The best combination for low memory clients is IE6 with JInitiator for E-Business Suite 11i and IE6 with JRE for E-Business Suite 12. Extending the longevity of existing hardware and keeping user stress levels to a minimum is more important than new browser features. This combination also provides the best possible performance.

Although the users might like additional features that newer browsers introduce,

being able to achieve the necessary business throughput will keep their stress

level lower.

• Use OAF/HTML screens: Wherever possible, use OAF/HTML screens in preference to Oracle Forms, especially if you are just reviewing information. Not only are the client requirements smaller, but also there is a smaller load on the network. This is something that you may want to consider when creating your own custom or bespoke applications.

• Minimize the Number of Applications Being Used: Keeping the number of applications that users run simultaneously to a minimum may sound obvious, but needs mentioning. When there is little free memory, Windows will reassign memory pages from idle programs, programs with minimized windows, and eventually the program you are using. While restoring pages that have not been reallocated to other programs from the standby list may be relatively quick, there is a comparatively large penalty when accessing the program from disk, or the page file.


• Minimize Unused Non-Oracle Applications as a Working Practice: The Process Trimming section showed the relationship between the Working Set and Private Bytes. Minimizing a form results in a substantial reduction of the Working Set and therefore the physical memory dedicated to the process; this is ultimately limited by Private Bytes, which is inaccessible to any other process. When Oracle E-Business Suite forms start, the windows tend to overlay other applications that are already running, as well as browser windows. The benefit of minimizing all the other applications and idle browser windows depends on the applications being used. Even if you have a relatively large amount of memory, the tendency to run more applications simultaneously and leave them on screen behind the application being used can result in unnecessary paging. These situations can be resolved by a change in working practice, i.e. do not keep applications open if they are not going to be used for a significant period as this will free up memory for use by the active applications.

• Avoid Minimizing Oracle E-Business Suite Components: Minimizing an Oracle E-Business Suite form or screen increases the probability of aggressive memory trimming, even when the released memory is not necessarily needed for other applications. Nothing can be done about the static costs of reading back from the standby list, disk, or paging file, and the Working Set pages will be aged if the form or screen is left idle for a long period. However, as you have seen, you cannot explicitly affect the memory management algorithm or processes. However, minimizing and later maximizing a form that contains a multi record block causes a loss of mouse focus within that block. It is therefore advisable not to use the minimize form option until the resolution of Oracle bug 6041461 (as mentioned in the Trimming section). Further information can be found in Oracle MetaLink Note 393931.1 for Oracle E-Business Suite 12, and 290807.1 for Oracle E-Business Suite 11i.

Minimizing releases physical memory. An application that has been radically trimmed

may take several seconds before it is responsive.

• Keep Menus Small: Keeping the number of menu items as small as possible not only helps to reduce client render time, but also alleviates the load on the network. Without any patches, the Manufacturing and Distribution Manager Menu took almost a minute to display on a network with only 256Kbps of available bandwidth and almost two minutes with 128Kbps (for all latencies from 50mm to 300ms). There is another current issue with this in that if you click on that responsibility, wait for it to render and then decide to do something else first, the next time you select the responsibility, you will have to wait the same time again. Review Oracle bug 6329486 for available patches.

• Consider Reallocation Rather than Purchasing: Create a list of customer-facing staff and power users who all need fast response times, and exchange their slower machines for faster machines from other parts of the organization where the users do not need such powerful machines. This can save buying new hardware whilst maintaining business throughput.

Starting forms before a shift is something that you will find many people do as a

matter of course. It is not that different from starting your PC when you arrive at

the office.


• Start the Application Prior to the Work Shift: Several of the charts show that the performance of the various machines in the test is very close once the JAR files have been validated. Regardless of the speed of their machines, in order to ensure the best possible performance, several large call-centers have adopted the practice of having their end-users start the applications that they are going to be using prior to the commencement of their shift. Not only does this ensure optimal performance per se, it enables staff to provide a more professional service; customers have to wait less time, and the transactions take place more quickly.

Memory Optimizers

If you have memory-constrained environments, do not be tempted by memory optimizers. Generally, their sole function is to access as much memory as possible as quickly as possible, which results in the Windows memory manager shrinking the Working Set (and hence the physical memory) assigned to other programs. Effectively, this is comparable to minimizing every program (including those that you were working with), with the memory later having to be faulted back from the standby list – assuming that it hasn’t been moved to the free list and needs to be re-read from disk or the paging file.

So, How Much Memory Does my Application Need? As mentioned previously, Working Set includes shared and private committed memory allocated to the process, but is usually overstated because it includes memory shared with other processes. In contrast, Private Bytes is the memory allocated to a process that cannot be shared by other processes remembering that some may be paged rather than in physical memory. Private Bytes is probably the most useful figure to use when trying to answer the question “How much memory is my application using?”

Use Private Bytes and Working Set together.

Process Explorer can be configured to show Private Bytes delta. Positive delta values show the process being allocated more memory, while negative value indicates that memory is being released or trimmed. As an aside, although not generally an issue any more, as memory can be leaked anywhere in the address space, Private Bytes (not Working Set) can be used to provide evidence of a leak.

Use Private Bytes and not Working Set to identify program leaks.

More Testing Tips The biggest lessons learnt during this investigation were as follows:

• Make sure that you have a complete list of forms (or business flows) and specific steps that you are going to measure at the start of the process.

• Use Process Explorer to provide detailed analysis of the environment.

• Record the spreadsheet row reference against each of the results, to ensure that a slight change in names does not introduce ambiguity. This approach helps sift through dozens of files when you are trying to drill down on a particular issue.

• Avoid retests by exporting a full set of results from Process Explorer for each test, to enable later drill down if anomalies ensue.


• Never minimize an E-Business Suite form or screen during a test, as the process’s Working Set will be understated and test results invalid.

• Create graphs and charts as you go through the tests, as this will help you to spot trends and patterns. It will also highlight anomalies that can then be immediately verified or further investigated. Although common sense, this may save time later when resetting the test environment.

• As Private Bytes does not include the shared memory that the process is using, restart the browser between each test to ensure consistency, regardless of whether you are measuring a single E-Business Suite form or screen, or an entire business flow. This is how the tests were conducted for this paper.

The evidence suggests that the Working Set may be automatically trimmed with a single E-Business Suite form or screen open on a PC client with only 1GB RAM, which will skew your results. This is why you need to monitor private virtual bytes.

If you use a similar specification client to those in this paper, you should be able to take a few measurements on your system and then scale your results, thereby making use of the charts in this paper. Drawing charts early in the collection process will enable you to see patterns and trends more quickly, such as memory utilization levels. Remember though, that there may always be significant exceptions to trends.

CONCLUSION This paper is aimed at customers who are trying to extend the useful life of their existing Windows clients, or trying to ensure that they specify an adequate client at the optimum price/performance point. A rigorous approach is essential when reviewing or planning for hundreds or thousands of existing or new clients, with the first half of the paper providing a theoretical background for the testing and terminology described in the second half.

Memory is far more important than CPU for performance.

Windows appears overly aggressive when reclaiming physical memory, and while you cannot control the process, you can compensate for it to some extent by adopting the good working practices outlined throughout this last section.

As stated, the best combination for low memory clients is Internet Explorer 6 with JInitiator for Oracle E-Business Suite 11i and Internet Explorer 6 with JRE for Oracle E-Business Suite 12. Oracle and Microsoft state that adding memory makes a significant difference to performance. This has been conclusively proven for the Oracle E-Business Suite. There is a very small difference in performance and response times (about 2 seconds) between the two fastest clients. Even though they are a generation apart in technology terms, one is double the cost of the other.

Oracle and Microsoft state that adding memory makes a significant difference to performance. This has been conclusively

proven for the Oracle E-Business Suite.

The importance of the Holistic Approach can be proven through performance auditing. Not only does it encompass modeling all the salient factors in your test environment, but it also extends to consider auditing system performance of your key critical business transactions over time. In one exceptional case, the users at one site complained only after performance had reduced by 50% over the period of a year, which is something a performance audit would have highlighted much sooner. A rule of thumb is that users will generally not notice any single change (positive or

Use the holistic approach to model all the salient factors in your test environment. If

users have large amounts of data on-screen at remote locations, make sure you

include all salient factors.


negative) in performance that is less than 20%, especially over time. Consider the case of an (expected) time of 10 seconds to open a form. Would you notice if this either declined to 8 seconds or lengthened to 12 seconds over the course of a month or two?

Put simply, if a users’ client is often seen to be thrashing and Private Bytes exceeds Working Set for the majority of applications, the client is running too many applications and/or needs more memory. The point of this paper is that costs incurred in upgrading a range of perhaps geographically dispersed machines can be prohibitive and maybe avoided by using the methodical approach provided in this paper. Ultimately, you need to ensure that the desktop configuration is sufficient to achieve the throughput necessary to sustain your business.


Appendix A: OAF/HTML Screens and Forms Used in the Tests

This appendix lists the forms and screens used during the tests. In order to be able to compare results you always need to follow the same path, to avoid any deviation that might affect the measurements.

Login Screen Main Menu User: operations/welcome Payables Menu User: operations/welcome Responsibility: Payables, Vision Operations (USA) Enterprise Asset Management Menu User: operations/welcome Responsibility: Enterprise Asset Management, Vision Operations Manufacturing and Distribution Manager Menu User: mfg/welcome Responsibility: Manufacturing and Distribution Manager Order Management Super User Menu User: mfg/welcome Responsibility: Order Management Super User, Vision Operations (USA) Asset Inquiry: Query Screen User: operations/welcome Responsibility: Asset Inquiry, Vision Operations (USA) Function: Asset Inquiry: Asset Inquiry Asset Inquiry: Detail Screen User: operations/welcome Responsibility: Asset Inquiry, Vision Operations (USA) Function: Asset Inquiry: Asset Inquiry

Instructions: Enter Asset Number: 100080 [GO] Click on Asset 100080

Sales dashboard User: operations/welcome Function: Sales User View Payslip User: operations/welcome Responsibility: Employee Self-Service Function: Employee Self-Service: Payslip Absence Management User: operations/welcome Responsibility: Employee Self-Service Function: Employee Self-Service: Absence Management


Customer Fulfilment Management Dashboard User: operations/welcome Responsibility: Daily Supply Chain Intelligence Function: Dashboards: Customer Fulfilment Management Inbound Telephony Management Dashboard User: operations/welcome Responsibility: Daily Interaction Center Intelligence Function: Dashboards: Inbound Telephony Management Payables Invoice Workbench (Inquiry) - APXINWKB User: operations/welcome Responsibility: Payables, Vision Operations (USA) Function: Invoice Inquiry: Invoices Purchase Order Workbench - POXPOEPO User: operations/welcome Responsibility: Purchasing, Vision Operations (USA) Function: Purchase Orders: Purchase Orders General Ledger Journals - GLXJEENT User: operations/welcome Responsibility: General Ledger, Vision Operations (USA) Function: Journals: Enter Project Creation - PAXPREPR User: operations/welcome Responsibility: Projects, Vision Operations (USA) Function: Projects, Vision Operations (USA): Projects Inventory: Item Costs - CSTFITCT User: mfg/welcome Responsibility: Manufacturing and Distribution Manager Function: Inventory: Costs: Item Costs

Instructions: Select: M1 (NB Screens are there - no need to click FIND) Order Entry - OEXOEORD User: mfg/welcome Responsibility: Order Management Super User, Vision Operations (USA) Function: Orders. Returns: Sales Orders Contact Center - CSCCCCRC User: service/welcome Responsibility: Customer Support, Vision operations Function: Customer Management: Contact Centre Customer Management: Create Service Request – CSXSRISR User: service/welcome Responsibility: Customer Support, Vision operations Function: Service Requests: Create Service Requests Service Contracts Authoring: Launch Contracts – OKCEXPOR User: lease/welcome Responsibility: Service Contracts Manager, Vision Enterprises Function: Service Contracts Manager, Vision Enterprises | Launch Contracts


Service Contracts Authoring: Agreements Form – OKSAUDET User: lease/welcome Responsibility: Service Contracts Manager, Vision Enterprises Function: Service Contracts Manager, Vision Enterprises | Launch Contracts

Instructions: Select Tools | New Enter: Operating Unit: Vision Operations Enter: Category: Service Agreements [Create]

Leasing Center – OKLCSHDR User: lease/welcome Responsibility: Lease Center Manager Function: Lease Center Manager: Lease Center Customer Interaction History – JTFIHGUI User: lease/welcome Responsibility: Lease Center Manager Function: Lease Center Manager: Lease Center

Instructions: Click [All Customer Interactions] [OK] (On popup) Supply Chain Planners Workbench Form – MSCFNSCW User: mfg/welcome Responsibility: Advanced Supply Chain Planner Function: Supply Chain Plan: Workbench

Instructions: Select TST:M1 Select: Plans Select: Constrained (Constrained Plan) Click [Gantt]

Supply Chain Planners Workbench (Gantt Chart) User: mfg/welcome Responsibility: Advanced Supply Chain Planner Function: Supply Chain Plan: Workbench

Instructions: Select TST:M1 Select: Plans Select: Constrained (Constrained Plan) [Gantt] Enter: Criteria: Order Type Enter: Condition: Equals Enter: From: Planned Order [Find]

WIP Discrete Job Workbench (No Data) User: mfg/welcome Responsibility: Manufacturing and Distribution Manager Function: WIP : Discrete - Job Workbench

Instructions: Select M1 [OK] WIP Discrete Job Workbench (With Data) User: mfg/welcome Responsibility: Manufacturing and Distribution Manager Function: WIP : Discrete - Job Workbench

Instructions: Select M1 [OK] Select: Filter [OK]


Appendix B: Tuning the Client In addition to tuning the browser, it is essential to tune low specification machines in order to ensure their usefulness and ensure they perform adequately. Tuning Windows is well documented and this task is straightforward. The quality of advice varies tremendously, and occasionally inaccurate or incorrect advice can be copied or replicated, in part or whole, across a plethora of forums and sites. Look for pages that discuss tuning myths and realities, as these tend to provide slightly better information. It is important to reiterate: Always look for supporting evidence and perform your own tests.

While many companies will roll out a locked-down corporate image, problems often occur when users are allowed to make changes to their system. It is worth reiterating the following recommendations, which have proved valuable for many customers, and should be followed for low-specification if not all business machines:

• Prevent the use of high-resolution wallpapers, including high-resolution corporate logos.

• Set Windows performance options to “Adjust for best performance” - This will disable animation, and your tests should show that this has a significant effect when rendering the screen.

• Reduce screen colors. Older technology is slow and problems increase when video memory is shared with the core system memory.

• When demands for physical memory result in excessive paging, a fixed-size page file is beneficial on clients with a slow disk, as Windows does not have to manage the growth. Although less of an issue with modern machines, the page file should ideally be located on a FAT32 partition, as this has less file management overhead than NTFS.

This set of changes was made to a 366MHz laptop. Bear in mind that old laptops had a number of technical compromises and were generally not as fast as their desktop equivalent. Although each of these has a relatively small affect on its own, collectively they can improve performance by 2 –3 seconds when restoring a screen from being minimized to being fully rendered.

If you need to work on a single user’s client that seems to be running slowly, one of the best utilities is Microsoft Autoruns as shown in Figure B1. This program shows programs that are automatically started from the Startup folder, and from the Run, RunOnce, and associated Registry keys. In addition, it shows the browser helper objects you need to identify when investigating and trying to reduce the browser footprint.


Figure B1: The Autoruns Screen


Appendix C: Windows Memory Internals Each 32-bit Windows process has 4GB of virtual address space available. By default, this is divided equally between applications (user-mode address space) and the Windows kernel (kernel-mode address space). For example, application code, DLL code, and variables are located in the user address space while the Windows kernel, page tables, and system reside in the kernel address space. It is possible to increase the user space beyond the 2GB restriction using 4-Gigabyte Tuning (4GT), but this is generally only used on servers rather than client machines. Although it is less likely that you will be reading this paper if you have exceptionally powerful PC clients, the use of a larger application address space can provide considerable performance benefits and may be something that you are considering. However, 4GT tuning is a trade-off as it reduces kernel features such as the file cache, which can have an adverse affect on some applications.

An application or process may reserve some or all of the allocated 2GB of user address space available to it, regardless of the amount of available physical memory; this reserved memory is referred to as virtual memory. The virtual address space contains three different types of memory pages, which are defined as follows:

• Reserved - A process will start by using a set amount of physical memory, but will also reserve a range of memory pages that it can use in the future. This protects its memory from allocation requests by other processes. Note that this is only an allocation of memory from its virtual address space, and no resources are actually used in that it does not use any physical memory or even reserve space in the page file.

• Committed - When a process needs to use address space that it already has reserved, it has to commit it. During this phase, memory is allocated as either physical memory, reserved space, or memory pages that are backed by the reserved space in the page file. This allows for future operations by the Virtual Memory Manager and Working Set Manager to trim pages from a process out of physical memory and into the page file, or onto the standby list if physical memory is needed by the operating system or another process.

• Free - In this context, free memory is memory within the process address space that remains available to be reserved or committed. Attempting to access free memory without first attempting to reserve or commit it results in an access violation and program termination.

Virtual memory can far exceed the amount of memory that a process is actually using, either in real or paged memory. A process may reserve hundreds of megabytes of address space from its pool of 2GB. For example, when a browser is started, it allocates over 100MB virtual address space (this typically increases to between 500-600MB with browsers running forms). As an example, Symantec Antivirus may have a virtual address space of 150MB, but only using 38MB of physical memory. The important point is that the majority of the pages will simply be reserved with the rest either committed or free; this ensures that when the process grows, future memory allocations will be from a contiguous address space. Consequently, virtual memory does not represent the real amount of memory that the process is using and, possibly to prevent confusion, it is not shown in the Windows Task Manager.

Virtual memory does not represent the real amount of memory

that a process is using.


Private Bytes, Shareable Memory, and Working Set In More Detail

Recall that committed memory pages are backed by the page file or reside in physical memory. To complicate the question of how to measure memory, committed memory can be private or shared. These are defined as follows:

• Private memory represents the private virtual memory allocated to the process and is committed memory that is protected and inaccessible by any other process. It can reside in physical memory or the page file and can be considered as the amount of page file space that would be occupied if the entire process was paged out of physical memory. The Performance Monitor counters Process/Private Bytes and Process/Page File Bytes are the same as VM Size in Task Manager and Private Bytes in Process Explorer (a free tool from Microsoft) .

• Shared memory can be shared by one or more processes. For example, this could be an image, in which case the address where the image is loaded is mapped to the address space of the process in question, or a mapped file, such as a dynamic link library (DLL). As the name implies, a mapped file is not loaded into memory, but each process will use addressing to map (load) only the sections that it is using.

Shared memory reduces the overall physical memory requirements of the system as not all the shareable memory is necessarily shared with other processes, and what is shared by the application being measured may actually be mapped from another process that was already running. Therefore, this analysis needs to measure the amount of shareable and shared memory and consider whether this level of complexity is relevant.

Process Trimming and the Memory Stacks

Each program has an allocated range of physical memory (its Working Set). If the memory requirements of an existing process increase, or you start another application and there is enough physical memory available, it is simply assigned from the available pool of free memory without affecting other processes. However, if insufficient free memory is available, Windows will trim (displace) memory pages from other processes.

When Windows trims memory from existing processes, it places unmodified pages on the Standby List, and modified pages on the Modified List (as shown in Figure C1). Modified pages have to be written to the paging file before they can be transitioned to the standby list. Importantly, these pages still contain the original data; if they are subsequently referenced, a soft page fault occurs and the requisite pages are moved back to the process’s Working Set in a memory-to-memory operation. If the program references modified pages that have been overwritten (and moved to one of the other lists), it will read original program code (such as a DLL) from the original source on disk, or modified pages from the page file.

The standby, free, and zero lists are added together and the sum classified as

available bytes.

Standby list pages are transitioned to the free list. Once the free list has a predefined set number of pages, the kernel pages thread executes to zero-fill the memory and pages are transitioned to the Zero List, where they remain until they are allocated to a new program or a process that needs more memory.


Figure C1: Windows Memory Lists

Although desirable, it is not possible to control Windows Working Set trimming unless the program explicitly invokes it, for example, by using the SetProcessWorkingSetSize API.

Which Processes will Windows Trim First?

Windows kernel memory consists of two sub-pools: the paged pool, which can be paged to disk and the non-paged pool. The latter includes essential processes such as drivers that cannot be paged out of memory. All processes have defined maximum and minimum values for working set size. These can either be set programmatically or more usually set to a default value, but the minimum is a non-zero value meaning that an application can never be completely paged out of memory.

Windows will dynamically adjust the rate at which it examines Working Sets based on memory utilization and the age of pages in the process Working Set. In order to decide which processes are candidates to be trimmed, the Windows memory manager examines each process. It compares the current memory allocation to its minimum Working Set, the age of the page, whether it has been accessed since the last Working Set trim, and other factors such as the number of hard page faults (physical disk reads) that would be incurred.

It is difficult to monitor and interpret all the Windows memory lists and therefore the best indicators that you have are Private Bytes and Working Set, and specifically the ratio between them.


Oracle E-Business Suite PC Client Performance October 2007 Author: Andy Tremayne Contributing Authors: Robert Azzopardi, David Hunter Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA 94065 U.S.A. Worldwide Inquiries: Phone: +1.650.506.7000 Fax: +1.650.506.7200 oracle.com Copyright © 2007, Oracle. All rights reserved. This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

Documents

Oracle E-Business Suite PC Client Performance and … E-Business Suite 11i and 12 PC Client Performance Page 4 Comparing ... consider which factors you need to include in your testing