Aaron Lopez, Jeff DeWeese

ArcGIS Enterprise:Tuning and Scaling

Session Overview

• We will cover all the components of ArcGIS Enterprise:

- ArcGIS Server- Authoring and Tuning Services- Optimizing a specific service or many services

- Portal for ArcGIS- Operational tasks such as backups

- ArcGIS Data Store(s)- Three types for storing information

- ArcGIS Web Adaptor- An application for forwarding requests

- Enterprise Geodatabase- Outside the scope of this session

Infrastructure: A Key Component

• Environments can vary:- Bare metal (physical)- VMs (virtual)- Cloud and related services- Containers/Kubernetes: Up and coming…

• Placement of Storage Components- Server config store- Server directories

- Input, output, cache, jobs- Portal content store- Data Store data content directory

Service Fundamentals

Map Authoring: Tuning Individual Services

• Desktop maps do not always scale for the web• Focus your map

- Scale dependencies- Generalization- Definition queries- Remove unneeded layers- Hide fields you are not using- Annotations instead of labeling- Use same coordinate system for data frame and data

• Modify the ArcGIS Server cacheControlMaxAge property

General Data Considerations

• Match resolution of feature class to the accuracy of the data- If data is only at a meter, there is no need for millimeter resolution

• Index the file and enterprise geodatabase data appropriately

• For enterprise geodatabases- Maintain statistics- Use the recommended spatial type for the RDBMS

- st_geometry, native spatial type, etc…

Enterprise Geodatabases

• Enterprise geodatabase data- Fast

- Keep statistics up-to-date- Indexes on queried fields

- Current data availability- Traditional Versioning

- Fine for desktop editing, more complicated for the web- Versioned queries are expensive….access non-versioned layers when possible

- Branch Versioning- ArcGIS Pro and Enterprise- Designed for many concurrent users and the web - Simpler table structure

File-Based Data

• File geodatabases- Optimization

- Performs best when kept local to server- Best with static data- Make read-only

- Better than shapefiles- Performance- Scalability

- Greater storage limits

Service Models

Tuning ArcGIS Server – How Instances Handle Requests

• Request lifecycle for traditional, dedicated service

A single service instance can only handle one request at a time!

Tuning ArcGIS Server – Services Models

• Three models that services run under:- Dedicated (Traditional)

- Fine tune service settings- Has concept of service instances (also known as SOCs or ArcSOCs)

- Hosted- Utilizes the Relational Data Store of ArcGIS Enterprise

- Shared- Provides a group of instances always available for various services- ArcSOC based

Any service that references user-managed data (e.g. file geodatabase or enterprise geodatabase) -- is run under the traditional model

Tuning ArcGIS Server – Dedicated Services

• The number of service instances impacts:- Memory usage at rest (min setting)- Concurrent requests to be serviced (max setting)

- Wait time for getting access to an instance

• A common administrator goal:- Find balance between min/max

• Tools can help determine service popularity and performance- System Log Parser- ArcGIS Server statistics

Tuning ArcGIS Server – Dedicated Services

• For predictable performance use min=max- Default setting: min = 1, max = 2

- Review and modify min/max based on popularity- Cached service: set max = 1 to conserve memory

- Tile requests not serviced by ArcSOC processes- Be aware of swap space and page file

- Could be factor for popular services if min and max delta is very large

Tuning ArcGIS Server – Dedicated Services

• Avoid resource overload with traditional services- Avoid over allocation

• Concurrent users is not the same as concurrent requests

• Which will the deployment have? - Many services with low-to-moderate popularity - Few services with high popularity- Mixture of both

• Min=0 can be an option for seldom used services- Publish as a Shared service type instead

Tuning ArcGIS Server – Hosted Services

• Hosted services- Does not utilize service instances (i.e., ArcSOCs)

- Highly scalable number of services

- Data returned as JSON/PBF- Client-side rendering

- Data resides in the Relational Data Store- ArcGIS managed

Tuning ArcGIS Server – Shared Services

• Shared services – a game changer!- Utilizes a shared instance pool- Introduced at ArcGIS Server 10.7- Requires services published via ArcGIS Pro- Not all service types are supported

• Provides a group of instances always available for various services- Default is 4- Reduces memory footprint

• Ideal for “seldom” used services- Better option for “Min=0” configuration

Tuning ArcGIS Server – Shared Services

• Dedicated- # Instances = 8

• Shared - # Instances = 6

Map Service Min Instances Max Instances

LosAngeles 0 1Orange 0 1Venture 0 1Riverside 0 1SanBernardino 0 1PalmSprings 0 1NetworkAnalysis 2 2

Map Service Min Instances Max Instances

LosAngeles

4

OrangeVentureRiversideSanBernardinoPalmSpringsNetworkAnalysis 2 2

Tuning

Tuning a Web Map

• Hosted feature service considerations:- Read-only feature services are smaller, faster- If some editors and viewers are needed, use views

• Reduce clicks- The default extent should perform well

• Cache (tiles) can reduce amount of traffic

• Large amounts of feature service data can be slow and overwhelming- Aggregate data using smart mapping- New releases of Enterprise utilize on-the-fly quantization

Tuning a Web Map

• A typical setup:

Enterprise portal

Web Layer

Web Layer

Web Layer

GIS Server

GIS Service

GIS Service

GIS Service

Data Store

Dataset

Tuning a Web Map

• A more efficient setup: consolidate layers with like security into a single service

Enterprise portal

Active Wells

Proposed Wells

Wells by Status

GIS Server

Wells

Data Store

Wells

Tuning Portal for ArcGIS

• Login settings:- Identity and group stores can greatly affect login performance

- Active Directory with users in many groups- Impact greater on older releases

• Backups- Choose right combination of full and incremental- After first full backup, Portal will allow database to grow unbounded- Perform backups on regular basis to keep size under control

- Backup cadence (off hours)

Tuning ArcGIS Data Store

• Understand the three types of data stores available:

- Relational data store- Stores the data for hosted feature layers- PostgreSQL

- Tile Cache data store- Supports 3D Scene Layers and related services- CouchDB

- Spatiotemporal Big Data Store- High volume archiving from GeoEvent Server- Large result data from GeoAnalytics Server

Managing ArcGIS Data Store

• Many command-line tools for managing ArcGIS Data Store:- ArcGIS Data Store command utility reference

• -changedatastoremode (relational data store)- Toggle read-only mode

• -changedbproperties (all types)- Disk space thresholds, RAM heap size for spatiotemporal big data store

• -changebackuplocation (all types)• -updatebackupretaindays (all types)• -updatebackupschedule (all types)

• The describedatastore command gives insight into the current machine’s configuration

Scaling

Scaling Direction – Scale Up or Scale Out?

• Scaling Up - Adding resources to existing machines- Increases throughput without adding machines (+CPU)- Supports increased services instances (+Memory)

• Scaling Out - Adding servers that work as a unit- Manage capacity by adding more servers

• Both strategies can be used together- Scale Up to a reasonable point…then Scale Out

4 cores16 GB

8 Cores32 GB

16 Cores64 GB

8 Cores32 GB

8 Cores32 GB

8 Cores32 GB

8 Cores32 GB

Scaling Constraints – Scaling Up

• Scaling Up- Cannot scale up forever due to practical CPU

and Memory limits…talk with your IT Dept.- Remember, you need at least two servers to

support high-availability- Large servers can lead to large capacity

reduction if a server is unavailable

16 Cores64 GB

16 Cores64 GB

8 Cores32 GB

8 Cores32 GB

8 Cores32 GB

8 Cores32 GB

50% Capacity Reduction

25% Capacity Reduction

Scaling Constraints – Scaling Out

• Scaling Out- Portal High-Availability is limited to 2 servers (active/active)- Relational Data Store is limited to 2 servers (active/passive)- GIS Server sites are highly scalable but there are practical limits- GeoAnalytics Server supports one and three-server site configurations only- GeoEvent Server can be challenging to scale out- More servers = more Windows Server licensing- More servers = more administration

Tiered Deployments: Single-Tier

• Use Case- Small deployments (10 or less users)- Proof of Concept (POC)- Supporting another targeted function (e.g., a

Raster Analytics deployment)

• Components- Portal for ArcGIS + Web Adaptors + Hosting GIS

Server + Relational Data Store

Tiered Deployments: Two-Tier

• Use Case- Deployments where the hosting function is not

dominant and the primary purpose is enabling a broad set of GIS capabilities.

• Components- Web Tier: Portal for ArcGIS + Web Adaptors- App Tier: Hosting GIS Server + Relational Data

Store

Tiered Deployments: Three-Tier

• Use Case- Deployments where the hosting function is

dominant.- Large number of hosted services- Using spatial analysis tools (e.g., Insights for

ArcGIS)

• Components- Web Tier: Portal for ArcGIS + Web Adaptors- App Tier: Hosting GIS Server- Data Tier: Relational Data Store

Workload Separation: Drivers

• Manage scalability for different workloads- Data Management- Analysis- Visualization

• Maximize performance/scalability

• Reduce the risk of service interruption

• Align with licensing strategies

• Allow for effective monitoring/troubleshooting

Workload Separation: Separate GIS Server Role Deployments

• Licensing Roles- GIS Server (e.g., Mapping Site)- Image Server- GeoEvent Server- GeoAnalytics Server- Business Analyst Server- Notebook Server

• Other Roles- Hosting Server- Geoprocessing/Printing Server- Utility Network

Image ServerGIS Server GeoEvent Server

GeoAnalytics Server

Business Analyst Server

NotebookServer

ArcGIS Server Licensing Roles

Workload Separation: Example Architecture

• Four GIS Server Sites- Hosting Site- Mapping Site- Utility Network Site- Imagery Site

Workload Separation: Example “Large” Architecture

• Eleven GIS Server Sites- Hosting Site- Imagery Site- Viewer Site 1- Viewer Site 2- Viewer Site 3- Geoprocessing Site- ArcFM Web Site- ArcFM Mobile Site- Mapping Site- StreetMap Site- Utility Network Site

Thank you!