An eBook by Sharegate

How to Optimize Your Migration Speed When Using Sharegate

Introduction

It’s finally time to get your SharePoint migration underway. You’ve followed all of the instructions in our Migration Checklist to make sure you’ve remembered everything and you’re ready to rock. Management has set a strict time table for your project and you want to make sure that you squeeze every possible ounce of performance out of Sharegate to get everything moved as quickly as possible.

Sharegate is a powerful and easy to use tool, but it can only go as fast as the infrastructure it has to interact with.

By the end of this eBook, you will know how Sharegate interacts with different components and what bottlenecks to look out for that could negatively impact the speed of your migration project.

About the Author

As Chief Technology Officer and one of the masterminds behind Sharegate, Christian Merat leads the long term technology vision. He’s been working with Sharegate since day one and is also a tutor for junior developers.

2

Overview of the Components Involved in Your Migration

Sharegate sits smack in the middle of your source and destination farms. This means that issues that occur at either of these locations (or even between these locations) can negatively impact the migration speed.

Let's have a look at theses different components and what you need to keep an eye out for.

3

Source and Destination Farms

It’s essential to have a healthy farm in order to obtain optimal performance out of your migration.

The entire migration process with Sharegate goes through the public APIs that are exposed by SharePoint over HTTP/S. All of the interactions go through the Web Front-End (WFE), some Application Services and finally the SQL database that holds all of SharePoint's content. In the case where Remote Blob Storage (RBS) is used in your farm, this service will also be solicited.

To make sure that your farm is in top shape, here are a few things you can monitor:

• The SharePoint Health Analyzer isn't flagging critical issues.

• The servers in your farm do not have any physical bottlenecks (see section on Resource Monitor).

Tip: Before starting your migration, follow the best practices outline in the SharePoint Performance Guide to make sure that your farm is running in tip-top shape.

4

Active Directory

SharePoint has a fairly strong coupling with Active Directory and during a migration, this link is stressed even more. This is because Sharegate will be doing a lot of user searches and resolutions to properly assign all of your permissions and author information to the right users.

This will usually result in a large number of calls to Active Directory. So, if your AD server is slow or unresponsive, this can cause a significant decrease in migration speed.

Sharegate does have its own internal caching mechanism, but it will still need to perform each lookup at least once.

5

3- Have the right number of CPU cores on your workstation (for a VM, make sure you have enough virtual processors).

• The number of processors also impacts the number of concurrent items that will be migrated depending on your Performance Level.

• Sharegate is optimized to run with 4 cores (64 concurrent threads) so you should aim for that.

4- Don’t start many parallel migrations on the same workstation (this includes PowerShell).

• These will fight for resources on the same workstation and can potentially lead to significantly decreased performance. We suggest to use multiple workstations running Sharegate in parallel.

Sharegate

The workstation on which Sharegate is running is going to be doing a lot of heavy lifting, since all of the data that transits from your source farm to your destination farm will pass through Sharegate.

There are steps to make sure that this workstation won't be the bottleneck in your migration:

1- Depending on your environment, select the right Performance Level in the Sharegate options.

While it might be tempting to go "High Performance", this will put a lot more strain on your source farm and can, in some cases, even reduce overall performance through excessive context switching. Try a few different performance levels out to find the one that works best for your environment.

2- Check for any physical bottlenecks (see section on Resource Monitor).

6

Network

All communication between Sharegate and both the source and destination farms is done over HTTP/S and as such, is done over your TCP/IP network. This means that it is subject to any bottlenecks you may have in your network infrastructure such as network congestion. This can lead to packet loss as well as high latency as well as bandwidth limitations.

Example of speed when moving a 1gb file

7

Having sufficient upstream bandwidth to upload all your content will help you respect your desired time frame. Also, factor in some overhead for the protocol as well as any other HTTP/S calls required to migrate. This can add up fairly quickly especially if you are migrating a lot of structure elements.

Bandwidth is typically not an issue within a LAN but can be an issue when migrating to Office 365 since the migration will need to travel on the public internet.

If you suspect you have network issues, check for these elements:

• Check router/switch for elevated CPU/RAM usage. (This can lead to dropped packets.)

• Perform a continuous network ping test between the Sharegate workstation and your WFE. (Check for jitter, high latency, dropped packets.)

• Check with your ISP to make sure you have sufficient upstream bandwidth if you are migrating to Office 365.

• If you are using a VPN to access your farms, there may be additional latency added.

• If you are using a Proxy to communicate with your farms, check the Proxy for elevated CPU/RAM usage since this can lead to dropped packets.

• Use a debugging tool such as Fiddler (or take a look at our built-in Diagnostic Mode) to inspect the traffic between Sharegate and your SharePoint farms. It can help you identify which calls are taking the longest to return and help you identify what part of your infrastructure is slowing the calls.

8

Insane Mode to Office 365

Sharegate leverage Microsoft's Office 365 Import API to help our customer reach record migration speeds. Migrations that used to be limited to around 500 MB/h could now reach speeds as high as 35 GB/h with typical users seeing 5-15 GB/h.

This API relies on a set of services and infrastructure with their own bottlenecks and limitations. The most prominent change is the need to stage all of the migration content in an Azure Storage Account for asynchronous ingestion by the Office 365 farm.

When importing to Office 365, look out for these items:

• There is a limited number of import bots available per Content Database in Office 365. There can only be a certain number of simultaneous import jobs to each separate Content Database. If you plan on importing a large amount of data (1 TB+) please contact Office 365 support to provision the required Content Databases ahead of time in order to spread out the load.

• If you opt to use a custom Azure Storage Account with the Insane Mode instead of the default containers, you should consider the data center in which you create the account. Your Office 365 tenant is located in a specific Azure data center and uploading to a storage account found in another geographic location will add additional delays to the import process. Using the default container will always ensure that the storage account is in the same data center so this consideration is removed.

GB/H

9

• Using encryption at rest for the contents found in the Azure Storage Account can slow down the import process by about 10%. The added security is, in most situations, entirely justified but there can still be a performance gain if you opt to not use encryption. If you are using the default storage account, encryption cannot be disabled.

• There are certain service limits built into Azure Storage Accounts. These are fairly generous and will probably never be hit in a typical migration scenario (10Gbps maximum ingress / 200,000 IOPS).

• When uploading content to an Azure Storage Account, you may run into bandwidth issues depending on the speed provided by your ISP. If you require additional speed when uploading to Azure, you can look into creating an Azure Express Route. This creates a fast, reliable and private connection directly between your data center and Azure.

10

Here are a few indicators that you should look for in the Windows Resource Monitor:

CPU

• Number of cores: Additional cores can help spread out execution and allow additional instructions to be executed simultaneously.

• CPU - Total: If this is consistently high (over 80%) then you might run into a bottleneck Try to see if you can reduce the number of active processes on the system.

Resource Monitor

As with all software that runs on a computer, performance will always be limited by the physical components that the computer is built from. In the case of a VM, the limit will depend on the specified settings for the VM as well as the hardware powering the host. While Sharegate does not typically add an unmanageable stress on these components, in some cases servers or workstations that are already under substantial load can become sluggish or unresponsive.

11

Memory

• Used Physical Memory: If this value is high, it means that processes will need to use virtual memory in order to store state and instructions. Attempting to read a value that is in virtual memory but not physical memory causes a Hard Fault (see below), which can significantly impact performance. The easiest solution here is to increase physical memory through additional resources either in the VM settings for a VM, or through added memory sticks for a physical server or workstation.

• Hard Faults/sec: Hard Faults occur when a process attempts to read a memory address that is not currently loaded in physical memory. Windows must then recover the memory block from the page file (stored on disk), and copy it to physical memory before execution can continue. This adds significant latency to an operation and is a common cause of application slowdown. If you see a lot of Hard Faults/sec, then increasing available Physical Memory can be an easy fix.

Disk

• Queue Length: This represents the time it takes between a request to read or write (I/O) something on disk and the start of the operation. A high queue time means that there are a lot of processes requesting different elements on the disk at the same time, adding significant latency to operations. There are only two ways to reduce this delay. You could change to a higher performing disk (either SSD, NAS, RamDisk, etc.) or reduce the number of I/O requests. Hard Faults are also a significant contributing factor to disk queue length.

12

There are a lot of components at play when performing a SharePoint migration that might not be obvious at first. While in most scenarios the infrastructure in place is more than capable of dealing with the migration project, in cases where the desired speed is not obtained it is useful to know where to look in order to find the bottleneck.

There will always be a limiting factor to all migrations, the idea is simply to adjust the different components until you reach a speed you are comfortable with and that matches your schedule.

Conclusion About Sharegate

Sharegate helps thousands of IT professionals worldwide manage, migrate and secure their SharePoint & Office 365 environments. A product made with love by Montreal-based software development firm GSOFT, where we truly believe that simplicity and happiness are key to success!

Want to learn more? Connect with us on Twitter and visit share-gate.com for more SharePoint-related content.

@sharegatetools

www.share-gate.com