Site Migrations with URL Changes

SITE MOVESWpromote - October, 2014

PREPARED BY:Mariel Martinez

SEO Manager

[email protected]

310.321.4095

Thursday, October 30, 14

mailto:[email protected]

mailto:[email protected]

Table Of Contents

2

SECTION 1

SECTION 2

SUMMARY

3

6

9


What’s a Site Move?


Introduction

A site move (or site transfer, rename, change, or migration) falls into two general categories:

What is a site move?

4

Site Move with URL Change

‣ Site moves without URL changes - Only the underlying infrastructure serving the

website is changed without any visible changes to the URL structure. Here,

you might move www.example.com to a different hosting provider while

keeping www.example.com as the same URL structure for your site. (I.E. Client

moves from Shopify to Magento)

‣ Site moves with URL changes* - Here, the URLs on the website change in any

number of ways:

The protocol—http://www.example.com to https://www.example.com

The domain name—example.com to example.net

The URL paths—example.com/page.php?id=1 to example.com/widget

*site moves with url changes can also involve a change in hosting providers.


https://support.google.com/webmasters/answer/6033388?ref_topic=6033383

https://support.google.com/webmasters/answer/6033388?ref_topic=6033383

http://www.example.com




https://support.google.com/webmasters/answer/6033049?hl=en&ref_topic=6033084

https://support.google.com/webmasters/answer/6033049?hl=en&ref_topic=6033084



https://www.example.com

https://www.example.com

Introduction

Notes

5

To change the configuration of how your site serves mobile and desktop users, see Smartphone Site Move Guidelines on

our developer's site.

For our purposes, a site redesign is not considered a site move.

General recommendations for Site Moves:

Split your move into smaller steps.

If you simultaneously change your hosting provider and URL, it is challenging to diagnose issues that might come up. We

recommend separating these steps, while doing them for all URLs at the same time if possible.

Time your move to coincide with lower traffic, if possible.

If your traffic is seasonal or dips on certain weekdays, it makes sense to move your site during traffic lulls.

This lowers the impact of anything that breaks, and also dedicates more of your server’s power to helping Googlebot

update our index.


https://developers.google.com/webmasters/smartphone-sites/change-configuration

https://developers.google.com/webmasters/smartphone-sites/change-configuration

Move a Site with URL Changes


Pre-Site Move


7

The details of site preparation vary for each site move, but typically you’ll do one or more of the following:

• Set up a new content management system (CMS) and add content to it.• Transfer images and downloads (such as PDF documents) you currently host.

These might already be getting traffic from search results or links, and it’s useful to tell users and Googlebot about their new location.

• For a move to HTTPS, get and configure the required TLS certificates on your server.

1. Prepare the New Site


Pre-Site Move


8

‣Setup Robots.txt File on Your New Site

The robots.txt file for a site controls which areas Googlebot (and Googlebot-Mobile) can crawl. Make sure the directives in the new site’s robots.txt file correctly reflect the parts you want blocked from crawling.

Note that some site owners block all crawling while in development. If you follow this strategy, make sure you prepare what the robots.txt file should look like once the site move starts. Likewise, if you use noindex directives during development, prepare a list of URLs from which you’ll remove the noindex directives when you start the site move. Learn more about controlling crawling and indexing.



https://developers.google.com/webmasters/control-crawl-index/

https://developers.google.com/webmasters/control-crawl-index/

Pre-Site Move


9

‣Provide Errors for Deleted or Merged Content


For content on the old site that will not be transferred to the new site, make sure those orphaned URLs correctly return an HTTP 404 or 410 error response code. You can return the error response code at the source URL in the configuration panel for your new site, or you can create a redirect for a new destination URL and have that return the HTTP error code.

*For Merged Content: Make sure you are also porting or updating the meta data to the new site

‣Avoid Irrelevant Redirects

Don’t redirect many old URLs to one irrelevant destination, such as the home page of the new site. This can confuse users and might be treated as a soft 404 error. However, if you have consolidated content previously hosted on multiple pages to a new single page, it is acceptable to redirect the older URLs to that new, consolidated page.

“We discourage the use of so-called "soft 404s" because they can be a confusing experience for users and search engines.” - GWT


https://support.google.com/webmasters/answer/181708


Pre-Site Move


10

‣Ensure Correct Webmaster Tools Settings

A successful site move depends on correct—and up to date—Webmaster Tools settings.

If you haven’t already, verify you own both the old and destination sites in Webmaster Tools. Be sure to verify all variants of both the source and destination sites. For example, you should verify www.example.com and example.com, and include both the HTTPS and HTTP site variants if you use HTTPS URLs. Do this for both source and destination sites.







Pre-Site Move


11

‣Review Webmaster Tools Verification

Make sure your Webmaster Tools verification will continue to work after the site move. If you're using a different method of verification, keep in mind that verification tokens may be different when the URL changes.

If you’re using the HTML file method to verify ownership of your site in Webmaster Tools, make sure you don’t forget to include your current verification file in your new copy of the site.

Likewise, if you verify ownership with an include file that references meta tag or uses Google Analytics to verify ownership, ensure the new CMS copy includes these as well.


Make Sure GWT Will Work After Site Move








Pre-Site Move


12


If you had changed some of the configuration settings in Webmaster Tools for your old site, make sure the new site’s settings are updated to reflect those changes as well. For example:

URL parameters: If you’ve configured URL parameters to control the crawling or indexing of your old URLs, make sure the settings are also applied to the new site if needed.

Geotargeting: Your old site might have explicit geotargeting, such as a geotargetable domain or a country-coded top-level domain (such as .co.uk). Apply the same setting to the new site if you want to continue targeting for the same region. However, if your site move is meant to help your business expand globally and you do not wish your site to be associated with any country or region, select Unlisted in the drop-down list of the Site Settings page.


URL Parameters on Old Site








Pre-Site Move


13


Crawl rate: We recommend not limiting Googlebot’s crawl rate in Webmaster Tools for both source and destination URLs. We advise you don’t configure a crawl rate setting, either. Only do this if you know that your site cannot handle Googlebot’s volume of crawling. If you have already limited Googlebot’s crawl rate for your old site, consider removing it. Google has algorithms that automatically detect that a site move has been implemented and we alter Googlebot’s crawling behavior so that our indexing quickly reflects the site move.

Disavowed backlinks: If you’ve uploaded a file to disavow links on your old site, we recommend you re-upload it again using the Webmaster Tools account of the new site.


Re-Upload Disavowed Links






Pre-Site Move


14

‣Clean Up Your Recently Purchased Domain

If your new site is for a recently purchased domain, you'll want to make sure it's clean of any outstanding issues from the previous owner. Check the following settings:

Manual action for previous spam. For sites that do not comply with our Webmaster Guidelines, Google is willing to take manual action, such as demoting them or even removing them from our search results altogether. Check the Manual Actions page in Webmaster Tools to see if any manual actions have been applied to the new site, and address any problems listed there before filing a reconsideration request.

Removed URLs. Make sure there aren’t any URL removals left over from the previous owner, especially a site-wide URL removal. Also, before submitting URL removal requests for your content, make sure you understand when not to use the URL removals tool.


Check For Manual Actions








https://support.google.com/customsearch/answer/2631051

https://support.google.com/customsearch/answer/2631051





Pre-Site Move


15

‣Use Web Analytics

During a site move, it’s important to analyze usage on both the old and new sites. Web analytics software can help with this. Typically, web analytics configuration consists of a piece of JavaScript embedded in your pages. The details for tracking different sites varies depending on your analytics software and its logging, processing, or filtering settings. Check with your analytics software provider for help. Additionally, if you have been planning to make any configuration changes to your analytics software, now is a good time. If you use Google Analytics, consider creating a new profile for your new site if you want clean separation in your content reports.


Setup GA on New Site


Pre-Site Move


16

It's important to map your old site's URLs to the URLs for the new site. This section describes a number of general approaches you can take to correctly assess the URLs on your two sites and facilitate mapping. The exact details of how you generate this mapping will vary depending on your current website infrastructure and the details of the site move.

2. Prepare a URL Mapping


Pre-Site Move


17

‣Determine your current URLS

In the simplest of site moves, you may not need to generate a list of your current URLs. For example, you could use a wildcard server-side redirect if you’re changing just your site’s host (e.g. moving from example.com to example.net).

In more complex site moves, you will need to generate a list of old URLs and map them to their new destinations. How you get a listing of old URLs depends on your current website’s configuration, but here are some handy tips:

• Start with your important URLs. To find them:◦ Look in your sitemaps because it's likely your most important URLs have been submitted in Webmaster Tools that way◦ Check your server logs or analytics software for the URLs that get the most traffic◦ Check the Links to your site feature in Webmaster Tools for pages that have internal and external links


Review GWT Features, Links to Your Site






Pre-Site Move


18


• Use your content management system, which can typically provide an easy way to get a listing of all URLs that host content.

• Check your server logs for URLs that were visited at least once recently. Pick a time period that makes sense for your site, keeping in mind seasonal variation of traffic.

• Include images and videos—Make sure that you include URLs of embedded content in your site move plans: videos, images, JavaScript, and CSS files. These URLs need to be moved in the same way as all other content on your website.


Export All Internal Links for Old Site Using Tools like ScreamingFrog


Pre-Site Move


19


• Review SEMRush to check for current ranking urls on the old site. Depending on the client’s goals, and plans for content merging, you’ll want to add the new keywords to your SEO strategy, and map out proper URL Redirects.


Use SEMRush to Identify Ranking Pages


Pre-Site Move


20

‣Keyword Strategy

• Once you have a revised keywords strategy in place, you’ll want to setup Ranking tracking for the new domain using tools like BrightEdge or ClientSEOReports. This will allow you to monitor keyword trends pre and post site migration.

Note: Do not delete keyword ranking tracking on old domain for post migration analysis purposes.



Pre-Site Move


21

‣Create a Mapping of Old to New URLs

Once you have the listing of old URLs, decide where each one should redirect to. How you store this mapping depends on your servers and the site move. You might use a database, or configure some URL rewriting rules on your system for common redirect patterns.


Map Old URLs to New URLs


Pre-Site Move


22

‣Update all URL Details

Once you have your URL mapping defined, you'll want to do three things to get the final URL mappings ready for the move.1. Update annotations in the HTML or sitemaps entry for each page:

a. Each destination URL should have a self-referencing rel="canonical" meta tag. b. If the site you moved has multilingual or multinational pages annotated using rel-alternate-hreflang annotations, be sure to

update the annotations to use the new URLs.c. If the site you moved has a mobile counterpart, make sure you update the rel-alternate-media annotations. Learn more in

our smartphone websites guidelines.2.Update internal links

Change the internal links on the new site from the old URLs to the new URLs. You can use the mapping generated earlier to help find and update the links as needed.


Update Internal Links






https://developers.google.com/webmasters/smartphone-sites/details#separateurls




Pre-Site Move


23

‣Update all URL Details

3. Create and save sitemap and link listsSave the following lists for your final move:

a. A sitemap file containing the new URLs in the mappingb. A sitemap file containing the old URLs in the mappingc. A list of sites linking to your current content

4. Learn more about sitemaps.


Save Two Sitemaps: with New URLs, and with Old URLs


https://support.google.com/webmasters/topic/4581190?ref_topic=4581352

https://support.google.com/webmasters/topic/4581190?ref_topic=4581352

Pre-Site Move


24

‣Prepare for 301 Redirects

Once you have a mapping and your new site is ready, the next step is to set up HTTP 301 redirects on your server from the old URLs to the new URLs as you indicated in your mapping.

Keep in mind the following:

• Use HTTP 301 redirects. Although Googlebot supports several kinds of redirects, we recommend you use HTTP 301 redirects if possible.

• Avoid chaining redirects. While Googlebot and browsers can follow a "chain" of multiple redirects (e.g., Page 1 > Page 2 > Page 3), we advise redirecting to the final destination. If this is not possible, keep the number of redirects in the chain low, ideally no more than 3 and fewer than 5. Chaining redirects adds latency for users, and not all browsers support long redirect chains.

• Test the redirects. You can use Fetch as Google for testing individual URLs or command line tools or scripts to test large numbers or URLs.


Use Fetch As Google to Test Redirects


https://webmasters/answer/93633

https://webmasters/answer/93633



During Site Move


25

Move all URLs at once. We advise you move all URLs on your site simultaneously instead of moving one section at a time. This helps users interact with the site better in its new form, and helps our algorithms detect the site move and update our index faster.

3. Start the Site Move


During Site Move


26

‣Initiate the Site Move

1. On the source site, remove all robots.txt directives.This allows Googlebot to discover all redirects to the new site and update our index.

2. On the destination site, make sure the robots.txt file allows all crawling.This includes crawling of images, CSS, JavaScript, and other page assets, apart from the URLs you are certain you do not want crawled.

3. Submit a Change of Address in Webmaster Tools for the old site. No need to submit a change of address if you are only moving your site from HTTP to HTTPS.


Change of Address Tool Only Works if You’re Migrating from Root to New Root Domain




During Site Move


27

‣Initiate the Site Move

4. Configure the old website to redirect users and Googlebot to the new site based on the URL mapping.5. On the destination site, submit the two sitemaps you prepared previously containing the old and new URLs.This helps our crawlers discover the redirects from the old URLs to the new URLs, and facilitates the site move. 6. Keep the redirects for as long as possible, and consider keeping them indefinitely.

The time it takes Googlebot and our systems to discover and process all URLs in the site move depends on how fast your servers are and how many URLs are involved. As a general rule, a medium-sized website can take a few weeks for most pages to move, and larger sites take longer. The speed at which Googlebot and our systems discover and process moved URLs depends the number of URLs and the server speed.

Note that the visibility of your content in web search may fluctuate temporarily during the move. This is normal and a site’s rankings will settle down over time.



During Site Move


28

‣Update Incoming Links

Immediately after the site move is started, try to update as many incoming links as possible. These include:

External links: Try to contact the sites in the saved list of sites linking to your current content, asking them to update their links to your new site.

Profile links such as from Google+, Facebook, Twitter, and LinkedIn.

Ad campaigns to point to the new landing pages.


Backlinks Report on Old Site


Post-Site Move


29

Many features of Webmaster Tools help you monitor a site move, including:

• Sitemaps: Submit the two sitemaps you saved earlier from the mapping. Initially, the sitemap containing the new URLs would have zero pages indexed, while the sitemap of the old URLs would have many pages indexed. Over time the number of pages indexed from the old URLs sitemap would drop to zero with a corresponding increase of indexing of the new URLs.

• Index Status: The graphs would reflect the site move, showing a drop in indexed URL counts on the old site and an increase of indexing on the new site.

• Crawl errors reports: Check regularly for any unexpected crawl errors.

• Search queries: As more pages of the new site get indexed and start ranking, the search queries reports would start showing the URLs on the new site getting search impressions and clicks.

4. Monitor the Traffic

‣Use Webmaster Tools to Monitor Traffic

Review Search Queries Reports in GWT for New URLs








Post-Site Move


30

‣Use other tools to monitor traffic & rankings

4. Monitor the Traffic

Keep an eye on your server access and error logs, checking in particular for crawling by Googlebot, any URLs that unexpectedly return HTTP error status codes, and, normal user traffic.

If you have installed any web analytics software on your site, or if your CMS provides analytics, it's also recommended you review traffic this way so that you can see the progress of traffic from your old to new site. In particular, Google Analytics offers real-time reporting, and this is a handy feature to use during the initial site move phase. You should expect to see traffic drop on the old site and rise on the new site. You’ll also want to review the keyword rankings on old and new site.

Monitor Traffic & Rankings on New Site


Summary


Summary

Mariel MartinezSEO Manager

[email protected]

310.321.4095

Thank You

32Thursday, October 30, 14

Marketing

Site Migrations with URL Changes