Technical Report FabricPool Best Practices - NetApp ?· Technical Report FabricPool Best Practices ONTAP…

  • Published on
    29-Jul-2018

  • View
    212

  • Download
    0

Embed Size (px)

Transcript

  • Technical Report

    FabricPool Best Practices ONTAP 9.4 John Lantz, NetApp

    July 2018 | TR-4598

    Abstract

    This technical report describes best practices for the NetApp ONTAP software component

    FabricPool. The capabilities, requirements, implementation, and best practices of this software

    are covered.

  • 2 TR-4598: FabricPool Best Practices in ONTAP 9.4 2018 NetApp, Inc. All rights reserved. Public

    TABLE OF CONTENTS

    Overview ...................................................................................................................................................... 3

    1 Primary Use Cases ............................................................................................................................... 4

    1.1 Reclaim Space on Primary Storage (Snapshot-Only, Auto) ............................................................................5

    1.2 Shrink Secondary Storage Footprint (Backup) ................................................................................................7

    1.3 Move Entire Volumes to the Cloud (Backup plus Volume Move) ....................................................................8

    2 Requirements ........................................................................................................................................ 9

    2.1 Platforms .........................................................................................................................................................9

    2.2 Intercluster LIFs ..............................................................................................................................................9

    2.3 Volumes ..........................................................................................................................................................9

    2.4 FabricPool License ....................................................................................................................................... 10

    2.5 Certificate Authority Certification ................................................................................................................... 13

    3 Architecture ......................................................................................................................................... 14

    3.1 Block Temperature ........................................................................................................................................ 14

    3.2 Data Movement ............................................................................................................................................. 14

    3.3 Object Storage .............................................................................................................................................. 18

    4 Configuration ...................................................................................................................................... 20

    4.1 Create a Bucket/Container ............................................................................................................................ 20

    4.2 Add an External Capacity Tier to ONTAP ..................................................................................................... 25

    4.3 Attach an External Capacity Tier to an Aggregate ........................................................................................ 27

    4.4 Set Volume Tiering Policies .......................................................................................................................... 29

    4.5 Security ......................................................................................................................................................... 31

    5 Interoperability .................................................................................................................................... 31

    6 Performance ........................................................................................................................................ 33

    6.1 Sizing ............................................................................................................................................................ 33

    6.2 Connectivity .................................................................................................................................................. 34

    6.3 Capacity ........................................................................................................................................................ 35

    6.4 NetApp Private Storage for AWS .................................................................................................................. 39

    Where to Find Additional Information .................................................................................................... 40

    Version History ......................................................................................................................................... 41

    Contact Us ................................................................................................................................................. 41

  • 3 TR-4598: FabricPool Best Practices in ONTAP 9.4 2018 NetApp, Inc. All rights reserved. Public

    Overview

    FabricPool, first available in ONTAP 9.2, is a NetApp Data Fabric technology that enables automated

    tiering of data to low-cost object storage tiers either on or off premises.

    Unlike manual tiering solutions, FabricPool reduces total cost of ownership by automating the tiering of

    data to lower the cost of storage. It delivers the benefits of cloud economics by tiering to public clouds

    such as Amazon S3 and Microsoft Azure Blob Storage as well as to private clouds created with NetApp

    StorageGRID Webscale.

    FabricPool is transparent to applications and allows enterprises to take advantage of cloud economics

    without sacrificing performance or having to rearchitect solutions to leverage storage efficiency.

    ONTAP supports FabricPool on AFF systems and all-SSD aggregates on FAS systems.

    ONTAP Select supports FabricPool. NetApp recommends using all-SSD FabricPool aggregates.

    Cloud Volumes ONTAP for AWS supports FabricPool on gp2, io1, and st1 EBS volumes.

    Cloud Volumes ONTAP for Azure supports

    Figure 1) Before and after FabricPool.

  • 4 TR-4598: FabricPool Best Practices in ONTAP 9.4 2018 NetApp, Inc. All rights reserved. Public

    1 Primary Use Cases

    The primary purpose of FabricPool is to reduce storage footprints and associated costs. Active data

    remains on high-performance SSDs; inactive data is tiered to low-cost object storage while preserving

    ONTAP data efficiencies.

    FabricPool has three primary use cases in ONTAP 9.4:

    Reclaiming space on primary storage

    Shrinking secondary storage

    Moving volumes to external capacity tiers

    Although FabricPool can significantly reduce storage footprints, it is not a backup solution. WAFL (Write

    Anywhere File Layout) metadata always stays on the performance tier. If a catastrophic disaster destroys

    the performance tier, a new environment cannot be created using the data on the capacity tier because it

    contains no WAFL metadata.

    For complete protection that includes WAFL metadata, consider using existing ONTAP technologies like

    SnapMirror and SnapVault. FabricPool can be used to reduce costs by tiering data on secondary storage

    as well.

    https://www.netapp.com/us/media/tr-4015.pdfhttps://www.netapp.com/us/media/tr-4183.pdf

  • 5 TR-4598: FabricPool Best Practices in ONTAP 9.4 2018 NetApp, Inc. All rights reserved. Public

    1.1 Reclaim Space on Primary Storage (Snapshot-Only, Auto)

    Snapshot-Only Tiering Policy

    Snapshot copies can frequently consume more than 10% of a typical storage environment. Although

    essential for data protection and disaster recovery, these point-in-time copies are rarely used and are an

    inefficient use of high-performance SSDs.

    Snapshot-Only, the default volume tiering policy for FabricPool, is an easy way to reclaim storage space

    on SSDs. When configured to use this policy, cold Snapshot blocks in the volume that are not shared with

    the active file system are moved to the capacity tier. If read, cold data blocks on the capacity tier become

    hot and are moved to the performance tier.

    Note: The FabricPool Snapshot-Only volume tiering policy reduces the amount of storage used by Snapshot copies on SSDs. It does not increase the maximum number of Snapshot copies allowed by ONTAP, which remains 1,023.

    Figure 2) Reclaiming space with the Snapshot-Only volume tiering policy.

  • 6 TR-4598: FabricPool Best Practices in ONTAP 9.4 2018 NetApp, Inc. All rights reserved. Public

    Auto Tiering Policy

    The majority of inactive (cold) data in storage environments is associated with unstructured data,

    accounting for more than 50% of total storage capacity in many storage environments.

    Infrequently accessed data associated with productivity software, completed projects, and old datasets is

    an inefficient use of high-performance SSDs, and tiering this data to a low-cost object store is an easy

    way to reclaim existing SSD capacity and reduce the amount of required SSD capacity moving forward.

    First available in ONTAP 9.4, the Auto volume tiering policy moves all cold blocks in the volume, not just

    blocks associated with Snapshot copies, to the capacity tier.

    If read by random reads, cold data blocks on the capacity tier become hot and are moved to the

    performance tier. If read by sequential reads such as those associated with index and antivirus scans,

    cold data blocks on the capacity tier stay cold and are not written to the performance tier.

    Figure 3) Reclaiming space with the Auto volume tiering policy.

  • 7 TR-4598: FabricPool Best Practices in ONTAP 9.4 2018 NetApp, Inc. All rights reserved. Public

    1.2 Shrink Secondary Storage Footprint (Backup)

    Secondary data (data protection volumes that are NetApp SnapMirror [disaster recovery] or NetApp

    SnapVault [backup] destination targets) is frequently stored on secondary clusters that share a 1:1 or

    greater ratio with the primary data that they are protecting (one baseline copy and multiple Snapshot

    copies). For large datasets, this approach can be prohibitively expensive, forcing users to make

    expensive decisions about the data they need to protect.

    Like Snapshot copies, backups are infrequently used, are an inefficient use of high-performance SSDs,

    and are expensive for large datasets even when using HDDs. FabricPools Backup volume tiering policy

    changes this paradigm.

    Instead of 1:1 primary-to-backup ratios, the FabricPool Backup policy allows users to significantly reduce

    the number of disk shelves on their secondary clusters, tiering the majority of the backup data to low-cost

    object stores. WAFL metadata remains on the secondary clusters performance tier.

    If read, cold data blocks in volumes using the Backup policy are not written to the performance tier. This

    reduces the need for high-capacity secondary storage performance tiers.

    Figure 4) Backup use case.

  • 8 TR-4598: FabricPool Best Practices in ONTAP 9.4 2018 NetApp, Inc. All rights reserved. Public

    1.3 Move Entire Volumes to the Cloud (Backup plus Volume Move)

    In addition to Snapshot copies and backups, one of the most common uses of FabricPool is to move data

    to an external capacity tier. Completed projects, legacy reports, or historical recordsany dataset that

    must be retained but is unlikely to be frequently readare ideal candidates to be moved to low-cost

    object storage.

    Moving entire volumes is accomplished by setting the Backup volume tiering policy on a volume when

    initiating a volume move. A volume move is the only time when a backup tiering policy can be set on a

    volume that is not a data protection destination target.

    During the volume move, data (excluding WAFL metadata) is immediately moved to the external capacity

    tier attached to the destination aggregate. After the volume move has completed, the tiering policy on the

    volume automatically reverts from Backup to Auto.

    If read, cold data blocks on the capacity tier become hot and are moved to the performance tier.

    Figure 5) Volume move use case.

  • 9 TR-4598: FabricPool Best Practices in ONTAP 9.4 2018 NetApp, Inc. All rights reserved. Public

    2 Requirements

    FabricPool requires ONTAP 9.2 or later and the use of SSD aggregates on any of the platforms listed in

    the next section. Additional FabricPool requirements depend on the capacity tier being attached.

    Although installation and use of certificate authority (CA) certificates are recommended best practices,

    starting in ONTAP 9.4, installation of CA certificates is not required for StorageGRID Webscale.

    2.1 Platforms

    FabricPool is supported on a variety of platforms:

    AFF

    A800

    A700S, A700

    A300

    A220, A200

    AFF8080, AFF8060, AFF8040

    FAS*

    FAS9000

    FAS8200

    FAS8080, FAS8060, FAS8040

    FAS2750, FAS2720

    FAS2650, FAS2620

    Note: Only SSD aggregates can use FabricPool.

    Cloud Volumes ONTAP for AWS

    Performance tier: gp2, io1, and st1 EBS volumes

    The Auto volume tiering policy is not supported with Standard pricing

    Cloud Volumes ONTAP for Azure

    Capacity tier

    Amazon S3 (Standard and Standard - Infrequent Access)

    AWS GovCloud S3*

    Microsoft Azure Blob Storage (Hot and Cool)

    StorageGRID Webscale 10.3+

    *FabricPool does not currently support FIPS 140-2compliant AWS GovCloud endpoints.

    2.2 Intercluster LIFs

    Cluster high-availability (HA) pairs that use FabricPool require two intercluster LIFs to communicate with

    the external capacity tier. NetApp recommends creating an intercluster LIF on additional HA pairs to

    seamlessly attach external capacity tiers to aggregates on those nodes as well.

    Note: Disabling or deleting an intercluster LIF interrupts communication to the external capacity tier.

    2.3 Volumes

    FabricPool cannot attach an external capacity tier to an aggregate that contains volumes using a space

    guarantee other than None (for example, Volume).

    volume modify -space-guarantee none

  • 10 TR-4598: FabricPool Best Practices in ONTAP 9.4 2018 NetApp, Inc. All rights reserved. Public

    Setting the space-guarantee none parameter assures thin provisioning of the volume. The amount of

    space consumed by volumes with this guarantee type grows as data is added instead of being

    determined by the initial volume size. This approach is essential for FabricPool because the volume must

    support external capacity tier data that becomes hot and is brought back to the performance tier.

    2.4 FabricPool License

    FabricPool requires a capacity-based license when attaching third-party object storage providers (such as

    Amazon S3) as capacity tiers for AFF and FAS hybrid flash systems. A FabricPool license is not required

    when using StorageGRID Webscale as the capacity tier or when using Amazon S3 or Microsoft Azure

    Blob Storage as the capacity tier for Cloud Volumes for ONTAP.

    Tiering to the capacity...