Integrating NetBackup with SwiftStack Object Storage

Embed Size (px)

Text of Integrating NetBackup with SwiftStack Object Storage

  • Runbook:IntegratingNetBackupwithSwiftStack

    IntegratingNetBackup7.7.xwith

    SwiftStackObjectStorage

    June1st,2016

    Page1of42

  • Runbook:IntegratingNetBackupwithSwiftStack

    TableofContents

    TableofContentsIntroductionArchitecture

    ReplicasErasureCodingMultipleRegionsAccount/ContainerServicesStorageUnitSizing

    CertificationSwiftStackSupportwithWindows/LinuxSwiftStackSupportwithUNIX

    PrerequisitesCloudConfigurationPackage

    InstallationVerificationOutput

    DeviceMappingPackageInstallationVerificationOutput

    EnablingSwift3MiddlewareConfiguringStorageUnits

    CloudStorageWizardDefiningStorageServerCredentialsCreatingDiskPoolsCreatingaStorageUnitfromDiskPoolWizardCreatingaStorageUnitCreatingStorageUnitGroups

    TuningBackupConcurrencyMediaServersMasterServers

    MigratingfromexistingstoragepoolstoSwiftStackDuplicatingBackupsIncreasingDuplicateJobSize

    Page2of42

  • Runbook:IntegratingNetBackupwithSwiftStack

    IntroductionForstorageadministratorslookingforamorecostefficient,reliable,scalable,andeasiertousesolutionsforbackup,archivingandrecovery,objectstorageisanexcellentdestinationtarget.Traditionally,backupapplicationsneededdiskstoragetoemulatetapeorbelocallymountedtothemediaserver.VeritasNetBackupversion7.7.1orlatercannownativelyutilizeobjectstorageusingtheS3API.Thistypeofstorageoffersseveralattractivecapabilitiesfortheseworkloads,including:

    Nearlyunlimitedscalabilityofcapacitywithoutreducingperformance.Nomorerunningintohardcapacitylimitationsofastoragesilo,forcingyoutorecycleyourbackupdataand/ormanageitacrossmanynamespaces.

    Automaticdisasterrecoveryprotectionbyreplicatingdataoffsite.Clusternodescanexistinmultiplephysicallocations,soyoudonotneedtoworryabouthaulingmediatothemountainanymore.

    Costefficienthardwareinfrastructurebasedonstandardservers,Ethernetbasednetworking,andaffordablediskdrivesratherthanonmoreexpensiveproprietarysolutions.Thatstheadvantageofsoftwaredefinedstorage.

    ThisrunbookwillwalkyouthroughthespecificstepsyouneedtotaketooptimallyconfigureVeritasNetBackuptoutilizeaSwiftStackobjectstorageclusterfornearlinebackup.Ifyouhavequestionswhendesigning,testing,ordeployingSwiftStackwithNetBackup,pleasefeelfreetocontactus,asweareheretohelp.

    Page3of42

    https://www.swiftstack.com/contact-us/

  • Runbook:IntegratingNetBackupwithSwiftStack

    Architecture

    ArchitectingaSwiftStackstoragecloudiseasyandreferencehardwareexamplesareavailable.Inthiscase,SwiftStackwillbeusedasadirecttargetforNetBackupMediaservers.SwiftStackstoragecanbedeployedalongsideyourexistingbackuptargetsorreplaceexistingtargetssuchasNAS,DAS,PBBAsoreventape.Startingsystemsconsistofaminimumofthreenodestoensuredataavailabilityanddurability.Youroverallperformancecanalsoimprove,asallnodesintheclustercanbedirectlyaddressedforbothbackupsandrestores.SwiftStackisascaleoutstoragesystemandallnodesshareacommonnamespace.Havingacommonnamespaceeliminatestheneedforsilosthatcomeswithdirect

    Page4of42

  • Runbook:IntegratingNetBackupwithSwiftStack

    attachedstorage,SANsorfilersthatdontscale.EachnodeinaSwiftStackclusterisactiveandperformancescaleslinearlyasmorestorageserversareadded.Whilenotaddressedinthisrunbook,SwiftStackclustersalsohavetheabilitytospanmultiplesitesorregions.Unliketypicalstoragesystems,nodesatothersitesareactive,sharethesamenamespace,andhandlecrosssitereplicationofdataautomatically.ThismakesSwiftStackunlikeanyothersolutiononthemarketwithitsabilitytoscaleoutwithoutboundariesandautomaticallyprotectyourvitalbusinessdatafromamajordisaster.

    ReplicasWithlessthan5nodes,SwiftStackdatawillalwaysbeprotectedusingreplication.Replicationhappensautomatically.Usingreplicaswiththe3nodesolutionreferencedabovewillallowSwiftStacksurvivethelossofanentirenodeormultipledrivefailures.Areplicapolicymeansthateverypieceofdatagiventotheobjectstorageiscopiedmultipletimesanddistributedacrossmanynodes.Whilereplicasincreasestheamountofrawstorageaclustermusthave,ithasanumberofimportantbenefits.

    1. Increasedavailabilityofdataevenifnetworksplitbrain2. Fastestread/writeperformance3. Geographicdistributionofdatawithhighlylatentnetworking4. Lowesttimetofirstbyte

    ErasureCodingAsanalternativetohavingfullreplicasofeachobject,erasurecodingisanotherdataprotectionmethodthatismorespaceefficient.WhilesimilartoRAID,erasurecodingprotectsatanobjectlevel,whereRAIDprotectsacollectionsofdrives,whereasthelossofmorethanonedrivecanresultinacompletelossofthecollectionsorRAIDgroup.ErasurecodingisafeatureincludedwithSwiftStack,butnotnecessarilyagoodselectionforNetBackup.Erasurecodingrequiresaminimumof5nodestobeenabledsoitismoresuitedforlargeinstallationsofover1PBofdata.InadditionErasureCodeddatacannotbereplicatedbetweenmultipleregions/sitesifthatisrequired.Lastly,ErasureCodingincursaperformancepenaltywhenusedwithNetBackups1MBobjects,cuttingperformanceinhalf.

    Page5of42

  • Runbook:IntegratingNetBackupwithSwiftStack

    MultipleRegionsRegionsaregeographicallydistantareasconnectedbynetworking.Replicatingdataacrossmultipleregionsistheultimatesolutionindisasterrecovery.SwiftStackisabletoscaleout,storingcopiesofdatainallregionsorjustselectedregions.Dataplacementwithinregionsiscontrolledbystoragepoliciesandreplicationhappensautomaticallywithouttheneedforschedules,snapshotsorothercomplexity.Storagepoliciesaredefinedbythestorageadministrator,butselectablebyusersandapplicationsonaperbucketbasis.Allregionsareactive,abletodistributedataandconsumedatawithnoregionneedingtobereadonly.Thesamenamespaceiscommontoallregions.Thisallowsmediaserversinallregionstoperformbackupsandrestoreslocally,thusreducingcomplexity.Abackupdoneinregion1canberestoredinregion2byadifferentmediaserver.ThismediaserverwilladdressthelocalSwiftStackstorageandtherestorewillproceed.Ifallofthedatahasnotreplicatedoverthewire,thelocalregionwillpullthedataasneeded.

    Account/ContainerServicesAccount/ContainerServicesinSwiftStackprovidelistinginformationforcontainers(alsoknownasbuckets).Eachcontainerhasitsowndatabasetokeeptrackofobjectlistings.Containerperformancedropsaslargenumbersofobjectsarestoredinasinglecontainer.ForthisreasonSwiftStackrecommendsassigningbackuppoliciestoStorageUnitGroupsortoseparateStorageUnitsifuseofAcceleratorisrequired. Account/ContainerServicesshouldalwaysbeassignedtoSSDsandnottoHDDsforperformance.

    StorageUnitSizingAsnotedinthepreviousheadertheamountofdatastoredinaStorageUnitshouldbelimitedtoensureContainerperformance.TherecommendedamountofdatatobestoredperStorageUnitis50TBorless.ThenumberofobjectsinaContainerrunningonSSDwillstillbequiteperformantat50TB.CalculatingnumberofobjectsinastorageunitSinceNetBackupobjectsare1MBinsize,50TBofdataequatestojustover50millionobjects.AsnotedinAccount/ContainerServices,performancedropswiththeincreaseinnumberofobjectsinacontainer.With50millionobjectsinacontainer,thatcontainerhasbeentestedtoingestdataat200MB/sec.

    Page6of42

  • Runbook:IntegratingNetBackupwithSwiftStack

    CertificationSwiftStackObjectStorageistestedandcertifiedbyVertiasforNetBackupversions7.7.1andabove.TheofficialHardwareCompatibilityLists(HCL)forNetBackupcanbefoundatthefollowinglink:https://www.veritas.com/support/en_US/article.TECH59978

    SwiftStack Support with Windows / Linux Cloud

    Storage Solution

    NetBackup Version Containing Plugin or VendorSupplied

    Plugin Version

    Supported OpenStorage Functionality

    Windows Server

    2008 x64

    Windows Server

    2012 x64

    Red Hat Enterprise Linux on x8664

    SUSE Linux Enterprise Server

    on x8664

    Object Storage [1][2][3]

    NetBackup 7.7.1 and later Accelerator, Accel_VMware, KMS, Opt_Synth

    Enterprise Edition

    Datacenter 6, 7 11, 12

    1. Creating a disk volume via the NetBackup Admin Console or CLI is not supported with this solution. Create the bucket outside of NetBackup using the cloud provider's interface.

    2. Requires Cloud Configuration Package version 2.1.2 or later. 3. This is an S3 APIcompatible solution.

    SwiftStack Support with UNIX Cloud Storage

    Solution NetBackup Version Containing Plugin or

    VendorSupplied Plugin Version Supported OpenStorage

    Functionality AIX on POWER

    HPUX on IA64

    Solaris on SPARC

    Object Storage [1][2][3]

    NetBackup 7.7.1 and later Accelerator, Accel_VMware, KMS, Opt_Synth

    6.1, 7.1 11.31 10, 11

    1. Creating a disk volume via the NetBackup Admin Console or CLI is not supported with this solution. Create the bucket outside of NetBackup using the cloud provider's interface.

    2. Requires Cloud Configuration Package version 2.1.2 or later. 3. This is an S3 APIcompatible solution.

    Page7of42

    https://www.veritas.com/support/en_US/article.TECH59978https://download.veritas.com/resources/content/live/SFDC/25000/000025228/en_US/nbu_77_hcl.html?__gda__=1463579767_89cb11c3285c0c478de1174d2082f9f7#f45-3https://download.veritas.com/resources/content/live/SFDC/25000/000025228/en_US/nbu_77_hcl.html?__gda__=1463579767_89cb11c3285c0c478de1174d2082f9f7#ost_secondary_storage_solutions-ost_cloud-ost_cloud_-_vendor_compatibility-swiftstack

  • Runbook:IntegratingNetBackupwithSwiftStack

    PrerequisitesAsnotedbytheVeritasHCL,NetBackupversions7.7.1&7.7.2require:

    CloudConfigurationPackageversion2.1.2orlater DeviceMappingFilePackage1.128orlater Swift3MiddlewareenabledforSwiftStack

    CloudConfigurationPackageAtthetimeofthiswritingthecurrentCloudConfigurationPackagewas2.1.3andcanbefoundatthefollowingsite:https://www.veritas.com/support/en_US/article.000097618ThemostuptodatepackagecanalwaysbefoundintheNetBackupHCLreferencedintheprevioussection.

    InstallationThefollowingneedstobeperformedoneachMasterServer:

    1.CopytheexistingCloudProvider.xmlandCloudInstance.xmlfilefromthefollowinglocationtoanalternateloca