Avamar Staging Tape Out - uk.emc.com · PDF fileMonitoring Tape Out Progress ... Appendix B — tapeout.pl Script ... a spare node). Besides hardware specifications

T E C H N I C A L N O T E S

AVAMAR STAGING TAPE OUT:OUTPUTTING AVAMAR BACKUPS TO TAPE

AVAMAR 4.X AND LATER SOFTWARE

TECHNICAL NOTEP/N 300-007-419

REV A04MAY 7, 2012

Table of Contents

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Applications Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Tape Out Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Monitoring Tape Out Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Crontab Automation of selectfortape.pl Application . . . . . . . . . . . . . . . . . . 31

Appendix A — selectfortape.pl Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Appendix B — tapeout.pl Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

1

Overview

OverviewThis document describes how to select specific Avamar backups and export them to tape using a redirected restore or "staging" server.

Exporting to tape consists of the following steps:

1. Run the selectfortape.pl script to select backups that you want to export to tape from the Avamar server database.

This script generates an output list of selected backups on the Avamar server. The output list is based on user-defined criteria.

2. Run the tapeout.pl script on the staging server to restore the selected data to the local hard drive and then export it to tape.

This script uses the output list generated by selectfortape.pl to perform a restore of the data to the staging server. This occurs until it reaches the end of the output list or when the hard drive fills up on the staging server.

The installation location of the two main applications (selectfortape.pl and tapeout.pl) determines on which servers each process occurs.

AVAMAR STAGING TAPE OUT: OUTPUTTING AVAMAR BACKUPS TO TAPE TECHNICAL NOTE 2

Overview

tapeout.pl Modes

When tapeout.pl begins exporting to tape, it runs in one of two sets of modes. You configure these modes through tapeout.pl command line options.

Scripted, Manual Modes

Nonincremental, Incremental Modes

The remainder of this document is organized as follows:

• Hardware and software requirements• Application setup• Tape out procedures• Monitoring and automation instructions• Script reference information

MODE DESCRIPTION

Designated tape dump script Uses a custom script for exporting backups in the staging directory to tape. Template scripts are available to modify for specific systems (for example, NetWorker). This is the default mode.

Manual Provides the means for you to manually backup the staging directory to tape.

MODE DESCRIPTION

Nonincremental mode After the complete contents of the staging area is exported to tape, the data is deleted.

Incremental mode Additively restores to the destination incrementally, thus reducing restoration times and allowing more frequent staging. When restoring, it creates an /INCREMENTAL directory in stagingdir.IMPORTANT: "Additive" means that incremental restores export only new and changed files from previous backups. Incremental backups do not delete files from the staging server and tape that have been deleted from clients.


Requirements

RequirementsThis section outlines the hardware and software requirements that servers in a tape export setup must meet.

Source Avamar Server

All backups chosen for tape export are initially stored on the Avamar node (single-node servers) or utility node (multi-node servers).

Hardware Requirements

There are no specific hardware requirements beyond the standard bill of materials for an Avamar server.

Software Requirements• Avamar server version 4.x or later

This software must already be installed and running for valid backups to exist.

• Perl 5.8.0 or later

This software is installed with the standard operating system.

• Perl-DBI module

This application is installed with the standard operating system.

• Perl-DBD-PgPP module

Refer to Applications Setup (page 7) for instructions about this module.

• selectfortape.pl and selectfortape_interactive.pl

These are Perl scripts.

Environment Requirements• Replication or source target

The Avamar server can be either a source or, preferably, a replication target. If it is a source target, EMC strongly recommends, for performance reasons, that Avamar capacity not exceed 40% or that incremental mode be utilized.

• Retention policies

If a specific kind of backup is being archived (for example, a monthly backup), you should use advanced retention policies. To reliably select monthly backups, monthly retention policies should be different from daily or weekly ones.


Requirements

Staging Server

EMC recommends that the storage utilized for staging activity is dedicated for staging purposes during operation and is on a separate storage space outside of the local operating system files. When running in nonincremental mode, the staging directory will be deleted by tapeout.pl upon completion as described in Overview (page 2) and Requirements (page 4).

Hardware Requirements

Nonincremental mode. A staging server can be any machine similar to any Avamar client. Although not preferred, it can even be an Avamar server (for instance, a spare node). Besides hardware specifications similar to any Avamar client, an Avamar staging server must have at least 20% more storage available than the largest backup chosen (as reported by Avamar Administrator). This requirement conservatively handles the likelihood that the data is larger on an actual file system due to file system block sizes and so forth.

Incremental mode. In incremental mode, the Avamar staging server should have enough backend storage to accommodate all backups to be exported to tape at the time of export. Unlike nonincremental mode, incremental mode does not delete data and manage the storage of the staging server. The storage requirements increase as more data is added to the Avamar server.

Software Requirements

Software requirements for a staging server are different from other nodes because cross-platform restores may not be supported depending on the backup product used. Both Microsoft Windows and Red Hat Enterprise Linux are supported.

Microsoft Windows Requirements

• Microsoft Windows 2000, XP, 2003 or 2008• Avamar client of the same version as the Avamar server.

Refer to the Avamar Backup Clients Installation Guide and Reference Manual for instructions, which is available on the Avamar server Documents and Downloads web page.

• ActivePerl 5.8.8 or later

ActivePerl is available at: http://www.activeperl.com.

• tapeout.pl program• avtar.cmd file• Supported version of tape backup software (for instance, the Networker agent)


Requirements

Linux Requirements

• Red Hat Enterprise Linux v3u8 or v4u6• Avamar client of the same version as the Avamar server.

Refer to the Avamar Backup Clients Installation Guide and Reference Manual for instructions, which is available on the Avamar server Documents and Downloads web page.

• Perl 5.8• tapeout.pl program• avtar.cmd • Supported version of tape backup software (for instance, the Networker agent)

Limitations

Although supported by Avamar and certain tape backup programs, cross-platform restores are not recommended for exporting to tape. When performing restores, Windows backups should return to Windows staging servers and the same with non-Windows backups. To maintain the same operating system, use the --osinclude or --osexclude options when running the tapeout.pl script.

If a single platform is used as the staging server to output to tape backups for multi-platform clients (for example, a Linux staging server is used to stage data for both Linux and Windows clients), the following should be noted: In nonincremental mode, deleting exported files and directories might be difficult after backup to tape. In that case, tapeout.pl attempts to move the directory structures that cannot be fully deleted to a renamed directory called [STAGING_DIRECTORY]_could_not_delete. Also, although data will be properly exported to tape, ACLs and permissions most likely will not be properly exported for the data from clients that do not match the operating system of the staging server.


Applications Setup

Applications SetupBefore performing export to tape procedures, you must set up the applications on the correct Avamar servers. The selectfortape.pl application must be set up on the source Avamar server and the tapeout.pl application and, if necessary, the tape dump script must be set up on the Linux or Windows staging server.

selectfortape.pl on Source Avamar Server

selectfortape.pl is an application that queries the Avamar server's backup database and selects backups that meet certain user-defined restrictions, including the following:

For multi-node servers, selectfortape.pl must be run on the Avamar utility node, and the following option is required:

--snapupoutput=PATH

Where PATH is the name of an Avamar client on the Avamar server that can house the newly created backup lists. The machine should be a new one created through the Avamar Management Console.

Refer to Appendix A — selectfortape.pl Script (page 32) for complete command instructions.

The following files are required for running selectfortape.pl on the source Avamar server:

• Perl-DBD-PgPP.rpm • selectfortape.pl • selectfortape_interactive.pl (only required for interactive mode)

RESTRICTION DESCRIPTION

Start date Two parameters that form a date range between which all successful backups must have been started and completed.End date

Minimum retention The minimum retention on the backup.Values that exceed this minimum value are selected.

Include The inclusion of certain domains or clients./REPLICATE is included by default.

Exclude The exclusion of domains or clients./MC_BACKUPS is excluded by default.To do a mutual exclusion, exclude "/" before doing includes.


Applications Setup

Setting up selectfortape.pl

The following instructions assume that the source Avamar server has been configured properly according to Avamar software installation guidelines.

Using winscp, ftp or other transfer mechanism, place files and applications of importance in some directory. Instructions that follow use the sample directory /tmp.

IMPORTANT: All commands must be run as the root user.

1. Install the perl-DBD-PgPP module rpm by entering:

rpm -ivh perl-DBD-PgPP-VERSION.rpm

Where VERSION is the latest version of the module.

2. Create a symlink from /usr/lib/perl5/site_perl/5.6.1/DBD.

• For Red Hat 3, enter:

cd /usr/lib/perl5/5.8.0/i386-linux-thread-multi

ln -s /usr/lib/perl5/site_perl/5.6.1/DBD .

• For Red Hat 4, enter:

cd /usr/lib/perl5/5.8.5/

ln -s /usr/lib/perl5/site_perl/5.6.1/DBD .

3. Move selectfortape.pl to /usr/local/avamar/bin by entering:

mv selectfortape.pl /usr/local/avamar/bin/

4. Make selectfortape.pl executable (change its permissions) by entering:

chmod a+rx /usr/local/avamar/bin/selectfortape.pl

5. For selectfortape_interactive.pl, move and change permissions by entering:

mv selectfortape_interactive.pl /usr/local/avamar/bin/

chmod a+rx /usr/local/avamar/bin/selectfortape_interactive.pl


Applications Setup

tapeout.pl on Avamar Staging Server

The tapeout.pl application receives output generated by selectfortape.pl. The mode of export to tape you used determines the options required for tapeout.pl.

When running the tapeout.pl application, the following arguments are required:

NOTE: Either the manual or tapescript option is required.

Refer to Appendix B — tapeout.pl Script (page 36) for complete command instructions.

Setting Up tapeout.pl on Linux Staging Server

The Linux staging server should be set up like an Avamar data or spare node with enough external storage to hold approximately 20% more space than the largest backup on the Avamar server. The additional space must exist on a single partition or mount point.

Using winscp, ftp or other transfer mechanism, place the files and applications of importance in some directory. Instructions that follow use the sample directory /tmp.

IMPORTANT: All commands must be run as the root user,or the user should have root level privileges.

The following files are required for running tapeout.pl on a Linux staging server:

• AvamarClient-VERSION.rpm • tapeout.pl

Because tapeout.pl does not perform database queries, no additional Perl modules are required.

ARGUMENT DESCRIPTION

--stage Instructs tapeout.pl to enter staging mode.

--stagingdir=[DIR] The directory to be used for the redirected restores.

--tapescript=[SCRIPTPATHNAME] The path to the tape script that is called to export the backups to tape when the backups have all been restored to the staging server.

--manual Prompts you to export backups to tape manually before proceeding with additional restores.


Applications Setup

The following instructions describe how to ensure that the export to tape process runs successfully:

1. Place tapeout.pl in the Avamar software path.

(a) Move the program tapeout.pl to /usr/local/avamar/bin and change its permissions so that it is executable by entering:

mv tapeout.pl /usr/local/avamar/bin

chmod a+rx /usr/local/avamar/bin/tapeout.pl

IMPORTANT: The tapeout.pl program creates files suchas logs and staging files in the directory the user runs tape-out.pl. This also applies to when tapeout.pl is run from anautomated script.

2. Perform a test avtar command to ensure that authentication is working properly by entering:

avtar --archives --path=/MC_BACKUPS

If the avtar.cmd file has been configured correctly, the command should return a list of the backups under /MC_BACKUPS. Every Avamar server has a /MC_BACKUPS directory that contains Enterprise Management Console flushes.

Setting Up tapeout.pl on a Windows Avamar Staging Server

In nonincremental mode, the following describes how to configure a Windows-based Avamar staging server. The server should have 20% more capacity than the largest backup on the Avamar server. The additional space must reside on a single partition or mount point.

Using winscp, ftp or other transfer mechanism, place the files and applications of importance in some directory. Instructions that follow use the sample directory /tmp.

IMPORTANT: All commands must be run as domain-leveladministrator.

The following files are required for running tapeout.pl on a Windows staging server:

• ActivePerl5.8 • AvamarClient-VERSION.exe • tapeout.pl • avtar.cmd

The following instructions describe how to ensure that the export to tape process runs successfully:

1. Create a working directory and place tapeout.pl in that directory.

2. Place avtar.cmd in c:\Program Files\avs\var.


Applications Setup

3. Edit avtar.cmd with the following user-specific information:

• --server=SERVERNAME

Where SERVERNAME is the name or IP address of the Avamar server that contains the backups to be exported.

• --id=USER

Where USER is the user name of an Avamar root-level administrator.

• --password=PASSWORD

Where PASSWORD is the password of the previously-mentioned Avamar user.

4. From a command prompt, perform a test avtar command to ensure that authentication is working properly by entering:

c:\Program Files\avs\bin\avtar.exe --archives --path=/MC_BACKUPS

If the avtar.cmd file has been configured correctly, the command should return a list of the backups under /MC_BACKUPS. Every Avamar server has a /MC_BACKUPS directory that contains Enterprise Management Console flushes.

Tape Dump Script

Except when manual mode is chosen, tapeout.pl calls an additional script that exports to tape the backups that have been restored to the staging server.

The following template scripts are available and can be used or modified to do tape backups:

• tapedump_networker.pl

For NetWorker (7.3.2).

• tapedump_netbackup.pl

For NetBackup (5.0, 5.1 or 6.0).

• tapedump_backupexec.pl

For Backup Exec (11d).

• tapedump_tsm.pl

For TSM. Refer to Additional Instructions for Avamar Tape Out Staging Method for TSM (page 13) for additional information.

These templates are very simple, and the degree of modification varies with respect to environmental requirements. They are written in Perl since its interpreter is always resident on the server that tapeout.pl is run on (tapeout.pl is a Perl program).

Depending on the backup policy configuration for the tape backup program, information including name or incremental characteristics may be necessary to perform a command line backup.


Applications Setup

Scripts can be written in any language as long as these requirements are met:

• No command line parameters allowed

The reason for this is tapeout.pl cannot pass through parameters to the tape dump script. As a result, directories must be modified in the tape dump script.

• Return code 0 for success• Return non-0 (1 by default) for failure

NOTE: Before backing up, a pre-defined tape backup policyfor that particular product must exist for the backups whenthey go to tape.

Example

The following is the content of the NetBackup template script including comments:

#!/usr/bin/perl# $Id: tapedump_netbackup.pl,v 1.1 2007/05/03 16:54:12 ekim Exp $# This is a simple script that performs a backup to tape on a# specified staging directory for Veritas NetBackup and returns the# exit codes. If the user wishes to have a different staging script,# the requirement is that the return code either be 0 for pass and# greater than 0 for not pass.## BEGIN USER VARIABLES# location of restored data during staging. passed into tapeout.pl as# --stagingdir# e.g. $stagedir = 'e:\stagedir';$stagedir = 'STAGEDIR';# NetBackup server# e.g. $server = 'nbu1.avamar.com';$server = 'SERVER';# path of NetBackup bpbackup command# e.g. $savecmd = '"c:\Program Files\VERITAS\NetBackup\bin\bpbackup.exe"';$savecmd = '"c:\Program Files\VERITAS\NetBackup\bin\bpbackup.exe"';# Netbackup policy to use (must be pre-defined)# e.g. $policy = "tapeout";$policy = "POLICY";## END USER VARIABLESprint "Saving $stagedir to $server...\n";# -w flag tells bpbackup to wait for error code

`$savecmd -w -S $server -p $policy $stagedir`;if ( $? == 0 ){ print "Backup to tape completed successfully.\n"; exit (0);} else { print "Backup to tape failed.\n"; exit (1);}


Tape Out Procedures

NOTE: The complexity and functionality of a tape dumpscript depends on environmental constraints.

Additional Instructions for Avamar Tape Out Staging Method for TSM

The tapedump_tsm.pl script uses the tsm command line client (dsmc) to invoke a TSM client "archive" function. This function performs tape out operations to the TSM server using the Avamar tape out staging method.

Several variables embedded in the tapedump_tsm.pl script must be adjusted for any one particular implementation.

Use the following general instructions to prepare for TSM tape out:

1. On the TSM Server, perform the following:

(a) Register the staging server as a new client node.

(b) If necessary, create new management classes and archive copy groups.

2. On the Avamar staging server, do the following:

(a) Install the TSM client software.

(b) Configure the dsm.opt and dsm.sys files with the following information:

• nodename• passwordaccess generate• tcpserveraddress

3. To generate the TSM client password, enter the following from an operating system command prompt:

dsmc q sess

Tape Out Procedures

Running selectfortape.pl

When running the selectfortape.pl application, the following considerations should be noted.

Besides required arguments, the following arguments are useful:

• --nah

• --cod

• --mod

• --include

• --exclude

Refer to Appendix A — selectfortape.pl Script (page 32) for complete descriptions of command line parameters.


Tape Out Procedures

Example 1

selectfortape.pl --start=2007-02-02 --end=2007-02-02 --retention=1 --utility=avamar01 --snapupoutput=/clients/snapupholder

This command produces the following results:

All scheduled backups performed on 2007-02-02.

Backups with no minimum retention. When specifying --retention=1, this means that backups with no end date are counted.

The backups to be searched reside on avamar01.

The backup list is stored in the client name /clients/snapupholder on the server avamar01.

Example 2

selectfortape.pl --start=2006-12-01 --end=2007-06-04 --retention=30 --utility=avamar01 --snapupoutput=/clients/snapupholder --cod --mod


All scheduled, client initiated (cod) and Management Console initiated (mod) backups occurring on any date from 2006-12-01 to 2007-06-04 inclusive.

Backups are of a minimum retention of 30 days.

The backups reside on the Avamar server named avamar01.

The backup list is saved in the client named /clients/snapupholder.

Example 3

selectfortape.pl --start=2006-12-01 --end=2007-06-04 --retention=30 --utility=avamar01 --cod --mod --nah --exclude=/REPLICATE --output=snapups.out


All scheduled, client initiated (cod), Management Console initiated (mod) and naked ad hoc (nah, command line avtar backups) backups except backups in the /REPLICATE domain performed from 2006-12-01 to 2007-06-04 inclusive.



The backup list is saved to a local file named snapups.out.


Tape Out Procedures

Example 4

perl selectfortape.pl --start=2009-01-01 --end=2009-06-30 --retention=30 --utility=avamar01 --snapupoutput=/clients/snapupholder --cod --mod --firstday






Returns the first (earliest) day's worth of backups (for each client) within 2009-01-01 and 2009-06-30.

Example 5

perl selectfortape.pl --start=2009-01-01 --end=2009-06-30 --retention=30 --utility=avamar01 --snapupoutput=/clients/snapupholder --cod --mod --firstbackup --lastbackup






Returns the first backup and last backup for each client within 2009-01-01 and 2009-06-30.

Selectfortape Interactive

The selectfortape.pl script can be run in interactive mode. In this mode, you are prompted each step of the selection process. To run selectfortape.pl in interactive mode, type:

root@aeesx3:~/#: selectfortape.pl --interactive

NOTE: No other options can be used with --interactive.

Interactive mode presents all prompt options in each dialog box. If you don’t use interactive mode, you must refer to Appendix A — selectfortape.pl Script (page 32) or use the --help parameter for a list of options. Also, use interactive mode if you feel more comfortable in a graphical user interface than at the command line.

Interactive mode displays windows described in the following topics, in the order shown. Use Tab to navigate around the interface and Enter to execute highlighted options. In windows that display a list, use up and down arrows to


Tape Out Procedures

highlight items and press the space bar to toggle selection (marked by an X) and deselection.

Upon completion of the following steps, a command line equivalent to a manual operation is displayed. This information is useful it you want to automate the selectfortape procedure for future tape out operations.

Selection - Utility Node

Enter the hostname of the utility node (or Avamar server, if a single node server). Display default is defined by --server=HOSTNAME in the usersettings.cfg file in the following path: /usr/local/avamar/etc.

Selection - Client List Options Filter

Select whether you initially want all clients or no clients (None) selected in the window that follows. For example, if you have 100 clients but want to export to tape just three of them, select None so that all clients appear deselected and you can then select the desired three clients. Likewise, if you want to export to tape 97 of 100 clients, select All and manually deselect the unwanted clients in the client list window.


Tape Out Procedures

Selection - Client List Options

Select the clients you want to export to tape. X indicates the client is selected. To toggle the selection, press the space bar on the highlighted client.


Tape Out Procedures

Selection - Start Date

Type the start date of the window for eligible backups.

Required format: YYYY-MM-DD. Default is 30 days before current date.

Selection - End Date

Type the end date of the window for eligible backups.

Required format: YYYY-MM-DD. Default is current date.


Tape Out Procedures

Selection - Retention

Type the minimum retention period (in days) for eligible backups. Default is 0.

Selection - OS Includes

Select which operating systems should be included in the selection process. X indicates the OS is selected. To toggle the selection, press the space bar on the highlighted OS. Default is all selected.


Tape Out Procedures

Selection - Additional Commands

The commands in the following window provide more granular selections.

NOTE: Scheduled backups are always included

COMMAND DESCRIPTION

nah Includes "naked adhoc" backups (backups done by command line).

cod Includes "client on demand" backups through the client interface.

mod Includes "client on demand" backups through the Avamar Administrator.

firstday Includes the entire first day of backups (for each client) within the selection criteria provided.

firstbackup Includes the first backup (for each client) within the selection criteria provided.

lastday Includes the entire last day of backups (for each client) within the selection criteria provided.

lastbackup Includes the last backup (for each client) within the selection criteria provided.

sun, mon, tue, wed, thu, fri, sat

Includes backups that occurred on the days selected.


Tape Out Procedures

Selection - Running Selection

The following window is displayed as the selectfortape.pl script gathers all backups that meet the selection criteria. To facilitate further investigation, if required, the debug log file location is displayed in the box.

Selection - selectfortape.pl Debug Log

This window contains the debug log file. Use the up and down arrow keys to scroll down and review the log. If all is satisfactory, click OK.


Tape Out Procedures

Selection - Query Results

This window displays all of the backups as determined by your selection criteria. You can exclude backups by deselecting them. This window displays the following:

• Column 1 - determines whether a backup is selected• Column 2 - line number• Column 3 - client• Column 4 - date of backup• Column 5 - client operating system• Column 6 - backup label number

Selection - Use Default User, Password

In this window, select whether to use default settings for user and password, which are displayed. If you click Accept Default, the following two windows do not appear.


Tape Out Procedures

Selection - Upload User ID

Specify the user ID the script will use for the upload of selected backup criteria to the Avamar server. This window does not appear if you accepted default settings in the Use Default User, Password window.


Tape Out Procedures

Selection - User Password

Type the password for the user specified in the previous window. This window does not appear if you accepted default settings in the Use Default User, Password window.

Selection - Avamar Path

Specify the path to an existing client on the Avamar server to which the output file (that contains backup selections) will be uploaded.

IMPORTANT: You must have already created a "fake"(unassigned) client solely for the purpose of storing outputfiles. For example, create a client named "snapupholder" inthe /clients domain. Refer to the Avamar System Adminis-tration Manual for instructions on creating clients.


Tape Out Procedures

Selection - Upload Status

The following window provides upload status. Initially, refer to the header to determine if the upload succeeded or failed. Use the up and down arrows to review the log file and, if the upload failed, to determine why.

If the upload failed, click OK to return to the previous window to reenter a valid path.

If the upload succeeded, click OK to exit the interactive script.

Following is a successful upload example (header displays "upload succeeded").

Selection - Exit Status

The following window provides exit codes and the manual command line equivalent to the operation the interactive mode performed. This information is useful for automating the selectfortape procedure as described in Crontab Automation of selectfortape.pl Application (page 31).


Tape Out Procedures

Running tapeout.pl

When running the tapeout.pl application, the following considerations should be noted.

Besides required arguments, the following arguments are strongly recommended:

--incremental

--onloggedavtarerror=[retry|skip]

--onnewavtarerror=[exit|skip|continue]

--logdir=[PATH]

--stagingfile=[PATHTOFILE]

Refer to Appendix B — tapeout.pl Script (page 36) for complete descriptions of command line parameters.

Example 1

tapeout.pl --stage --stagingdir=/data01/restore_data --tapescript=/tmp/export_to_tape.pl --snapupoutput=/clients/snapupholder


Uses the staging mode.

The directory /data01/restore_data is the target for the re-directed restores.

Uses the script /tmp/export_to_tape.pl to export the backups to tape after tapeout.pl has finished staging data.

Uses the list that was generated by selectfortape.pl that is stored on the Avamar server in /clients/snapupholder.

Example 2

tapeout.pl --stage --stagingdir=/data01/restore_data --tapescript=/tmp/export_to_tape.pl --snapupoutput=/clients/snapupholder --onnewavtarerror=skip --onloggedavtarerror=retry



The directory /data01/restore_data is the target for the redirected restores.

Uses the script /tmp/export_to_tape.pl to export the backups to tape after tapeout.pl has finished staging data.


Every time an error occurs with one of the Avamar restores, marks as failed and skipped and then proceeds to delete the data and continue to the next restore.

Every time there is an existing failure from a previous run, it tries to re-restore and export to tape.


Tape Out Procedures

Example 3

tapeout.pl --stage --stagingdir=/data01/restore_data --manual --snapupoutput=/clients/snapupholder --onnewavtarerror=continue --onloggedavtarerror=skip --stagingfile=/tmp/staging_info --osinclude=windows



The directory /data01/restore_data is the target for the redirected restores.

Instead of having a tape script, the user manually exports to tape the restored data.


Every time an error occurs with one of the Avamar restores, it marks as failed and then proceeds to the next restore without deleting any data.

Every time there is an existing failure from a previous run, it tries to re-restore and export to tape.

Creates the file /tmp/staging_info instead of the default tapeout_staging.txt to contain all of the information about the status information of the restore and tape out. This file acts as a database for the tapeout.pl.

Only restores Microsoft Windows backups to the staging server. Includes and excludes are mutually exclusive and can range from the following choices: unknownfilesystem, linux, solaris, hp-ux, aix, mac, unix, windows or ndmp.


Monitoring Tape Out Progress

Monitoring Tape Out ProgressThe staging file is important to both the tapeout.pl application and the user for tape out progress monitoring purposes. It is used by the application as a status database which, since it is human-readable, can be used to accurately monitor tape out progress.

The staging file is specified by --stagingfile=FILENAME (tapeout_staging.txt, by default). A description of the fields and possible status messages in the file are shown below.

Field Definitions

The fields of the staging file are formatted as:

FULL CLIENT PATH WITH LABELNUM | CLIENT NAME | PLUGIN TYPE | LABEL NUMBER | BACKUP SIZE (BYTES) | BACKUP DATE | RESTORE STATUS | RESTORE DATE/TIME | TAPEOUT STATUS | TAPEOUT DATE/TIME

FULL CLIENT PATH This field shows the full Avamar client path as shown in the Avamar Administrator. This is also the path that is created in the staging directory.

CLIENT NAME This field shows the name of the client.

OS This field shows the operating system of the client. Valid OS types are the following:

• unknownfilesystem• linux• solaris• hp-ux• aix• mac• unix• windows• ndmp

LABEL NUM This field shows the backup label number as it is recorded in the Avamar server.

BACKUP SIZE This field shows the size of the backup.

BACKUP DATE This field shows the date when the backup was performed.



AVTAR STATUS This field shows the status of the restore to disk. Valid states are the following:

When avtar fails, an error code and short summary are received. Error codes include the following:

• 151:EXIT_AUTH_FAIL• 152:EXIT_BAD_FLAGS• 153:EXIT_CACHE_OR_MUTEX_LOCKED• 154:EXIT_LOCAL_PERMISSIONS• 155:EXIT_EXCLUDELIST• 156:EXIT_MOUNTTABLE• 157:EXIT_MISC• 158:EXIT_CONNECT• 159:EXIT_INCLUDELIST• 160:EXIT_OK_WITH_OPENFILES• 161:EXIT_RESTORE_FAIL• 162:EXIT_TARGETLIST

STATE DESCRIPTION

avtar none The restore to disk has not run yet.

avtar in progress The restore to disk is in progress.

avtar complete The restore to disk is complete.

avtar failed continued The restore failed but tape out continued. Occurs only when--onnewavtarerror is set to continue.

avtar failed skipped The restore failed and tape out skipped.Occurs only when --onnewavtarerror or --onloggedavtarerror is set to skip.

avtar failed exit The restore failed and tape out exited. Occurs only when --onnewavtarerror is set to exit.

avtar failed skipped (too large for FS)

The restore failed and tape out skipped. Occurs only when --onnewavtarerror is set to skip.

avtar failed (too large for FS) The restore failed because the backup size was too large for the disk. Occurs only when --onnewavtarerror is set to continue.

avtar failed exit (too large for FS) The restore failed and exited because the backup size was too large for the disk. Occurs only when --onnewavtarerror is set to exit.



• 163:EXIT_CANCELED• 164:EXIT_CRASHED• 165:EXIT_INVALIDTICKET• 166:EXIT_KILLED• 167:EXIT_SSL_ERROR• 168:EXIT_AUTH_NO_MACHINE• 169:EXIT_MCSCANCEL• 170:EXIT_OK_WITH_ERRORS• 171:EXIT_ABORT• 172:EXIT_SIGINT• 173:EXIT_SIGTERM• 174:EXIT_DUMPSTREAM• 175:EXIT_UNHANDLEDEXCEPTION• 176:EXIT_FATALSIGNAL• 177:EXIT_NO_ARCHIVES

AVTAR DATE This field shows the time the restore to disk last reported status (format YYYY/MM/DD-HH:MM:SS).

TAPEOUT STATUS This field shows the status of the tape operations. Valid states are the following:

TAPEOUT DATE This field shows the time the tape backup last reported status (format YYYY/MM/DD-HH:MM:SS).

STATE DESCRIPTION

tapeout none No tape activity has occurred yet.

tapeout in progress Tape backup is in progress.

tapeout complete Tape backup has completed using the tape script.

tapeout complete (manual)

Tape backup has completed using the --manual method.

tapeout failed Tape backup script returned an exit code that was not 0.


Crontab Automation of selectfortape.pl Application

Example Staging File Contents/REPLICATE/server/clients/tester.example.com/0000032/ | tester.example.com | Linux | 32 | 55255040 | avtar complete | 2007/05/08-11:32:55 | tapeout complete | 2007/05/10-11:32:55/REPLICATE/server/clients/tester.example.com/0000033/ | tester.example.com | Linux | 33 | 55113728 | avtar failed continued | 2007/05/09-11:32:55 | tapeout complete | 2007/05/10-11:32:55/REPLICATE/server/clients/tester.example.com/0000034/ | tester.example.com | Linux | 34 | 40440832 | avtar complete | 2007/05/10-11:32:55 | tapeout complete | 2007/05/10-11:32:55/REPLICATE/server/clients/tester.example.com/0000035/ | tester.example.com | Linux | 35 | 40440956 | avtar in progress | 2007/05/12-11:32:55 | tapeout none | 2007/05/08-11:32:55/REPLICATE/trne12/clients/tester2.example.com/0000028/ | tester2.example.com | Linux | 28 | 40097792 | avtar none | 2007/05/08-11:32:56 | tapeout none | 2007/05/08-11:32:56/REPLICATE/trne12/clients/tester3.example.com/0000027/ | tester3.example.com | Linux | 27 | 39536640 | avtar none | 2007/05/08-11:32:56 | tapeout none | 2007/05/08-11:32:56

Crontab Automation of selectfortape.pl ApplicationUse the following information to automate running the selectfortape.pl application.

Display the current cron schedule for the utility node by entering the following (as user root):

crontab -u dpn -l

For a utility node, the default cron output looks similar to the following:# <<< BEGIN AXION ADMINISTRATOR MANAGED ENTRIES -- DO NOT MANUALLY MODIFY >>>0 6 * * * /usr/local/avamar/bin/cron_env_wrapper morning_cron_run0 18 * * * /usr/local/avamar/bin/cron_env_wrapper evening_cron_run# <<< END AXION ADMINISTRATOR MANAGED ENTRIES >>>

Modify the crontab directly with the automated selectfortape.pl line by entering the following (as user root):

crontab -u dpn -e

Example

To run the selectfortape.pl at 6 a.m. on the third day of every month, specifying two days prior as the start date and the current day as the end date, add the following to the crontab output:

0 6 3 * * /root/selectfortape.pl --start=`date -d '2 days ago' +\%Y-\%m-\%d` --end=`date -d 'today' +\%Y-\%m-\%d` --retention=61 --snapupoutput=/clients/snapupholder --utility=[UTILITY NODE] --exclude=/clients/clientx

NOTE: The start and end dates should be modified accord-ing to site-specific requirements.


Appendix A — selectfortape.pl Script

The resulting cron would look similar to the following:0 6 3 * * /root/selectfortape.pl --start=`date -d '2 days ago' +\%Y-\%m-\%d` --end=`date -d 'today' +\%Y-\%m-\%d` --retention=61 --snapupoutput=/clients/snapupholder --utility=[UTILITY NODE] --exclude=/clients/clientx# <<< BEGIN AXION ADMINISTRATOR MANAGED ENTRIES -- DO NOT MANUALLY MODIFY >>>0 6 * * * /usr/local/avamar/bin/cron_env_wrapper morning_cron_run0 18 * * * /usr/local/avamar/bin/cron_env_wrapper evening_cron_run# <<< END AXION ADMINISTRATOR MANAGED ENTRIES >>>

Appendix A — selectfortape.pl Scriptselectfortape.pl is a Perl script that queries the Avamar server, selects backups according to user-defined criteria and restores them to a specific location (usually on the staging server), where they can be processed by tapeout.pl (page 36).

Backups can be filtered according the the following criteria:

• Start date and end date

A range of dates for determining backups of interest.

• Minimum retention

A minimum retention setting in days for determining backups of interest; backups with a retention setting of this number of days or higher will be included in the archive to tape operation.

• Include

A list of explicitly included clients and domains; for example, /REPLICATE domain is included in tape archive operations by default.

• Exclude

A list of explicitly excluded clients and domains; for example /MC_BACKUPS is excluded in tape archive operations by default.

Synopsis

selectfortape.pl --start=STARTDATE --end=ENDDATE --retention=MINRETENTION --utility=UTILITYNODE

[ --avtar=AVTARPATH ] [--root=ROOTDIR] [--output=OUTFILE] [--append] [--exclude=EXCLUDECLIENTS] [--include=INCLUDECLIENTS] [--cod] [--mod] [--nah] [--quiet] [--debug=DEBUGFILE] [--snapupoutput=SNAPUPPATH] [--osinclude=OS,OS] [--osexclude=OS,OS] [--firstday] [--firstbackup] [--lastday] [--lastbackup] [--dayofweek=DAYOFWEEK(S)] [--interactive=PATH]



Required Arguments

All of the following arguments are required in order for selectfortape.pl to successfully complete.

Options

--start=DATE Used with --end=DATE to specify a range of dates for determining backups of interest. Backups starting on or after this date will be included in this restore operation. Valid format is YYYY-MM-DD, where YYYY is a four-digit year, MM is a two-digit month and DD is a two-digit day of the month.

--end=DATE Used with --start=DATE to specify a range of dates for determining backups of interest. Backups ending on or before this date will be included in this restore operation. Valid format is YYYY-MM-DD, where YYYY is a four-digit year, MM is a two-digit month and DD is a two-digit day of the month.

--retention=MINRETENTION Specifies a minimum retention setting in DAYS for determining backups of interest. Backups with a retention setting of this number of DAYS or higher will be included in this restore operation.

--utility=UTILITYNODE Host name or IP address of the Avamar server utility node that contains the backups to be exported to tape.

--append Appends backup list to --output=FILE. Default setting overwrites OUTFILE.

--avtar=AVTARPATH Specifies full PATH to the avtar command. Default setting is /usr/local/avamar/bin.

--cod Includes client on-demand backups found within the specified date range are included in this restore operation. If not supplied, default behavior is to exclude client on-demand backups from restore operations.

--dayofweek=DAYOFWEEK(S) Can specify three-letter day(s) of week that are to be returned within the given start/end dates.Valid days: sun, mon, tue, wed, thu, fri, sat.

--debug=FILE Enables writing of extended debugging messages to the speficified FILE. FILE is always overwritten.



--exclude=EXCLUDECLIENTS Excludes these domains and clients (EXCLUDECLIENT list) from this restore operation. This option accepts both valid client names or domain names.Use a new --exclude argument for each exclusion you want to add. Default exlusions, which need not be explicitly added, are: /MC_BACKUPS, /mc_backups, /MC_DELETED, /MC_RETIRED and /REPLICATE.

--firstbackup Returns the first (earliest) backup per client returned by the selection criteria.

--firstday Returns the first (earliest) day's worth of backups per client returned by the selection criteria.

--help | --usage Shows help, then exits.

--include=INCLUDECLIENTS Adds back specific clients and domains after --exclude=CLIENT-LIST is processed.Use a new --include argument for each exclusion you want to add.

--interactive=PATH Calls selectfortape.pl in interactive mode (GUI). Script automatically searches for selectfortape_interactive.pl in /usr/local/avamar/bin. This option must not be used with any other command line option.You can specify an alternate path to selectfortape_interactive.pl with the =PATH suffix.

--lastbackup Returns the last (most recent) backup per client returned by the selection criteria.

--lastday Returns the last (most recent) day's worth of backups per client returned by the selection criteria.

--local Immediately restores backups listed in --snapupoutput=LOCATION to local hard disk drive.

--mod Includes administrator-initiated on-demand backups in this restore operation.If not supplied, default behavior is to exclude administrator-initiated on-demand backups from restore operations.



NotesExit Codes:0 Success

1 Failure

--nah Includes "naked ah-hoc" backups in this restore operation.If not supplied, default behavior is to exclude naked ah-hoc backups from restore operations.

--output=OUTFILE Specifies full path of the an output file, which lists all backups that will be restored in this operation. Default name is snapups2tape.txt.

--quiet Runs quietly (suppresses screen messages).

--root=ROOTDIR Specifies top-level (root) directory (DIR) where backups will be staged for eventual archive to tape. Default is Dackupdir.

--snapupoutput=SNAPUPPATH Upon completion, backs up the --output=OUTFILE to this domain /client SNAPUPPATH on the Avamar server.


Appendix B — tapeout.pl Script

Appendix B — tapeout.pl Scripttapeout.pl is a Perl script that receives output generated by selectfortape.pl (page 32). The method of tape out used determines required arguments for tapeout.pl.

Synopsis

tapeout.pl {--manual | --tapescript=SCRIPT} --stage --stagingdir=DIR

[ --avtar=AVTARPATH ] [--root=ROOTDIR] [--quiet] [--log=LOGFILE] [--snapupoutput=SNAPUPPATH] [--output=FILENAME] [--id=ID] [--password=PASSWORD] [--clientdump] [--stage] [--stagingdir=DIR] [--tapescript=PATHTOSCRIPT] [--stagingfile=PATHTOFILE] [--logdir=DIR] [--osinclude=OS,OS] [--osexclude=OS,OS] [--manual] [--incremental] [--onloggedavtarerror=retry|skip] [--onnewavtarerror=exit|skip|continue] [--avtarflags="--flag1,--flag2"] [--help | --usage]

Use --usage or --help options for extended usage information.

Mode Control Arguments

tapeout.pl can operate in two modes and the arguments controlling the operational mode are mutually exclusive:

Other Required Arguments

All of the following arguments are required in order for tapeout.pl to successfully complete.

--manual If supplied, tapeout.pl operates in manual mode. Following succesful staging of the backups, user is prompted to manually archive backups to tape using the tape backup software installed on the staging server.

--tapescript=SCRIPT If supplied, tapeout.pl automatically invokes this tape dump script, which exports the tape backups that have been restored to the staging server. SCRIPT value is path and script file name. Refer to 1 Failure (page 38) for additional information.

--stage This argument must be supplied. Otherwise, tapeout.pl will assume that an obsolete method for exporting backups has been used.

--stagingdir=DIR Specifies target directory (DIR) that is the target for the redirected restores during the staging process



Options

--avtar=AVTARPATH Specifies full PATH to the avtar command. Default setting is the default OS-specific location of the avtar binary on the staging server.

--avtarflags="--FLAG1,--FLAG2"

When performing the avtar restores, if additional flags are to be passed in through the commandline and an avtar.cmd is not enough, then the user can specify additional avtar flags in this flag. Be sure to encapsulate all of the additional avtar commands in quotes.

--clientdump Lists out the clients only from the snapupoutput to STDOUT. Can be used in conjunction with the --output or --snapupoutput option depending on the file to be used.

--help | --usage Shows help, then exits.

--id=USER Authenticate as this Avamar user ID (account name). Default user ID is root.

--incremental [default: disabled]

Incremental mode is a restore in which the destination is incrementally restored, thus reducing restoration times and allowing for more frequent staging. When restoring, an "INCREMENTAL" directory is created in the stagingdir.

--log=LOGFILE Enables writing of logging messages to the speficified FILE. Default FILE name is tapeout.log.

--logdir=DIR Specifies directory (DIR) containing all of the avtar restore logs as well as tape logs. The default directory is tapeout_logs.

--onloggedavtarerror={retry | skip}

When encountering a previously failed backup from the stagingfile, the behaviour can be controlled. retry means to retry restoring it, while skip means do not do anything with it and leave it alone. skip is the default

--onnewavtarerror={exit | skip | continue}

When encoutering an error on an avtar, the behaviour can be controlled. exit means to exit on a failed restore. skip means to delete all data that was restored and mark as failed and continue on. continue means to mark as failed but do not delete the data and continue on. exit is the default.



NotesExit Codes:0 Success

1 Failure

--osexclude=OS,OS When specifying an exclude(s), any OS that was not excluded is automatically be included. The valid OS types are: unknownfilesystem, linux, solaris, windows, hp-ux, aix, mac, unix, ndmp.

--osinclude=OS,OS When specifying an include(s), any OS that was not included will automatically be excluded. The valid OS types are: unknownfilesystem, linux, solaris, windows, hp-ux, aix, mac, unix, ndmp.

--output=FILE This must be the same output file path that was specified by selectfortape.pl (page 32).

--password=PASSWORD Specifies PASSWORD for the --id=USER account.

--policy=POLICY Specifies the POLICY base name to be used.

--quiet Runs quietly (suppresses screen messages).

--root=ROOTDIR TBD - Eugene, Gary ???

--snapupoutput=LOCATION This must be the same domain/client LOCATION on the Avamar server that was specified by selectfortape.pl (page 32).


Copyright © 2002 - 2011 EMC Corporation. All rights reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THISPUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date regulatory document for your product line, go to the Technical Documentation and Advisories section on EMC Powerlink.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.
