Netbackup Important Terminologies -- Part 2

Major Daemons Running in Master/Media/Client

Master - 4 Daemons

           i) bprd       --  request daemon started during the system.

          ii) bpsched  -- schedule daemon

         iii) bpdbm    -- database manager started by bpsched

         iv) bpjobd    -- Job monitor daemon started by bpdbm

                   

               bprd --> bpsched --> bpdbm --> bpjobd

 Media - 9 Daemons 

          i) bpcd   -- Backup Client (Since media itself is a client)

         ii) bpbrm -- Backup and restore daemon

        iii) bpdm  -- Backup disk manager

        iv) bptm   -- Backup tape manager

         v) ltid     -- Media manager daemon started during the system boot

        vi) avrd    -- Automatic Volume Recognition daemon

       vii) vmd    -- Volume Manager daemon 

      viii) tldd     -- robotic tape library daemon

        ix) tldcd   -- robotic tape library controller daemon

Client - 2 daemons  

        i) bpcd -- NB client daemon

       ii) bpbkar -- Daemon used to take backup images.

How Netbackup process works?

        There are 7 steps involve in NB

    

     In Master Server 

             bprd --> bpsched --> bpdbm --> bpjobd

 

        Step 1:

             *  When the daemon starts "bprd" on the master server start "bpsched" 

       Step 2:

             *  "bpsched"  calls bpdbm to check the policy for auto backup schedules 

                  "bpsched"  calls the child process to handle the backup

               "bpdbm"  calls the bpjobd to communicate with job catalog. 

         Step 3:

           *  "bpsched" child contacts the media server to start the bpcd

     

     In Media Server

       

             bpcd --> bpbrm --> bptm 

      

      Step 4: 

           * “bpcd” invokes  bpbrm

           * “bpbrm”  starts “bptm”  which requests a tape mount and spans a child process to communicate  with client. 

           * Vmd (Volume Manager Daemon) - manages the Volume catalog and handles media requests through out the course of the backup job.

           * “bpbrm”  contacts the client to start the “bpcd” in client. 

            

    In Client 

        Step 5:  "bpcd"  in the client starts the "bpbkar"

        Step 6:  "bpbkar" reads data from client

             * Communicates directly with "bpbrm" to send catalog meta data which is ultimately written to images catalog. 

             * Send the backup data stream to the "child bptm" process

       Step 7: 

             * The child bptm processes passes the data to the "parent bptm"

process which writes the data directly to the destination data.

                

What is Volume Group and Volume Pool?

Volume Pool

• Is nothing but the “grouping of tape”. 
• Meant  Volume/media/catridge/tape.
• It is a Logical Set of Tape media/Volume.
• If we want to use the newly introduced tape and to write to the data, it should be under one of the “volume pool” 

    We can we the Volume pools by the following ways. 
         In GUI,  Media and Device Management --> Media --> Volume Pools

         In CLI, #vmpool -list_all -bx

 

  root@MasterServer# vmpool -list_all -bx
  pool                    index   max partially full   description
  ----------------------------------------------------------------------------------
  None                     0         0                the None pool
  NetBackup              1         0                the NetBackup pool
  DataStore               2         0                the DataStore pool
  CatalogBackup        3         0                NetBackup Catalog Backup pool
  SCRATCH_pool        4         0                for scratch media
  APPSERVER_LOGS    5         1                APP server logs
  DATABASE_LOGS     6         3                Database full
  WEBSERVER_FULL    7         1                Web server full

Volume Group

• Is a Physical location  for a media. 
• Manages a group of tape for purpose of admin actions that allows movement of tapes from slots to drive and vice versa.
• All volumes in the VG should be the same media type. (HCART,DLT)
• All volumes in a robotic library must belong to a volume group. You cannot add volumes to a robotic library without specifying a group.
• We can view the Volume groups in 2 ways.

      i) In GUI, Media and Device management --> Media --> Volume Groups
      ii)In CLI, #vmquery -a |grep "media ID:"
   root@SS73VPBK01 # vmquery -a |grep "media ID:"
   media ID:              CLNU01
   media ID:              PL5001
   media ID:              PL5002
   media ID:              PL5003
   media ID:              PL5004
  

What are the Default Volume Pools?

     There are 4 volume pools created by default

       i) Netbackup --> new tapes are go to this pool after inventory

      ii) None (for cleaning tapes)

      iii) Datastore

      iv) Catalog backup

What is Scratch Pool ?

     Scratch pool is the manually created volume pool and following are the procedures of scratch pool 

     i) Expired Tapes (volumes) are automatically moved to scratch pool by netbackup.

    ii) If some other pools used for taking backup doesn't have any tapes available, it will take empty tapes from here.

What is Multistreaming and Multiplexing?

Multistreaming 

        i) Sends data from single client to multiple data drives. Normally used in high bandwidth environment.

      ii) Backups can be divided into streams. Different jobs in different tape.

     iii) Taken place in software level. 

Multiplexing

      i) Send different  jobs/data into a single tape. Schedule level backup

     ii) It is very fast especially for small environment. 

    iii) Taken place in hardware level.

Netbackup Important Terminologies -- Part 1

Master Server

• Acts Enterprise Media Manager server (contains info about robots, drives , Storage Units etc.,).
• Control unit or Centralized management server of Netbackup architecture.
• Controls all the backup scheduling for each media server.
• Each of the media server performs the actual backup operations under direction from the master and backup data stays local to the media severs or to their corresponding storage units (like Tape) .
• Also tracks the media being used.
• Stores the catalog information (what are the backups taken) information.
• Master server should be the dedicated server, in some cases both master and media server resides in same server.

Media Server or why media server is required ?

• Assistance for master server. 
• Data mover in the netbackup architecture.
• To increase performance by providing additional processing ability and better distribution of workload (load balancing).
• Master server communicates client through Media, all the clients are connected to the media server only.
• Media is divided into 2 , disk storage and tape. Backup has been written to tape through media server instead of writing directly to the Tape. It is due to whenever we need to restore recent backup it take much time to restore it from tape instead directly from the server.
• All the tape libraries should be connected to Media server only.               

Backup Client

• During the backup or archive the client system generates the backup as a data stream and sends it across the network to the media server (tape).
• No license required for client servers. 

Archive bit

• The archive bit is a binary bit. As such, it is either 1 or 0.
• Set 1 and clear or reset 0. It represents whether the file has (1) or has (0), since the last full backup.

Difference between backup and archive

     When a backup is made, a copy of the file is written to media. When an archive is made, a copy of the file is written to the media and then original file is deleted.

Difference between backup and “True image”
    A regular backup can backup and restore individual files. 
    A TI backup is a snapshot of files done at the directory level at a certain point in time. Additionally, when a “True Image” backup is restored, the directory restored will be brought to the same state as when it was backed up.
   Any files or sub-directories that did not exist at the time of backup will be deleted when the restore occurs if it is restored to the same location. 

Types of Backup in Netbackup
a)Full Backup
     Is the starting point of all the backups, contains all the data in the folders and files that are selected to be backed up.
Advantages: Restore is fast
Disadvantages: Backup is slow and Consume more space.
b)Differential Incremental (VPS)
     Takes the backup of the changes since the latest backup.  Archive bit is reset. Backup had taken based on the timestamp.
Advantages:
      Less space occupancy in the tape, since only the changes had been taken as backup.
      Backup window is shorter
Disadvantages:  
      Slow restore, (Backup window is longer) since the images are stored in scattered manner, while restoring, organizing the data will take time and restore.
      In case of retention, to restore all files requires last full backup and all the differential backups since last full backup, so all the differentials must be kept until the next full backup occurs.
c) Cumulative Incremental
       A cumulative incremental backup is the backup of all files that have changed since the last full backup
Advantages:
     Fast restore, since the last cumulative incremental is required for a complete restore ( in addition to full)
Disadvantages:
     More space occupancy in Tape/media
     Backup window is longer
d) Synthetic Backup
    A Synthetic full or cumulative incremental backup is a backup assembled from previous backups.
    Can be written to tape or disk storage units, or a combination of both. 
e) User initiated Backup
     Its also called manual backup, initiated by the user.
f) Archive Backup
     After taking the backup the original file is deleted.

Difference between Differential and Cumulative incremental

Differential Incremental Backup

    
If the disk fails after sometimes on Jan 4 (after backup) the full backup and 3 diff incremental backups requires to restore data 

Cumulative Incremental  Backup

 

   If the disk fails on Jan4, only Jan4th cumulative incremental backup is required to restore the data. 

Define Catalog Backup   

• Netbackup have special kind  of backup called “Catalog Backup”. 
• It is an internal database that contains backup information of the files that have been backed up and media where it stored and NB configuration.
• The catalogs also contain the information about the media and storage devices.
• Netbackup requires the catalog information in order to “restore backups” , so it is extremely important to configure a  catalog backup before using  netbackup for regular client backups.
• Also used to search for a backup image in order to verify the contents of media with what is recorded in the NB catalog. To duplicate a backup image, to promote a backup image from a copy to the primary copy. To import expired backup images or images from another NB server.
• Hence loosing the catalog means loosing the backups. 
• In  General Catalog Backup is configured as “FULL backup”.

Parts of Catalog 
      There are 3 parts of catalog 
         image database
         NB relational database
         NB configuration files
i)Image database 
      Biggest part of the catalog image database.
   root@MasterServer # cd /usr/openv/netbackup/db/images/
   root@MasterServer # ls      
   Appserver Webserver dbserver 
      This images folder contains subfolders for each server and its corresponding image.
   root@MasterServer #cd Appserver/
   root@MasterServer #ls -lrth
  drwxr-xr-x   4 root     other          7 Apr 29 06:34 1332000000
  drwxr-xr-x   4 root     other          7 Jun  2 06:36 1335000000
  drwxr-xr-x   4 root     root          49 Jul  1 18:41 1340000000
  root@MasterServer # cd 1332000000
  root@MasterServer # ls -lrth
  total 30
  -rw-------   1 root     other         35 Mar 27 06:32     Appserver_Policy_1332665618_FULL.f.Z
-rw-r--r--   1 root     other       1.3K Mar 27 06:32   Appserver_Policy_1332665618_FULL
  drwxr-xr-x   3 root     other          4 Mar 29 01:04 tmp
  drwx------   2 root     other         11 Apr 18 06:33 catstore
ii) Relational DB files
root@MasterServer # cd /usr/openv/db/data/
root@MasterServer # ls
DARS_DATA.db        DBM_DATA.db         EMM_DATA.db         NBAZDB.db           NBAZDB.log          NBDB.log  DARS_INDEX.db       

iii) NB Configuration Files
   The configuration files include flat files concerning policy, schedule information, licensing information etc ., in the folder /usr/openv/var

How to recover the catalog in case of accidental removal or Master server got crashed?
      There are 2 way of procedures for catalog recovery 
          i) Actually, the catalogs are stored in "Stroage LUN" hence if the master server got crashed, then we need the NB master server in some other server and map the LUN to the server  and process catalog restore activity.
        ii) If primary (tape) catalog backup files the secondary backup is converted into Primary if the secondary is in expired state, first we want to unexpire and update.