Video Screencast Help
Search Video Help Close Back
to help

Getting the error bpstart_notify failed (73) | Please I need a Solution

Created: 13 Jun 2012 | 10 comments
Wazid Khan's picture
Wazid Khan
0 0 Votes
Login to vote

Hi Guys,

I have netbackup 7.0.1 in my environment.Backup for one of the client is failing with Status code 73.The client is a Solaris 10 server.

I have tried the below

(i) Run bpstart_notify script on the client.I shows no output.Just goes to the next command prompt

(ii) Restarted netbackup services on the client

(ii) I have observed one thing when I have done the following few of its child streams have run successfully.

first of all checked and foundin bpstart_notify.logs  the following in /usr/openv/netbackup/logs

============================================================
PREPARATION FOR BACKUP CANCELLED AT Wed Jun 13 12:56:51 IST 2012.
FILE /tmp/.bp_sched_dir/client/aborted DETECTED
============================================================
============================================================
Stream [32]: Found [/tmp/.bp_sched_dir/client/aborted]

- The last backup did not cleanup OR a stream aborted backup (check bpend logfile). [rc=127]

I am stream [32], First stream is [32] ...
Stream: [32]:
Directory [/tmp/.bp_sched_dir/client] not found.

clientroot) #: cd [/tmp/.bp_sched_dir/client]

client(root) #: cd client

client(root) #: ls -ltr

total 0

-rw-r--r-- 1 root root 0 Jun 13 12:56 aborted

-rw-r--r-- 1 root root 0 Jun 13 12:57 bpend_proc_list.25935

client(root) #: rm aborted

After doing this i have restarted one of its child stream failed job it completed successfully with status code 0.

But when restarted parent stream many jobs failed though few of them were successfull.Later on I have also realized that that "aborted" file got recreated.

few other useful logs

Jun 13, 2012 4:38:08 PM - requesting resource vombsdel1-hcart-robot-tld-0
Jun 13, 2012 4:38:08 PM - requesting resource vombsdel1.NBU_CLIENT.MAXJOBS.client
Jun 13, 2012 4:38:08 PM - requesting resource vombsdel1.NBU_POLICY.MAXJOBS.ENIQ_STATS_DATA_client
Jun 13, 2012 4:38:08 PM - Info nbrb (pid=1154) Limit has been reached for the logical resource vombsdel1.NBU_CLIENT.MAXJOBS.client
Jun 13, 2012 4:47:36 PM - awaiting resource vombsdel1-hcart-robot-tld-0. No drives are available.
Jun 13, 2012 4:48:28 PM - Info nbrb (pid=1154) Limit has been reached for the logical resource vombsdel1.NBU_CLIENT.MAXJOBS.client
Jun 13, 2012 5:18:56 PM - awaiting resource vombsdel1-hcart-robot-tld-0. No drives are available.
Jun 13, 2012 5:20:07 PM - granted resource  vombsdel1.NBU_CLIENT.MAXJOBS.client
Jun 13, 2012 5:20:07 PM - granted resource  vombsdel1.NBU_POLICY.MAXJOBS.ENIQ_STATS_DATA_veniqdel1-bkp
Jun 13, 2012 5:20:07 PM - granted resource  C125L4
Jun 13, 2012 5:20:07 PM - granted resource  HP.ULTRIUM4-SCSI.001
Jun 13, 2012 5:20:07 PM - granted resource  vombsdel1-hcart-robot-tld-0
Jun 13, 2012 5:20:07 PM - estimated 650035240 kbytes needed
Jun 13, 2012 5:20:08 PM - started process bpbrm (pid=18684)
Jun 13, 2012 5:20:08 PM - connecting
Jun 13, 2012 5:20:08 PM - connected; connect time: 0:00:00
Jun 13, 2012 5:20:18 PM - Error bpbrm (pid=18684) from client client: ERR - bpstart_notify returned 80
Jun 13, 2012 5:20:18 PM - end writing
bpstart_notify failed  (73)
 

I am a beginner to netbackup and facing difficulty in resolving this issue.

So somebody please help me to resolve the issue.Please let me know if you need some other logs

Regards

Wazid

Discussion Filed Under:
, , ,

Comments 10 Comments • Jump to latest comment

AlanTLR's picture
AlanTLR
Getting the error bpstart_notify failed (73) | Please I need a Solution - Comment:13 Jun 2012 : Link

Wazid,

  Your bpbrm log should have more detail.  If you don't have a bpbrm directory, create it and run your job again  (/usr/openv/netbackup/logs/bpbrm).  Also, can you please post your bpstart_notify?  Thanks

Also, check that permissions are set properly for bpstart_notify.

And review this technote, as well: http://www.symantec.com/business/support/index?page=content&id=TECH182631

--Alan

0
Login to vote
  • Actions
Marianne van den Berg's picture
Marianne van den Berg Trusted Advisor Partner Accredited Certified
Getting the error bpstart_notify failed (73) | Please I need a Solution - Comment:13 Jun 2012 : Link

I agree with request for bpstart_notify script.

Please help us to understand what bpstart_notify is supposed to do? If multiple data streams is enabled in policy, the same bpstart_notify will run for each stream. Is this what you want or what is required?

Please also post policy details. On master, run following and post output:

bppllist <policy-name> -U

 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

0
Login to vote
  • Actions
Wazid Khan's picture
Wazid Khan
Getting the error bpstart_notify failed (73) | Please I need a Solution - Comment:13 Jun 2012 : Link

Thanks Alan and Marianne for responding

The following is the bppllist output for the policy

bash-3.00# /usr/openv/netbackup/bin/admincmd/bppllist Policyname -U

------------------------------------------------------------

 

Policy Name: POLICYNAME

 

Policy Type: Standard

Active: yes

Effective date: 01/01/1970 05:30:00

Client Compress: no

Follow NFS Mounts: no

Cross Mount Points: no

Collect TIR info: yes, with move detection

Block Incremental: no

Mult. Data Streams: yes

Client Encrypt: no

Checkpoint: no

Policy Priority: 0

Max Jobs/Policy: Unlimited

Disaster Recovery: 0

Collect BMR info: yes

Residence: vombsdel1-hcart-robot-tld-0

Volume Pool: ENIQ_STATS_Pool

Server Group: *ANY*

Keyword: (none specified)

Data Classification: -

Residence is Storage Lifecycle Policy: no

 

Granular Restore Info: no

Ignore Client Direct: no

HW/OS/Client: Solaris Solaris_x86_1 ClientName

 

Include: NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_10

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_temp_dbspace/dbspace_dir_5

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_temp_dbspace/dbspace_dir_4

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_temp_dbspace/dbspace_dir_3

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_temp_dbspace/dbspace_dir_2

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_temp_dbspace/dbspace_dir_1

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_9

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_8

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_7

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_6

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_5

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_4

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_3

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_2

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main_dbspace/dbspace_dir_1

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_temp_dbspace/tmp_000001

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_temp_dbspace/__SQLAny__

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_temp_dbspace/dwhdb

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/rep_temp

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/rep_main

NEW_STREAM

/.ENIQ_BACKUP/eniq/database/dwh_main

NEW_STREAM

/.ENIQ_BACKUP/eniq/sql_anywhere

NEW_STREAM

/.ENIQ_BACKUP/eniq/installation

NEW_STREAM

/.ENIQ_BACKUP/eniq/sybase_iq

NEW_STREAM

/.ENIQ_BACKUP/eniq/sentinel

NEW_STREAM

/.ENIQ_BACKUP/eniq/upgrade

NEW_STREAM

/.ENIQ_BACKUP/eniq/archive

NEW_STREAM

/.ENIQ_BACKUP/eniq/fmdata

NEW_STREAM

/.ENIQ_BACKUP/eniq/admin

NEW_STREAM

/.ENIQ_BACKUP/eniq/home

NEW_STREAM

/.ENIQ_BACKUP/eniq/data

NEW_STREAM

/.ENIQ_BACKUP/eniq/log

NEW_STREAM

/.ENIQ_BACKUP/eniq/sw

 

Schedule: Full_Backup

Type: Full Backup

Maximum MPX: 1

Synthetic: 0

PFI Recovery: 0

Retention Level: 3 (1 month)

Number Copies: 1

Fail on Error: 0

Residence: (specific storage unit not required)

Volume Pool: (same as policy volume pool)

Server Group: (same as specified for policy)

Calendar sched: Enabled

Allowed to retry after run day

Sunday, Week 1

Wednesday, Week 1

Friday, Week 1

Sunday, Week 2

Wednesday, Week 2

Friday, Week 2

Sunday, Week 3

Wednesday, Week 3

Friday, Week 3

Sunday, Week 4

Wednesday, Week 4

Friday, Week 4

Sunday, Week 5

Wednesday, Week 5

Friday, Week 5

Residence is Storage Lifecycle Policy: 0

Daily Windows:

Sunday 10:00:00 --> Monday 10:00:00

Monday 10:00:00 --> Tuesday 10:00:00

Tuesday 10:00:00 --> Wednesday 10:00:00

Wednesday 10:00:00 --> Thursday 10:00:00

Thursday 10:00:00 --> Friday 10:00:00

Friday 10:00:00 --> Saturday 10:00:00

Saturday 10:00:00 --> Sunday 10:00:0

 

bpstart_notify script not giving any output.I am running this as root

Client #: cd /usr/openv/netbackup/bin
Client #: ./bpstart_notify
Client #:

Here are the file permissions.

-rwxr-xr-x   1 root     root       59935 Jun 11 11:38 ./bpstart_notify

I will attach the bpstart_notify script output in a moment ...

Basically I would like to know what else can be tried to run the backup successfull with regards

to this status code.

 

 

0
Login to vote
  • Actions
revaroo's picture
revaroo Symantec Employee
Getting the error bpstart_notify failed (73) | Please I need a Solution - Comment:13 Jun 2012 : Link

It's impossible to tell until we see the contents of the bpstart_notify script.

That is the bit that is failing. The actual backup part doesn't even start because bpstart_notify fails.

 

0
Login to vote
  • Actions
Marianne van den Berg's picture
Marianne van den Berg Trusted Advisor Partner Accredited Certified
Getting the error bpstart_notify failed (73) | Please I need a Solution - Comment:14 Jun 2012 : Link

In addition to bpstart_notify script, we also need clarification on the following:

Please help us to understand what bpstart_notify is supposed to do?
If multiple data streams is enabled in policy, the same bpstart_notify will run for each stream.
Is this what you want or what is required?

We can see that you are indeed doing multi-streaming, so, the script will run at the beginning of each stream.
If this is not want you want, you need to add some test criteria into the script.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

0
Login to vote
  • Actions
Michael Simcorp's picture
Michael Simcorp Certified
Getting the error bpstart_notify failed (73) | Please I need a Solution - Comment:14 Jun 2012 : Link

Guessing the problem is you are trying to close down/set the datbase in backup mode multiple times

What happens if you run the bpstart_notify multiple times on the command line ?

 

 

 

0
Login to vote
  • Actions
Michael Simcorp's picture
Michael Simcorp Certified
Getting the error bpstart_notify failed (73) | Please I need a Solution - Comment:14 Jun 2012 : Link

Below is an example of a bpstart_notify.bat we use for a multi-stream backup.

It is for windows, but think you can get the idea

--------------------------------------------------------------------

timeout /t %STREAM_NUMBER%

if exist c:\temp\wget_script_running.lock (goto waiting)
echo "" > c:\temp\wget_script_running.lock

<<wget command stuff >>

set RETURN=%errorlevel%

del /f c:\temp\wget_script_running.lock
if %RETURN% NEQ 0 (
echo 1 > %6
)

:waiting
if EXIST c:\temp\wget_script_running.lock (
timeout /t 1
goto waiting
)

0
Login to vote
  • Actions
Mark_Solutions's picture
Mark_Solutions Trusted Advisor Partner Accredited Certified
Getting the error bpstart_notify failed (73) | Please I need a Solution - Comment:14 Jun 2012 : Link

Perhaps as important is that the bpend_notify script will also run everytime a stream finishes

So if the bpstart shuts something down the first stream that ends will start it up again

For this situation you either need to make the bpstart clever enough to know this and have it create lock file for the bpend to check or you would need to use parent start notify scripts on the Master, specifically for the policy in question

So we need to see both the bpstart and bpend scripts

Hope this helps point you in the right direction

Authorised Symantec Consultant

Don't forget to give a "Thumbs Up" or mark as "Solution" if someones advice has helped you.

+1
Login to vote
  • Actions
rookie11's picture
rookie11
Getting the error bpstart_notify failed (73) | Please I need a Solution - Comment:18 Jun 2012 : Link

Jun 13, 2012 5:20:18 PM - Error bpbrm (pid=18684) from client client: ERR - bpstart_notify returned 80  --- it should return '0' 

go to /usr/openv/netbackup/bin  then more bpstart_notify and share the output

0
Login to vote
  • Actions
E-Lara's picture

Hi to All.

 

I have the same problem. Do you know the solution??? 

I share he bpstart_notify:

 

 

# cat bpstart_notify
#!/bin/sh
# ####################################################################
# # COPYRIGHT Ericsson AB 2005 - 2010
# #
# # The copyright to the computer program(s) herein is the property
# # of ERICSSON AB, Sweden. The programs may be used
# # and/or copied only with the written permission from ERICSSON
# # AB or in accordance with the terms and conditions
# # stipulated in the agreement/contract under which the program(s)
# # have been supplied.
# #
# ####################################################################
#
#------ History --------------------------------------------------------------------------
# Rev   Date     Prepared       Description
#
# R1A   20040602                First version
# R1B   20050816 lmckhte        Turn on fast resynch when license is valid.
# R1C   20050908 enoemcl        Use VXLICENSE variable instead of vxlicense -p.  i.e. support R3 vxlicrep
#                               look for FASTRESYNC in Solaris 8 and FlashSnap in Solaris 9
# R1D   20060201 lmcphta        Enabled SNMS always and updatred fastresync code
# R1E   20080314 enoemcl        Merged niq bpstart notify. Added SNMS 5.0 support.
# R1F   20080414 eeitfy         NIQ Statistics x86 support
# R1F   20080513 efrador        Solaris x86 support
# R1G   20080523 enoemcl        Improve hacs/framework stream handling
# R1H   20080606 enoemcl        Handle OSS_HARS_<hostname> Policies.
# R1J   20081006 enoemcl        Incorporate NIQ_Backup changes; Convert to bourne shell
# R1K   20081104 enoemcl        First stream might be stream X not always stream 1.
# R1L   20081111 enoemcl        Define SAVE_CONFIG for NIQ Sparc ROOT-Policy.
# R1M   20080204 enoemcl        Update check on fastresync license.
# R1N   20080210 enoemcl        If any abort sitution occurs, ensure cleanup backup runs.
# R1P   20080215 enoemcl        If fastsync not enabled(licensed), ensure feature=off on vols.
# R1Q   20080302 enoemcl        Fix for fastsync endable/disable code (process all vols).
# R1R   20090508 enoemcl        nbuClient now in sparc/ or x86/ subdirectory. OMBSC_PLATFORM var.
# R1S   20090529 enoemcl        Set NET_BUFFER_SZ if not already done.
# R1T   20090702 efrador        Introduction of BMRclient for OSS Suites and NIQ.
# R1U   20091215 enoemcl        Support new policy names (DATA/ROOT)
# R1V   20100527 enoemcl        Update save_vxvm_info; specify path /ericsson/ombsc/backup
# R1W   20100807 enoemcl        Attempt to recover from OMBS on Solaris X86 HACS issue.
# R1X   20101002 enoemcl        Add Eniq Events and OSS on X86 support to this script.
# R1y   28102010 ebahram        change the ALT_SAVE_CONFIG for Enig Events.
# R1Z   15112020 ebahram        All events scripts need /usr/bin/bash prefix!?
#
#------ Purpose --------------------------------------------------------------------------
#
# bpstart_notify.sh
#
# This script is called by NetBackup when bpbkar is started up on the client to do a
# or a archive.  The code here is ERICSSON OMBS Policy code.
#
#-----------------------------------------------------------------------------------------
#env
OMBSC_HOME=/ericsson/ombsc
#cmds
AWK=/usr/bin/awk
BASENAME=/usr/bin/basename
BC=/usr/bin/bc
CAT=/usr/bin/cat
CUT=/usr/bin/cut
CP=/usr/bin/cp
DATE=/usr/bin/date
DIRNAME=/usr/bin/dirname
DF=/usr/sbin/df
ECHO=/usr/bin/echo
EGREP=/usr/bin/egrep
ENV=/usr/bin/env
EXPR=/usr/bin/expr
GETENT=/usr/bin/getent
GREP=/usr/bin/grep
HEAD=/usr/bin/head
HOSTNAME=/usr/bin/hostname
ID=/usr/bin/id
KSH=/usr/bin/ksh
LS=/usr/bin/ls
METASTAT=/usr/sbin/metastat
MKDIR=/usr/bin/mkdir
MORE=/usr/bin/more
MV=/usr/bin/mv
NAWK=/usr/bin/nawk
PKGINFO=/usr/bin/pkginfo
PWD=/usr/bin/pwd
PS=/usr/bin/ps
RM=/usr/bin/rm
SED=/usr/bin/sed
SLEEP=/usr/bin/sleep
SORT=/usr/bin/sort
SU=/usr/bin/su
SYNC=/bin/sync
TAIL=/usr/bin/tail
TEE=/usr/bin/tee
TOUCH=/usr/bin/touch
TPUT=/usr/bin/tput
UNAME=/usr/bin/uname
USERADD=/usr/sbin/useradd
UNIQ=/usr/bin/uniq
VOLD=/usr/sbin/vold
VXPRINT=/usr/sbin/vxprint
VXVOL=/usr/sbin/vxvol
VXDISK=/usr/sbin/vxdisk
VXLICENSE=/opt/VRTSvxvm/bin/vxlicense
VXLICREP=/sbin/vxlicrep
VXLICTEST=/sbin/vxlictest
VXTASK=/usr/sbin/vxtask
WC=/usr/bin/wc
HAGRP=/opt/VRTS/bin/hagrp
 
#********************************************************************
# Environment
#********************************************************************
OMBS_COMMON=/ericsson/ombs_common
TPP_HOME=/usr/openv/netbackup
OUTF1=${TPP_HOME}/bin/BPSTART_CALLED
OUTF2=${TPP_HOME}/bin/BPEND_CALLED
 
# Save the parameters
BKUP_CLIENT=${1}
POLICY_ID=${2}
SCHED_NAME=${3}
SCHED_TYPE=${4}
 
[ -z "${STREAM_COUNT}"  ] && STREAM_COUNT=0
[ -z "${STREAM_NUMBER}" ] && STREAM_NUMBER=0
 
###Quit if hot catalog###
echo "${POLICY_ID}" | egrep -i "catalog|hot_catalog" >/dev/null 2>&1
if [ ${?} -eq 0 ]
then
        exit 0
fi
 
# Save the time of running.
RUN_TIME=` $DATE '+%Y-%b-%d_%H.%M.%S' `
[ -z "${BPSTART_TIMEOUT}" ] && BPSTART_TIMEOUT=1440
 
#********************************************************************
# Configuration Section
#********************************************************************
OS_VERSION=` $UNAME -r | $CUT -c3-4 `
BPCONF=/usr/openv/netbackup/bp.conf
BP_CLIENT_NAME=` $CAT ${BPCONF} | $EGREP '^[    ]*CLIENT_NAME[  ]*=' | $NAWK -F\= '{print $2}' | $SED -e 's| ||g' `
THIS_HOST=`hostname`
 
#X86?
I386=`$UNAME -p|grep -w i386`
if [ "${I386}" ]
then
        OMBSC_PLAT=i386
else
        OMBSC_PLAT=sparc
fi
 
###Just in case ERICombsc was not upgraded yet.###
if [ ! -d "${OMBSC_HOME}/bin/${OMBSC_PLAT}" ]
then
        OMBSC_PLAT=x86
        if [ ! -d "${OMBSC_HOME}/bin/${OMBSC_PLAT}" ]
        then
                OMBSC_PLAT=""
        fi
fi
 
#*********************************************
# Log handling and Stream handling
#*********************************************
##Flag to determine whether to compress log or not (y or n)
COMPRESS_LOG=n
 
#Number of log files to keep
NUM_LOGS=10
 
#Size in kbytes that the log file is allowed to grow to
SIZE_LOGS=2048000
 
#**************************
# Pre execution operations
#**************************
ABORTING_BACKUP=false
DEBUG_MODE=0
DEBUG_FLAG_FILE=/tmp/bpstart_debug_mode
if [ -f "${DEBUG_FLAG_FILE}" ]
then
        DEBUG_MODE=1
fi
 
#-------------------------------#
#-------------------------------#
# First, the Common Functions   #
#-------------------------------#
#-------------------------------#
 
### Peek at temp_sched_dir contents
###
peek_at_temp_sched_dir()
{
  L_PEEK_DEBUGMODE=$1
 
        if [ "${DEBUG_MODE}" -eq 0 ]
        then
                return 0
        fi
 
        if [ "${L_PEEK_DEBUGMODE}" -eq 1 ]
        then
                if [ -d "${TEMP_SCHED_DIR}" -a -f "${LOGFILE}" ]
                then
                        echo >> ${LOGFILE}
                        echo "-> Contents of [${TEMP_SCHED_DIR}/]" >> ${LOGFILE}
                        $LS ${TEMP_SCHED_DIR} >> ${LOGFILE}
                        if [ -d "${TEMP_SCHED_DIR}/run" ]
                        then
                                echo "-> Contents of [${TEMP_SCHED_DIR}/run]" >> ${LOGFILE}
                                $LS ${TEMP_SCHED_DIR}/run >> ${LOGFILE}
                        fi
                        echo >> ${LOGFILE}
                fi
        fi
 
        return 0
}
 
 
### Function: abort_script ###
#   This will is called if the script is aborted thru an error
#   error signal sent by the kernel such as CTRL-C or if a serious
#   error is encountered during runtime
#
# Arguments:
#       $1 - Error message from part of program
#       $2 - Return Code
#
# Return Values:
#          none
###
abort_script()
{
        if [ "$1" ]
        then
                _err_msg_="$1 [rc=$2]"
        else
                echo "Stream $STREAM_NUMBER: Trapped Interrupt `$DATE`)" >>${LOGFILE}
                _err_msg_="Script aborted.......[rc=$2]\n"
        fi
 
        if [ "${LOGFILE}" ]
        then
                if [ ! -f "${LOGFILE}" ]
                then
                        chk_create_logfile 2>/dev/null
                fi
                echo "\n${_err_msg_}\n" >> ${LOGFILE}
        else
                echo "\n${_err_msg_}\n"
        fi
        unset _err_msg_
 
        echo ${OMBS_POLICY_ID} | $EGREP "SNMS|OSS|ENIQ_STATS|ENIQ_EVENTS|_VX" >/dev/null 2>&1
        if [ $? -eq 0 ] && [ ${STREAM_NUMBER} != 0 ]
        then
                peek_at_temp_sched_dir 1
                cd $SCRIPTHOME
 
                #*************************************************#
                # Seems we cannot assume STREAM 1 is first Stream #
                #*************************************************#
                $MKDIR -p ${TEMP_SCHED_DIR} 2>/dev/null
                if [ ! -f ${TEMP_SCHED_DIR}/first_stream ]
                then
                        #---------------------------------#
                        # ok, lets try to be first stream #
                        #---------------------------------#
                        echo $STREAM_NUMBER >${TEMP_SCHED_DIR}/first_stream 2>/dev/null
                fi
 
                #---------------------------------------------#
                # Confirm who is first (was someone quicker?) #
                #---------------------------------------------#
                FIRST_STREAM=` $CAT ${TEMP_SCHED_DIR}/first_stream | $BC `
                ${ECHO} "I am stream [$STREAM_NUMBER], First stream is [$FIRST_STREAM] ..." >> ${LOGFILE}
 
                if [ -f "${ABORT_FILE}" ] || [ $STREAM_NUMBER -eq $FIRST_STREAM ]
                then
                        case ${OMBS_POLICY_ID} in
                        _SNMS)          ###Cleanup for SNMS Backup###
                                        ${PS} -ef | $GREP "${SNMS_BACKUP} -d" | $GREP -v grep >/dev/null 2>&1
                                        if [ $? -ne 0 ]
                                        then
                                                ${SNMS_BACKUP} -d >>${LOGFILE} 2>&1
                                                ABORTING_BACKUP=true
                                        fi
                                        ;;
                          *)            ABORTING_BACKUP=true
                                        ;;
 
                        esac
 
                        if [ "${ABORTING_BACKUP}" = true ]
                        then
                                echo "Stream: [${STREAM_NUMBER}]:" >>${LOGFILE}
                                if [ -d ${TEMP_SCHED_DIR} ]
                                then
                                        if [ ! -f "${ABORT_FILE}" ]
                                        then
                                                $TOUCH ${ABORT_FILE} 2>&1
                                                echo "- created [${ABORT_FILE}] to abort backup!" >>${LOGFILE}
                                        fi
                                        if [ -f ${TEMP_SCHED_DIR}/backup_prepared ]
                                        then
                                                echo "- removing file [${TEMP_SCHED_DIR}/backup_prepared]" >>${LOGFILE}
                                                $RM ${TEMP_SCHED_DIR}/backup_prepared 2>/dev/null
                                        fi
                                else
                                        echo "- Directory [${TEMP_SCHED_DIR}] not found." >>${LOGFILE}
 
                                fi
                        fi
                fi
        fi
 
        if [ ${STREAM_NUMBER} != 0 ]
        then
                #-------------------------------------------------------------------------------------#
                # For multi-stream backups, merge bpstart_notify.<x> to main bpstart_notify_log file. #
                #-------------------------------------------------------------------------------------#
                touch ${LOGDIR}/${LOGNAME} 2>/dev/null
 
                $CAT ${LOGFILE} >>${LOGDIR}/${LOGNAME} 2>/dev/null
                $RM ${LOGFILE} 2>/dev/null                              ##delete bpstart_notify_log.<num>#
        else
                $MV ${LOGFILE} ${LOGFILE}.aborted.${RUN_TIME} 2>/dev/null
        fi
 
        if [ "$2" ]
        then
                exit ${2}
        else
                exit 919
        fi
}
 
### Function: check_id ###
#   Check that the effective id of the user is correct
#   If not print error msg and exit.
#
# Arguments:
#          $1 : User ID name
# Return Values:
#          none
###
check_id()
{
        _check_id_=` $ID  | $AWK -F\( '{print $2}' | $AWK -F\) '{print $1}' `
        if [ "$_check_id_" != "$1" ]
        then
                _err_msg_="You must be $1 to execute this script."
                abort_script "$_err_msg_" 201
        fi
 
        return 0
}
 
### Function: chk_create_logfile ###
# Check/Create Logfile
#
# Arguments:
#       none
# Return Values:
#       none
##
chk_create_logfile()
{
        $MKDIR -p ` $DIRNAME ${LOGFILE} `
        if [ $? -ne 0 ]
        then
                unset LOGFILE
                _err_msg_="Could not create directory `$DIRNAME ${LOGFILE}`"
                abort_script "$_err_msg_" 202
        fi
 
        $TOUCH -a ${LOGFILE}
        if [ $? -ne 0 ]
        then
                unset LOGFILE
                _err_msg_="Could not write to file ${LOGFILE}"
                abort_script "$_err_msg_" 203
        fi
 
        return 0
}
 
### Function: get_absolute_path ###
# Determine absolute path to software
#
# Arguments:
#       none
# Return Values:
#       none
###
get_absolute_path()
{
        _dir_=` $DIRNAME $0 `
        SCRIPTHOME=` cd $_dir_ 2>/dev/null && pwd || echo $_dir_ `
 
        return 0
}
 
### Function: touch_bpstart_called ###
# Log the fact that backup is starting
#
# Arguments:
#       none
# Return Values:
#       none
touch_bpstart_called()
{
        ###Create/Overwrite BPSTART_CALLED file###
        if [ ${STREAM_NUMBER} -eq 0 ]
        then
                echo >${OUTF1} 2>/dev/null
                echo >${OUTF2} 2>/dev/null
        else
                if [ ! -f ${TEMP_SCHED_DIR}/first_stream ]
                then
                        echo >${OUTF1} 2>/dev/null
                        echo >${OUTF2} 2>/dev/null
                else
                        FIRST_STREAM=`$CAT ${TEMP_SCHED_DIR}/first_stream | $BC `
                        if [ ${STREAM_NUMBER} -eq $FIRST_STREAM ]
                        then
                                echo >${OUTF1} 2>/dev/null
                                echo >${OUTF2} 2>/dev/null
                        fi
                fi
        fi
 
        echo >>${LOGFILE}
        echo "[JOB ARRIVED]" >>${LOGFILE}
        case ${SCHED_TYPE} in
        FULL)   echo >>${OUTF1}
                echo "Full backup starting on ${BKUP_CLIENT}" >>${OUTF1}
                echo "Full backup starting on ${BKUP_CLIENT}" >>${LOGFILE}
                echo " - Policy ${POLICY_ID} Schedule ${SCHED_NAME} ${RUN_TIME}" >>${OUTF1}
                ;;
        INCR)   echo >>${OUTF1}
                echo "Differential Incremental backup starting on ${BKUP_CLIENT}" >>${OUTF1}
                echo "Differential Incremental backup starting on ${BKUP_CLIENT}" >>${LOGFILE}
                echo " - Policy ${POLICY_ID} Schedule ${SCHED_NAME} ${RUN_TIME}" >>${OUTF1}
                ;;
        CINC)   echo >>${OUTF1}
                echo "Cumulative Incremental backup starting on ${BKUP_CLIENT}"   >>${OUTF1}
                echo "Cumulative Incremental backup starting on ${BKUP_CLIENT}"   >>${LOGFILE}
                echo " - Policy ${POLICY_ID} Schedule ${SCHED_NAME} ${RUN_TIME}" >>${OUTF1}
                ;;
        esac
 
        #Save environment#
        ${ENV} >>${OUTF1} 2>/dev/null
        echo "************************" >>${OUTF1}
        echo >>${OUTF1}
        echo >>${OUTF1}
 
        return 0
}
 
 
chk_args()
{
        #--------------------------------------#
        # Ensure we have sufficient arguments. #
        #--------------------------------------#
        if [ ${#} != 4 ]
        then
                echo "-- Date: ${RUN_TIME} --" >>${OUTF1} 2>/dev/null
                echo "Args: [${*}]" >>${OUTF1} 2>/dev/null
                printf "\n[ ` $BASENAME ${0}` ) needs 4 args!]\n" >>${OUTF1} 2>/dev/null
                echo "--" >>${OUTF1} 2>/dev/null
 
                _err_msg_="The script [$0], requires 4 args."
                abort_script "$_err_msg_" 216
        fi
 
        return 0
}
 
cancel_backup_prepare()
{
        # This may be because stream one failed or because the file is left
        # hanging around from a previous run so we should quit!
        echo "============================================================" >> ${LOGFILE}
        echo "============================================================" >> ${LOGFILE}
        echo "PREPARATION FOR BACKUP CANCELLED AT `$DATE`." >> ${LOGFILE}
        echo "FILE ${ABORT_FILE} DETECTED" >> ${LOGFILE}
        echo "============================================================" >> ${LOGFILE}
        echo "============================================================" >> ${LOGFILE}
        peek_at_temp_sched_dir 1
 
        echo "Stream [$STREAM_NUMBER]: Found [${ABORT_FILE}]" >> ${LOGFILE}
        _err_msg_="- The last backup did not cleanup OR a stream aborted backup (check bpend logfile)."
        abort_script "$_err_msg_" 127
 
        return 0
}
 
 
check_for_timed_out_streams()
{
 
        #********************************************#
        # Cancel backup if a recent stream timed out #
        #********************************************#
        A_FILE=${LOGDIR}/${LOGNAME}
        if [ -f ${A_FILE} ]
        then
                $GREP "timed out waiting for backup" ${A_FILE} >/dev/null 2>&1
                if [ $? -eq 0 ]
                then
                        $TOUCH ${ABORT_FILE} 2>/dev/null
                        echo "Stream: ${STREAM_NUMBER}" >>${LOGFILE}
                        echo "- Found a timed out stream in [${A_FILE}]" >>${LOGFILE}
                        TIMED_OUT=`$GREP "timed out waiting for" ${A_FILE} |$SORT |$UNIQ |$AWK '{print $2}' `
                        for A_VAL in $TIMED_OUT
                        do
                                if [ -f ${A_FILE}.${A_VAL} ]
                                then
                                        $CAT ${A_FILE}.${A_VAL} >>${A_FILE} 2>/dev/null
                                        $RM ${A_FILE}.${A_VAL} 2>/dev/null
                                fi
                                if [ -f ${LOGDIR}/bpend_notify_log.${A_VAL} ]
                                then
                                        $CAT ${LOGDIR}/bpend_notify_log.${A_VAL} >>${LOGDIR}/bpend_notify_log
                                        $RM ${LOGDIR}/bpend_notify_log.${A_VAL} 2>/dev/null
                                fi
                                echo "--> Found failed stream: [${A_VAL}]" >>${LOGFILE}
                        done
                        cancel_backup_prepare
                fi
        fi
 
        return 0
}
 
 
loop_until_backup_is_prepared()
{
        ###Did another Stream timeout?###
        if [ -f ${ABORT_FILE} ]
        then
                cancel_backup_prepare
        fi
 
        #********************************************************************#
        # Loop until ready to proceed. I will try for limited amount of time #
        # until backup preparation is ready. If time exceeds then error.     #
        #********************************************************************#
        _loop_count_=1
        if [ ${STREAM_NUMBER} -gt 60 ]
        then
                _sleep_cntr_=`$EXPR ${STREAM_NUMBER} - 60`
        else
                _sleep_cntr_=`$EXPR ${STREAM_NUMBER} + 5`
        fi
 
        ###Do not proceed until backup is prepared or aborted (other stream)###
        while [ -d ${TEMP_SCHED_DIR} -a ! -f ${TEMP_SCHED_DIR}/backup_prepared ]
        do
                if [ ${_loop_count_} -eq 1 ]
                then
                        $ECHO
                        $ECHO ". . waiting for backup preparation . . " >> ${LOGFILE}
                fi
                _loop_count_=`$EXPR ${_loop_count_} + 1`
                if [ ${_loop_count_} -ge ${BPSTART_TIMEOUT} ]
                then
                        _err_msg_="Stream ${STREAM_NUMBER} timed out waiting for backup preparation"
                        abort_script "$_err_msg_" 100
                fi
                if [ ${DEBUG_MODE} -eq 1 ]
                then
                        $ECHO "Stream ${STREAM_NUMBER}: Wait $count `$DATE`." >> ${LOGFILE}
                fi
                if [ -f ${ABORT_FILE} ]
                then
                        break
                fi
                $SLEEP ${_sleep_cntr_}
        done
        $ECHO "Stream ${STREAM_NUMBER}: Waited `$EXPR ${_sleep_cntr_} \* ${_loop_count_} `secs." >>${LOGFILE}
 
        ###Did any Stream abort?###
        check_for_timed_out_streams
 
        if [ -f ${TEMP_SCHED_DIR}/backup_prepared ]
        then
                ###The Stream can proceed now###
                $ECHO "Stream ${STREAM_NUMBER} with backup id ${BACKUPID} started at `$DATE`"  >> ${LOGFILE}
                $ECHO "${BACKUPID}"  >> ${TEMP_SCHED_DIR}/run/${STREAM_NUMBER}.startfile
 
                _num_procs_done_=` $LS ${TEMP_SCHED_DIR}/run/*.startfile | $WC -l | $BC 2>/dev/null `
                if [ ${_num_procs_done_} -ge $STREAM_COUNT ]
                then
                        $ECHO "Stream ${STREAM_NUMBER} deemed to be last bpstart (to execute) at `$DATE`"  >> ${LOGFILE}
                        $TOUCH "${TEMP_NODE_DIR}/last_bpstart"
                else
                        $ECHO "Stream ${STREAM_NUMBER} is not last bpstart (to execute) at `$DATE`" >>${LOGFILE}
                fi
        else
                ###We should quit###
                _err_msg_="Stream ${STREAM_NUMBER} is quitting because backup preparation is not done"
                abort_script "$_err_msg_" 130
        fi
 
        return 0
}
 
 
### Function: printlog_job_header ###
#   Print a header to log
# Arguments: none
# Return Values: 0
###
printlog_job_header()
{
        printf "\n*********************************************************************************" >>${LOGFILE} 2>&1
        printf "\nJob: [${1}] [${2}] [${3}] [${4}] @ ${RUN_TIME}" >>${LOGFILE} 2>&1
        printf "\n***********************************************************************************" >>${LOGFILE} 2>&1
        if [ "${STREAM_NUMBER}" ]
        then
                printf "\nStream:\t${STREAM_NUMBER} of ${STREAM_COUNT}\n" >>${LOGFILE} 2>&1
        fi
        date >>${LOGFILE} 2>&1
        echo >>${LOGFILE} 2>&1
 
        return 0
}
 
### Function: rotate_compress_logs ###
#
# Rotate and compress the logs if necessary.
#
# Arguments:
#       $1 - Logfile
#       $2 - Num Logs to keep. This number is addition to the original log
#       $3 - Size Log in Kbytes (0 means size not relevant. Rotation always occurs)
#       $4 - Param to compress log or not (y/n)
# Return Values:
#       0 : Rotation sucessful
rotate_compress_logs()
{
        # Save Params
        _logfile_=${1}
        _numlog_=${2}
        _sizelog_kbytes_=${3}
        _compress_log_=${4}
 
        if [ ! "${_logfile_}" -o ! "${_numlog_}" -o ! "${_sizelog_kbytes_}" -o ! "${_compress_log_}" ]
        then
                echo "Incorrect Parameters supplied to function rotate_compress_logs"
                return 1
        fi
 
        if [ ! -s "${_logfile_}" ]
        then
                return 0
        fi
 
        # Test for numeric
        echo ${_numlog_} | $EGREP '[^0-9]' >> /dev/null 2>&1
        if [ $? -eq 0 ]
        then
                echo "Numeric value only for param 2 in function rotate_compress_logs"
                return 1
        fi
 
        # Test for numeric
        echo ${_sizelog_kbytes_} | $EGREP '[^0-9]' >> /dev/null 2>&1
        if [ $? -eq 0 ]
        then
                echo "Numeric value only for param 3 in function rotate_compress_logs"
                return 1
        fi
 
        # Param must be either y/n
        if [ "${_compress_log_}" != y -a "${_compress_log_}" != n ]
        then
                echo "Param 4 can only have (y/n) value in function rotate_compress_logs"
                return 1
        fi
 
        # Is the _logfile_ big enough to need rotation
        if [ ${_sizelog_kbytes_} -ne 0 ]
        then
                _chk_size_=`$LS -l ${_logfile_}|$NAWK '{print $5}'`
                if [ ${_chk_size_} -lt ${_sizelog_kbytes_} ]
                then
                        return 0
                fi
        fi
 
        tem_file=/tmp/temfile.$$
        # Loop until I get a file name that does not exist
        while :
        do
                $LS ${tem_file} >> /dev/null 2>&1
                if [ $? -ne 0 ]
                then
                        break
                fi
                tem_file=${tem_file}.$$
        done
        $RM -f  ${tem_file}
 
        $LS ${_logfile_}_#*@* 2> /dev/null \
                    |$EGREP "${_logfile_}_#[1-9]+[0-9]*@[0-9]+-[0-9]+-[0-9]+.[0-9]+.[0-9]+.[0-9]+" \
                    |$SORT -t \# -k 2,2n > ${tem_file}
        num_exist_log=` $CAT ${tem_file} | $WC -l | $BC 2>/dev/null `
        if [ `$EXPR ${num_exist_log} + 1` -ge ${_numlog_} ]
        then
                _cnt_=0
                for _file_ in `$CAT ${tem_file}`
                do
                        _cnt_=`$EXPR ${_cnt_} +  1`
                        if [ ${_cnt_} -ge ${_numlog_} ]
                        then
                            $RM -f ${_file_}
                        fi
                done
        fi
 
        $LS ${_logfile_}_#*@* 2> /dev/null \
                    |$EGREP "${_logfile_}_#[1-9]+[0-9]*@[0-9]+-[0-9]+-[0-9]+.[0-9]+.[0-9]+.[0-9]+" \
                    |$SORT -t \# -k 2,2nr > ${tem_file}
 
        # Rotate the dumps
        for _file_ in `$CAT ${tem_file}`
        do
                _file_num_=`echo ${_file_}|$NAWK -F\# '{print $2}'|$NAWK -F\@ '{print $1}'`
                _new_file_num_=`$EXPR ${_file_num_} + 1`
                _new_file_name_=`echo ${_file_}|$SED -e "s|#${_file_num_}@|#${_new_file_num_}@|"`
                $MV -f ${_file_} ${_new_file_name_}
        done
        $RM -f  ${tem_file}
 
        _run_time_=`$DATE '+%Y-%m-%d_%H.%M.%S'`
        $MV ${_logfile_} ${_logfile_}_#1@${_run_time_}
        if [ "${_compress_log_}" = y ]
        then
                $COMPRESS -f ${_logfile_}_#1@${_run_time_} 2>/dev/null &
        fi
 
        return 0
}
 
 
### Function: solaris_client_checks ###
#   If its Solaris, do a few SVM, X86 checks
# Arguments: none
# Return Values: 0
###
solaris_client_checks()
{
        ###Ensure a swap-device is configured on the client###
        if [ ! -f /tmp/bpstart_noswapcheck ]
        then
                swap -l | grep -i "No swap devices configured" >/dev/null 2>&1
                if [ $? -eq 0 ]
                then
                        _err_msg_="No swap devices configured on this client, fix this situation!"
                        abort_script "$_err_msg_" 869
                fi
        fi
 
        ###This is for any Solaris client ###
        if [ "${I386}" ]
        then
                if [ -s /lib/libc.so.1 ]
                then
                        echo "Copying libc.so.1 to .save ........ `$DATE`"  >>${LOGFILE}
                        ###This is because of a limitation in Veritas Netbackup###
                        ${MKDIR} -p  /tmp/lib
                        mount -o nosub -F lofs /lib /tmp/lib
 
                        ${CP} -p /tmp/lib/libc.so.1 /lib/libc.so.1.save 2>/dev/null
                        if [ $? -ne 0 ]
                        then
                                umount /tmp/lib
                                rmdir /tmp/lib
 
                                _err_msg_="Failure making copy of /lib/libc.so.1"
                                abort_script "$_err_msg_" 861
                        fi
                        umount /tmp/lib
                        rmdir /tmp/lib
                else
                        _err_msg_="Could not locate /lib/libc.so.1 to make backup of"
                        abort_script "$_err_msg_" 862
                fi
        fi
 
        if [ ! -s "${SAVE_CONFIG}" ]
        then
                _err_msg_="Could not locate [${SAVE_CONFIG}]"
                if [ ! -s "${ALT_SAVE_CONFIG}" ]
                then
                        abort_script "$_err_msg_" 204
                fi
                SAVE_CONFIG=${ALT_SAVE_CONFIG}
        fi
 
        ###Save configuration of system for OMBS BMR###
        L_CMD="/usr/bin/bash ${SAVE_CONFIG}"
        ${L_CMD} >>${LOGFILE} 2>&1
        if [ $? -ne 0 ]
        then
                echo "[ombs rc=205] Error, command [ ${L_CMD} ] failed!" >>${LOGFILE} 2>&1
                _err_msg_="Failure saving config data for Bare Metal Restore (ombs)."
                abort_script "$_err_msg_" 205
        fi
 
        save_svm_info
 
        ###backup vfstab###
        ${CP} /etc/vfstab /etc/vfstab_bmr >/dev/null 2>&1
 
        #-------------------------------------------------------------
        # List filesystems that exist before executing bpstart_notify
        #-------------------------------------------------------------
        printf "\n\n*******************" >>${LOGFILE} 2>&1
        printf "\nFilesystem Listing" >>${LOGFILE} 2>&1
        printf "\n*******************\n\n" >>${LOGFILE} 2>&1
        ${DF} -h  >>${LOGFILE} 2>&1
        printf "\n====================================================================\n" >>${LOGFILE} 2>&1
 
        return 0
}
 
 
### Function: streamed_data_policy ###
# Do what is required pre-backup of Multiple Stream DATA Policy #
# This operates on the principle that First STREAM_NUMBER will do #
# all the backup preparation. All other streams will sleep while#
# preparation is going on.                                      #
#
# Arguments:
#       none
# Return Values:
#       none
##
streamed_data_policy()
{
  _sleep_cntr_=0
 
        # Check for command-line execution - are enough variables set. #
        if [ -z "${STREAM_NUMBER}" ]
        then
                echo "No STREAM_NUMBER environment variable set!"
                return 1212
        fi
        if [ -z "${STREAM_COUNT}"  ]
        then
                echo "No STREAM_COUNT  environment variable set!"
                return 1213
        fi
 
        # Are multi-streams allowed for backup
        if [ ${STREAM_NUMBER} -eq 0 ]
        then
                # Just one Stream is present #
 
                echo "\n============================================================" >> ${LOGFILE}
                echo "============================================================\n" >> ${LOGFILE}
                echo "PREPARATION FOR SINGLE-STREAMED BACKUP STARTED AT `$DATE`." >> ${LOGFILE}
                echo "\n============================================================" >> ${LOGFILE}
                echo "============================================================\n" >> ${LOGFILE}
 
                if [ ${DEBUG_MODE} -eq 1 ]
                then
                        $ENV | $SED "s/^/Stream ${STREAM_NUMBER}: /" >> ${LOGFILE}
                fi
 
                ###########################
                ### OMBS Policy Startup ###
                ###########################
                echo ${OMBS_POLICY_ID} | grep "_ONBLADE_RAW" >/dev/null 2>&1
 
                if [ $? -ne 0 ]
                then
                        solaris_client_checks
                fi
 
                case ${OMBS_POLICY_ID} in
                _SNMS)          ###### SNMS  ######
                                snms_prepare_for_backup
                                ;;
                *)              ###### Other ######
                                ${ECHO} "Multiple Streams should be enabled for this Policy" >>${LOGFILE}
                                ;;
                esac
                ################
                ###### OK  #####
                ################
        else
                # More than one stream is present #
                #**********************************************************************#
                # Remove the end file for the stream (if exists). This is required for
                # termination condition if we are on a retry scenario.
                #**********************************************************************#
                if [ -f  ${TEMP_SCHED_DIR}/run/${STREAM_NUMBER}.endfile ]
                then
                        if [ ${DEBUG_MODE} -eq 1 ]
                        then
                                echo "Stream ${STREAM_NUMBER}: REMOVED PREVIOUS ENDFILE at `$DATE`" >> ${LOGFILE}
                        fi
                        $RM -f ${TEMP_SCHED_DIR}/run/${STREAM_NUMBER}.endfile 2>/dev/null
                fi
                check_for_timed_out_streams
 
                #*************************************************#
                # Seems we cannot assume STREAM 1 is first Stream #
                #*************************************************#
                $MKDIR -p ${TEMP_SCHED_DIR} 2>/dev/null
                if [ ! -f ${TEMP_SCHED_DIR}/first_stream ]
                then
                        if [ -f ${TEMP_NODE_DIR}/secound_node  ]
                        then
                                SECOUND_NODE=` $CAT ${TEMP_NODE_DIR}/secound_node`
                                if [ ${SECOUND_NODE} = ${BKUP_CLIENT} ]
                                then
                                        #---------------------------------#
                                        # do not need the prepare backup, make first stream to 0#
                                        #---------------------------------#
                                        echo 0 >${TEMP_SCHED_DIR}/first_stream 2>/dev/null
                                else
                                        echo $STREAM_NUMBER >${TEMP_SCHED_DIR}/first_stream 2>/dev/null
                                fi
                        else
 
                                #---------------------------------#
                                # ok, lets try to be first stream #
                                #---------------------------------#
                                echo $STREAM_NUMBER >${TEMP_SCHED_DIR}/first_stream 2>/dev/null
                        fi
                fi
 
                #---------------------------------------------#
                # Confirm who is first (was someone quicker?) #
                #---------------------------------------------#
                FIRST_STREAM=` $CAT ${TEMP_SCHED_DIR}/first_stream | $BC `
 
                #********************#
                # Am I first stream? #
                #********************#
                if [ $STREAM_NUMBER -eq $FIRST_STREAM ]
                then
                        echo "First Stream deemed to be  $STREAM_NUMBER" >>${LOGFILE}
                        #**********************************************************************#
                        # If so, prepare for backup
                        #**********************************************************************#
                        peek_at_temp_sched_dir 1
                        $MKDIR -p ${TEMP_SCHED_DIR}/run 2>/dev/null
 
                        ###Did a Stream abort; Or last backup failed and cleanup did not occur?###
                        if [ -f ${ABORT_FILE} ]
                        then
                                cancel_backup_prepare
                        fi
 
                        # Okay prepare for backup
                        if [ ! -f ${TEMP_SCHED_DIR}/backup_prepared ]
                        then
                                echo "\n============================================================" >> ${LOGFILE}
                                echo "============================================================" >> ${LOGFILE}
                                echo "PREPARATION FOR MULTI-STREAMED BACKUP STARTED AT `$DATE`." >> ${LOGFILE}
                                echo "============================================================" >> ${LOGFILE}
                                echo "============================================================\n" >> ${LOGFILE}
                                if [ ${DEBUG_MODE} -eq 1 ]
                                then
                                        $ENV | $SED "s/^/Stream ${STREAM_NUMBER}: /" >> ${LOGFILE}
                                        echo "Stream ${STREAM_NUMBER}: Preparing for backup `$DATE`" >> ${LOGFILE}
                                fi
 
                                # Prepare for the backup
                                ###########################
                                ### OMBS Policy Startup ###
                                ###########################
                                echo ${OMBS_POLICY_ID} | grep "_ONBLADE_RAW" >/dev/null 2>&1
                                if [ $? -ne 0 ]
                                then
                                        solaris_client_checks
                                fi
 
                                save_vxvm_info
 
                                case ${OMBS_POLICY_ID} in
                                _ENIQ_EVENTS)                   ###### NIQ ENIQ_EVENTS  ######
                                                                events_prepare_backup
                                                                ;;
                                _ENIQ_STATS_MULTIBLADE_DATA)    ###### NIQ Statistics  ######
                                                                statistics_prepare_backup
                                                                ;;
                                _ENIQ_STATS_ONBLADE_DATA)       ###### NIQ Statistics  ######
                                                                statistics_prepare_backup
                                                                ;;
                                _ENIQ_STATS_ONBLADE_RAW)       ###### NIQ Statistics  ######
                                                                statistics_prepare_backup_raw
                                                                ;;
 
                                _ENIQ_STATS)                    ###### NIQ Statistics  ######
                                                                statistics_prepare_backup
                                                                ;;
 
                                _SNMS)                          ###### SNMS  ######
                                                                snms_prepare_for_backup
                                                                ;;
 
                                _OSS|_HARS)                     ###### OSS_<hostname> and OSS_HARS_<hostname> Policies #####
                                                                oss_ss_detach_mount_offline_mirror
                                                                ;;
 
                                _OSS_i386|_HACS|_DDS|_VX)       # Cluster or Multi-Node (nbuclinet waits until 1st splits mirrors #
                                                                are_vxvm_mirrors_split_yet
                                                                failed_over_ha_check
                                                                case ${?} in
                                                                0)
                                                                FIRST_NODE=` $CAT ${TEMP_NODE_DIR}/first_node_${STREAM_COUNT} `
                                                                if [ "$FIRST_NODE" = "$BKUP_CLIENT" ]
                                                                then
                                                                        oss_ss_detach_mount_offline_mirror
                                                                fi
                                                                ;;
                                                                1)      start_nbuclient_process
                                                                        are_vxvm_mirrors_split_yet
                                                                        ;;
                                                                esac
                                                                ;;
                                esac
                                ################
                                ###### OK  #####
                                ################
 
                                if [ $? -ne 0 ]
                                then
                                        _err_msg_="Fault encountered preparing for backup"
                                        abort_script "$_err_msg_" 934
                                fi
 
                                $TOUCH ${TEMP_SCHED_DIR}/backup_prepared
                                $TOUCH ${TEMP_NODE_DIR}/backup_prepared
                                if [ ${DEBUG_MODE} -eq 1 ]
                                then
                                        echo "Stream ${STREAM_NUMBER}: Backup Prepared `$DATE`" >> ${LOGFILE}
                                fi
                        else
                                # This may be because stream one failed and is on a retry or could #
                                # be because the file is left hanging around from a previous run.  #
                                echo "============================================================" >> ${LOGFILE}
                                echo "============================================================" >> ${LOGFILE}
                                echo "PREPARATION FOR BACKUP SKIPPED AT `$DATE`." >> ${LOGFILE}
                                echo "FILE ${TEMP_SCHED_DIR}/backup_prepared DETECTED" >> ${LOGFILE}
                                echo "============================================================" >> ${LOGFILE}
                                echo "============================================================" >> ${LOGFILE}
                                peek_at_temp_sched_dir 1
                        fi
                else
                        ##### This is not STREAM ONE #####
 
                        # Introduce some sort of randomness
                        if [ ${STREAM_NUMBER} -gt 60 ]
                        then
                                _sleep_cntr_=`$EXPR ${STREAM_NUMBER} - 60`
                        else
                                _sleep_cntr_=${STREAM_NUMBER}
                        fi
                        $SLEEP ${_sleep_cntr_}
 
                        if [ ${DEBUG_MODE} -eq 1 ]
                        then
                                $ENV | $SED "s/^/Stream ${STREAM_NUMBER}: /" >> ${LOGFILE}
                                echo "Stream ${STREAM_NUMBER}: Backup started on `$DATE`" >> ${LOGFILE}
                                peek_at_temp_sched_dir 1
                        fi
                fi
        fi
        loop_until_backup_is_prepared
 
        echo "Stream $STREAM_NUMBER: proceeding $BACKUPID `$DATE`" >>${LOGFILE}
        echo >>${LOGFILE}
 
        #----------------------------------------------------------------#
        # Merge/Move bpstart_notify.<x> to main bpstart_notify_log file. #
        #----------------------------------------------------------------#
        if [ ! "${LOGDIR}/${LOGNAME}" ]
        then
                touch ${LOGDIR}/${LOGNAME} 2>/dev/null
        fi
        cat ${LOGFILE} >>${LOGDIR}/${LOGNAME} 2>/dev/null
        $RM ${LOGFILE} 2>/dev/null                              ##delete bpstart_notify_log.<num>#
 
        return 0
}
 
### Function: check_svm_metadevices_config ###
# Check if Plexes are being attached.
#
# Arguments:
#       none
# Return Values:
#       none
###
check_svm_metadevices_config()
{
  L_CNT_SUBMIRR=0
  _sleep_cntr_=0
 
        if [ -s "${METASTAT}" ]
        then
                ${METASTAT} | grep "there are no existing databases" >/dev/null
                if [ $? -ne 0 ]
                then
                        L_SVM_MIRRORS=` ${METASTAT} | grep Mirror | awk -F":" '{print $1}' `
 
                        for L_MIRR in ${L_SVM_MIRRORS}
                        do
                                ###how many submirrors attached to metadevice###
                                printf "\nMirror Device [${L_MIRR}] has " >>${LOGFILE}
                                L_CNT_SUBMIRR=` ${METASTAT} ${L_MIRR} | grep "Submirror of" | wc -l | bc `
                                printf "${L_CNT_SUBMIRR} SubMirrors\n" >>${LOGFILE} 2>&1
                        done
 
                        ###Are metadevices attaching?###
                        if [ ${STREAM_NUMBER} -gt 60 ]
                        then
                                _sleep_cntr_=`$EXPR ${STREAM_NUMBER} - 60`
                        else
                                _sleep_cntr_=${STREAM_NUMBER}
                        fi
                        while :
                        do
                                ###Are mirrors okay?###
                                ${METASTAT} | grep "Resync in progress" >/dev/null
                                if [ $? -eq 0 ]
                                then
                                        echo "A Mirror appears to be resyncing: Sleep ${_sleep_cntr_}secs" >>${LOGFILE} 2>&1
                                        $SLEEP ${_sleep_cntr_}
                                else
                                        break
                                fi
                        done
                fi
        fi
 
        return 0
}
 
 
### Function: save_svm_info ###
# Do some checks on svm state (metastat)
#
# Arguments:
#       none
# Return Values:
#       none
###
save_svm_info()
{
        if [ -f "${METASTAT}" ]
        then
                ###Save metastat status###
                printf "\n\n********************" >>${LOGFILE} 2>&1
                printf "\nSVM METASTAT STATUS" >>${LOGFILE} 2>&1
                printf "\n********************\n" >>${LOGFILE} 2>&1
 
                if [ -x "${METASTAT}" ]
                then
                        ${METASTAT} >>${LOGFILE} 2>&1
                else
                        err_msg_="The command [${METASTAT}] is not executable"
                        abort_script "$_err_msg_" 237
                fi
                printf "\n==============================================================\n" >>${LOGFILE} 2>&1
 
                #------------------------------------------
                # Check that metadevices are not syncing
                #------------------------------------------
                check_svm_metadevices_config
        fi
 
        return 0
}
 
 
 
#********************************************************************
# OSS Functions
#********************************************************************
 
 
### Function: ha_first_node_check ###
# If a Netbackup policy has multiple backup clients (virutal ipaddrs)
# belonging to a High Availability system then at some point these
# virtual ipaddrs could exist on same physical server? FailOver condition?
#
# In this situation multiple backup clients are running on same server.
# Set a flag to state "who is first node" (usally myself, but ...)
#
# Arguments:
#       none
# Return Values:
#       none
###
ha_first_node_check()
{
        ### VERITAS VCS ###
        ${PKGINFO} VRTSvcs >/dev/null 2>&1
        if [ $? -ne 0 ]
        then
                return 0
        fi
        if [ ${STREAM_COUNT} -eq 0 ]
        then
                return 0
        fi
 
        #************************************************#
        # Try to be first Node (i.e. Failedover Cluster) #
        #************************************************#
        $MKDIR -p ${TEMP_SCHED_DIR}/run 2>/dev/null
        if [ ! -f ${TEMP_NODE_DIR}/first_node_${STREAM_COUNT} ]
        then
                #---------------------------------#
                # ok, lets try to be first stream #
                #---------------------------------#
                $ECHO $BKUP_CLIENT >${TEMP_NODE_DIR}/first_node_${STREAM_COUNT} 2>/dev/null
                $TOUCH ${TEMP_NODE_DIR}/wait_for_node1_stream1 2>/dev/null
                MOUNTPOINTS="` ${OMBSC_HOME}/bin/get_sync_filesystem `"
                for MPOINT in ${MOUNTPOINTS}
                do
                        ${MKDIR} -p ${MPOINT} >/dev/null 2>&1
                done
        fi
 
        #---------------------------------------------#
        # Confirm who is first (was someone quicker?) #
        #---------------------------------------------#
        FIRST_NODE=` $CAT ${TEMP_NODE_DIR}/first_node_${STREAM_COUNT} `
        $ECHO "${BACKUPID}"  >> ${TEMP_SCHED_DIR}/run/${STREAM_NUMBER}.startfile
        _num_procs_started_=` $LS ${TEMP_SCHED_DIR}/run/*.startfile | $WC -l | $BC 2>/dev/null `
 
        #---------------------------------------------------------------#
        # Due to default OMBS setting (max jobs per client=1) Netbackup #
        # allows each node equal priority for FIRST STREAM only before  #
        # giving NODE1 full priority (other nodes/streams suspended).   #
        #---------------------------------------------------------------#
        COUNTER=50
        if [ "$FIRST_NODE" = "$BKUP_CLIENT" ]
        then
                if [ "${_num_procs_started_}" -eq 1 ]
                then
                        sleep 5
                        WAIT_FILES="${TEMP_NODE_DIR}/wait_for_node2_stream1"
                        LET_BACKUP_PROCEED=yes
                else
                        WAIT_FILES="${TEMP_NODE_DIR}/wait_for_node2_stream1 ${TEMP_NODE_DIR}/wait_for_node1_stream1"
                fi
                for A_FILE in ${WAIT_FILES}
                do
                        #----------------------------------------------------------------------------#
                        # FIRST/SECOND STREAM: Waiting for Node2 (1st stream) and Node1 (1st stream) #
                        #----------------------------------------------------------------------------#
                        if [ -f "${A_FILE}" ] && [ -f "${LOGFILE}" ]
                        then
                                ${ECHO} "${BKUP_CLIENT}: Waiting to start Node1 (backup preparation) ...." >>${LOGFILE}
                        fi
 
                        while [ -f "${A_FILE}" ]
                        do
                                ### when this file is gone, we can proceed. ###
                                sleep 5
                                COUNTER=`$EXPR ${COUNTER} + 5`
                                if [ $COUNTER -gt $BPSTART_TIMEOUT ]
                                then
                                        if [ -f "${LOGFILE}" ]
                                        then
                                                ${ECHO} "${BKUP_CLIENT}: Timeout waiting for other Node" >>${LOGFILE}
                                        fi
                                        return 1
                                fi
                        done
                done
                if [ "${LET_BACKUP_PROCEED}" = yes ]
                then
                        ${RM} ${TEMP_NODE_DIR}/wait_for_node1_stream1 >/dev/null 2>&1
                fi
        else
                if [ "${_num_procs_started_}" -eq 1 ]
                then
                        $TOUCH ${TEMP_NODE_DIR}/wait_for_node2_stream1 >/dev/null 2>&1
                else
                        #--------------------------------------------------------------------------#
                        # NOT FIRST NODE/STREAM: We must wait until Node1 has done backup cleanup. #
                        #--------------------------------------------------------------------------#
                        if [ -f "${LOGFILE}" ] && [ -d "${TEMP_NODE_DIR}/${FIRST_NODE}" ]
                        then
                                $ECHO "${BKUP_CLIENT}: Waiting for Node1 to perform backup cleanup ...." >>${LOGFILE}
                        fi
                        while [ -d "${TEMP_NODE_DIR}/${FIRST_NODE}" ]
                        do
                                ### when this directory is gone, we can proceed. ###
                                sleep 5
                                COUNTER=`$EXPR ${COUNTER} + 5`
                                if [ $COUNTER -gt $BPSTART_TIMEOUT ]
                                then
                                        if [ -f "${LOGFILE}" ]
                                        then
                                                ${ECHO} "${BKUP_CLIENT}: Timeout waiting for Node1 to cleanup" >>${LOGFILE}
                                        fi
                                        break
                                fi
                        done
                fi
                return 1
        fi
 
        $ECHO "First Node [${FIRST_NODE}] deemed to be myself: [${BKUP_CLIENT}]" >>${LOGFILE}
 
        return 0
}
 
ha_first_node_check_new()
{
        ### VERITAS VCS ###
        ${PKGINFO} VRTSvcs >/dev/null 2>&1
        if [ $? -ne 0 ]
        then
                return 0
        fi
        if [ ${STREAM_COUNT} -eq 0 ]
        then
                return 0
        fi
 
        #************************************************#
        # Try to be first Node (i.e. Failedover Cluster) #
        #************************************************#
        $MKDIR -p ${TEMP_SCHED_DIR}/run 2>/dev/null
        if [ ! -f ${TEMP_NODE_DIR}/first_node_${STREAM_COUNT} ]
        then
                #---------------------------------#
                # ok, lets try to be first stream #
                #---------------------------------#
                $ECHO $BKUP_CLIENT >${TEMP_NODE_DIR}/first_node_${STREAM_COUNT} 2>/dev/null
                $TOUCH ${TEMP_NODE_DIR}/wait_for_node1_stream1 2>/dev/null
                MOUNTPOINTS="` ${OMBSC_HOME}/bin/get_sync_filesystem `"
                for MPOINT in ${MOUNTPOINTS}
                do
                        ${MKDIR} -p ${MPOINT} >/dev/null 2>&1
                done
        else
                FIRST_NODE=` $CAT ${TEMP_NODE_DIR}/first_node_${STREAM_COUNT} `
                if [ ${FIRST_NODE} != ${BKUP_CLIENT} ]
                then
                        if [ ! -f ${TEMP_NODE_DIR}/secound_node ]
                        then
                                $ECHO $BKUP_CLIENT >${TEMP_NODE_DIR}/secound_node 2>/dev/null
                        fi
                fi
        fi
 
        #---------------------------------------------#
        # Confirm who is first (was someone quicker?) #
        #---------------------------------------------#
        FIRST_NODE=` $CAT ${TEMP_NODE_DIR}/first_node_${STREAM_COUNT} `
        $ECHO "${BACKUPID}"  >> ${TEMP_SCHED_DIR}/run/${STREAM_NUMBER}.startfile
        _num_procs_started_=` $LS ${TEMP_SCHED_DIR}/run/*.startfile | $WC -l | $BC 2>/dev/null `
 
        #---------------------------------------------------------------#
        # Due to default OMBS setting (max jobs per client=1) Netbackup #
        # allows each node equal priority for FIRST STREAM only before  #
        # giving NODE1 full priority (other nodes/streams suspended).   #
        #---------------------------------------------------------------#
        COUNTER=50
        if [ "$FIRST_NODE" = "$BKUP_CLIENT" ]
        then
       # $ECHO "${BACKUPID}"  >> ${TEMP_SCHED_DIR}/run/${STREAM_NUMBER}.startfile
       # _num_procs_started_=` $LS ${TEMP_SCHED_DIR}/run/*.startfile | $WC -l | $BC 2>/dev/null `
 
                if [ "${_num_procs_started_}" -eq 1 ]
                then
                        sleep 5
                        WAIT_FILES="${TEMP_NODE_DIR}/wait_for_node2_stream1"
                        #LET_BACKUP_PROCEED=yes
                else
                        WAIT_FILES="${TEMP_NODE_DIR}/wait_for_node2_stream1 ${TEMP_NODE_DIR}/wait_for_node1_stream1"
                fi
                for A_FILE in ${WAIT_FILES}
                do
                        #----------------------------------------------------------------------------#
                        # FIRST/SECOND STREAM: Waiting for Node2 (1st stream) and Node1 (1st stream) #
                        #----------------------------------------------------------------------------#
                        if [ -f "${A_FILE}" ] && [ -f "${LOGFILE}" ]
                        then
                                ${ECHO} "${BKUP_CLIENT}: Waiting to start Node1 (backup preparation) ...." >>${LOGFILE}
                        fi
 
                        while [ -f "${A_FILE}" ]
                        do
                                ### when this file is gone, we can proceed. ###
                                sleep 5
                                COUNTER=`$EXPR ${COUNTER} + 5`
                                if [ $COUNTER -gt $BPSTART_TIMEOUT ]
                                then
                                        if [ -f "${LOGFILE}" ]
                                        then
                                                ${ECHO} "${BKUP_CLIENT}: Timeout waiting for other Node" >>${LOGFILE}
                                        fi
                                        return 1
                                fi
                        done
                done
                if [ -f ${TEMP_NODE_DIR}/backup_prepared ]
                then
                        LET_BACKUP_PROCEED=yes
                fi
                if [ "${LET_BACKUP_PROCEED}" = yes ]
                then
                        ${RM} ${TEMP_NODE_DIR}/wait_for_node1_stream1 >/dev/null 2>&1
                fi
        else
                        #--------------------------------------------------------------------------#
                        # NOT FIRST NODE/STREAM: We must wait until Node1 has done backup cleanup. #
                        #--------------------------------------------------------------------------#
                        if [ -f "${LOGFILE}" ] && [ -d "${TEMP_NODE_DIR}/${FIRST_NODE}" ]
                        then
                                $ECHO "${BKUP_CLIENT}: Waiting for Node1 to perform backup cleanup ...." >>${LOGFILE}
                        fi
                        while [ -d "${TEMP_NODE_DIR}/${FIRST_NODE}" ]
                        do
                                if [ -f ${TEMP_NODE_DIR}/backup_prepared ]
                                then
                                        ${RM} ${TEMP_NODE_DIR}/wait_for_node1_stream1
                                        break
                                fi
 
                                ### when this directory is gone, we can proceed. ###
                                sleep 5
                                COUNTER=`$EXPR ${COUNTER} + 5`
                                if [ $COUNTER -gt $BPSTART_TIMEOUT ]
                                then
                                        if [ -f "${LOGFILE}" ]
                                        then
                                                ${ECHO} "${BKUP_CLIENT}: Timeout waiting for Node1 to cleanup" >>${LOGFILE}
                                        fi
                                        break
                                fi
                        done
                return 1
        fi
 
        $ECHO "First Node [${FIRST_NODE}] deemed to be myself: [${BKUP_CLIENT}]" >>${LOGFILE}
 
        return 0
}
 
### Function: failed_over_ha_check ###
# Run the hagrp command to see if both sides working correctly
#
# Arguments:
#       none
# Return Values:
#       none
###
failed_over_ha_check()
{
        ### VERITAS VCS ###
        ${PKGINFO} VRTSvcs >/dev/null 2>&1
        if [ $? -ne 0 ]
        then
                return 0
        fi
        echo  >>${LOGFILE} 2>&1
 
        L_SYB_SERVER="` ${HAGRP} -state 2>/dev/null | $GREP 'Sybase1 ' | $GREP ONLINE | $AWK '{print $3}'"
        L_OSS_SERVER="` ${HAGRP} -state 2>/dev/null | $GREP 'Oss '     | $GREP ONLINE | $AWK '{print $3}'"
 
        if [ "$L_SYB_SERVER" = "$L_OSS_SERVER" ]
        then
                echo "-> SYBASE and OSS are on same server - single-node backup required ..."  >>${LOGFILE} 2>&1
                echo  >>${LOGFILE} 2>&1
                return 0
        else
                echo "-> SYBASE and OSS on different servers (HA-CS is online) - multi-node backup required ..."  >>${LOGFILE} 2>&1
                echo  >>${LOGFILE} 2>&1
                return 1
        fi
 
        return 0
}
 
### Function: ensure_vxvm_plexes_are_synced ###
# Check if Plexes are being attached.
#
# Arguments:
#       none
# Return Values:
#       none
###
ensure_vxvm_plexes_are_synced()
{
  _sleep_cntr_=` $EXPR ${STREAM_NUMBER} + 15 `
  A_CNT=1
 
        while :
        do
                ###Are mirrors okay?###
                $VXTASK list | grep PLXATT | $GREP -v grep >/dev/null 2>&1
                if [ $? -eq 0 ]
                then
                        if [ ${A_CNT} -eq 1 ]
                        then
                                ########Dump output from vxtask list to logfile#######
                                echo
                                echo
                                echo "***********************************************************" >>${LOGFILE}
                                $VXTASK list >>${LOGFILE}
                                echo "***********************************************************" >>${LOGFILE}
                                echo
                                echo
                                A_CNT=` $EXPR $A_CNT + 1 `
                        fi
                        echo "Mirrors appear to be syncing: Sleep ${_sleep_cntr_}Secs." >>${LOGFILE}
                        $SLEEP ${_sleep_cntr_}
                else
                        break
                fi
        done
 
        return 0
}
 
 
### Function: are_vxvm_mirrors_split_yet ###
# Check if the VERITAS Volumes have split yet.
#
# Arguments:
#       none
# Return Values:
#       none
###
are_vxvm_mirrors_split_yet()
{
  _sleep_cntr_=` $EXPR ${STREAM_NUMBER} + 15 `
 
        ###Split mirrors using ERICombsc###
        while :
        do
                ###Are we splitting mirrors still?###
                ${PS} -ef | $GREP "${OMBSC_TOOL} de" | $GREP -v grep >/dev/null 2>&1
                if [ $? -eq 1 ]
                then
                        $SLEEP ${_sleep_cntr_}
                        break
                fi
 
                # slight randomizing sleep to try to prevent any race conditions on exit.
                $SLEEP ${_sleep_cntr_}
        done
 
        return 0
}
 
### Function: save_vxvm_info ###
# Do some checks on vxvm licenses, volumes, etc.
#
# Arguments:
#       none
# Return Values:
#       none
###
save_vxvm_info()
{
 
        ${PKGINFO} VRTSvxvm >/dev/null 2>&1
        if [ $? != 0 ]
        then
                return 0
        fi
 
        for L_LIC_CMD in ${VXLICREP} ${VXLICENSE} error
        do
                case ${L_LIC_CMD} in
                error)  _err_msg_="Could not find [${VXLICREP}] or [$VXLICENSE] commands"
                        abort_script "$_err_msg_" 232
                        ;;
 
                *)      [ ! -x "${L_LIC_CMD}" ] && continue
                        printf "\n*************************" >>${LOGFILE} 2>&1
                        printf "\n VERITAS LICENSES LISTING" >>${LOGFILE} 2>&1
                        printf "\n*************************" >>${LOGFILE} 2>&1
                        ${L_LIC_CMD} >>${LOGFILE} 2>&1
                        printf "\n*******************\n\n" >>${LOGFILE} 2>&1
                        break
                        ;;
                esac
        done
 
        ###Check for VERITAS FastSNAP###
        echo >>${LOGFILE} 2>&1
        echo "-> Check for VxVM FastSNAP [FASTRESYNC] feature .." >>${LOGFILE} 2>&1
        if [ -x $VXLICTEST ]
        then
                diskgrps=` $VXDISK list | $AWK '{ print $4 }' | $EGREP -v '^\-$'| $GREP -v GROUP | $SORT -u `
                echo "-> Found Diskgroups [$diskgrps]" >>${LOGFILE} 2>&1
                for dg in $diskgrps
                do
                        vols=` $VXPRINT -g ${dg} -v | $AWK '{print $2}' | $GREP -v NAME | $SORT -u `
                        echo "--> Found volumes [$dg]->[$vols]" >>${LOGFILE} 2>&1
                        echo
                done
 
                ###Turn FMR/Fastsync on/off###
                L_RC=1
                for L_FEATURE in FMR_DGSJ FASTRESYNC
                do
                        $VXLICTEST -n "VERITAS Volume Manager" -f ${L_FEATURE} -sup >/dev/null 2>&1
                        if [ $? -eq 0 ]
                        then
                                $VXLICTEST -n "VERITAS Volume Manager" -f ${L_FEATURE} -sup >>${LOGFILE} 2>&1
                                echo "Licensed feature [${L_FEATURE}] is enabled" >>${LOGFILE} 2>&1
                                L_RC=0
                                break
                        else
                                echo "Licensed feature [${L_FEATURE}] is not enabled" >>${LOGFILE} 2>&1
                        fi
                done
 
                if [ $L_RC -eq 0 ]
                then
                        ###ON###
                        for dg in $diskgrps
                        do
                                for vol in ` $VXPRINT -g ${dg} -v | $AWK '{print $2}' | $GREP -v NAME | $SORT -u `
                                do
                                        echo "---> Process volume [$dg]->[$vol] .." >>${LOGFILE} 2>&1
                                        ACMD="$VXPRINT -g ${dg} -F%fastresync $vol"
                                        ###ACMD="$VXPRINT -g ${dg} -l ${vol} | $GREP fastresync"###not working?###
                                        if [ "` $ACMD `" = off ]
                                        then
                                                $VXVOL -g ${dg} set fastresync=on ${vol} >>${LOGFILE} 2>&1
                                                case $? in
                                                0)      echo "<--- [success] fastresync enabled for [$vol]" >>${LOGFILE} 2>&1
                                                        ;;
                                                *)      echo "<--- [error] fastresync not enabled on [$vol]" >>${LOGFILE} 2>&1
                                                        ;;
                                                esac
                                        else
                                                echo "<--- fastresync already enabled on [$vol]" >>${LOGFILE} 2>&1
                                        fi
                                done
                        done
                else
                        ###OFF###
                        for dg in $diskgrps
                        do
                                for vol in ` $VXPRINT -g ${dg} -v | $AWK '{print $2}' | $GREP -v NAME | $SORT -u `
                                do
                                        echo "---> Process volume [$dg]->[$vol] .." >>${LOGFILE} 2>&1
                                        ACMD="$VXPRINT -g ${dg} -F%fastresync $vol"
                                        ###ACMD="$VXPRINT -g ${dg} -l ${vol} | $GREP fastresync"###not working?###
                                        if [ "` $ACMD `" = on ]
                                        then
                                                $VXVOL -g ${dg} set fastresync=off ${vol} >>${LOGFILE} 2>&1
                                                case $? in
                                                0)      echo "<--- [success] fastresync disabled for [$vol]" >>${LOGFILE} 2>&1
                                                        ;;
                                                *)      echo "<--- [error] fastresync not disabled on [$vol]" >>${LOGFILE} 2>&1
                                                        ;;
                                                esac
                                        else
                                                echo "<--- fastresync already disabled on [$vol]" >>${LOGFILE} 2>&1
                                        fi
                                done
                        done
                fi
        else
                echo "Command [$VXLICTEST] not found" >>${LOGFILE} 2>&1
        fi
 
        echo "<- Done" >>${LOGFILE} 2>&1
        echo >>${LOGFILE} 2>&1
 
        #------------------------------------------------------------------------------
        # List volumes that exist before executing bpstart_notify for this client
        #------------------------------------------------------------------------------
        ${VXPRINT} -htr | grep "^v  " | awk '{print $2}' >>${LOGFILE} 2>&1
        printf "\n*******************\n\n" >>${LOGFILE} 2>&1
 
        #------------------------------------------
        # Check that mirrors are not syncing
        #------------------------------------------
        ensure_vxvm_plexes_are_synced
 
        return 0
}
 
### Function: oss_ss_detach_mount_offline_mirror ####
# Preapare server for backup
#
# Arguments:
#       none
# Return Values:
#       none
###
oss_ss_detach_mount_offline_mirror()
{
        #--------------------------------------------------------------------
        # OSS: Multiple stream backup after Split-Mirror Filesystems mounted
        #--------------------------------------------------------------------
        ${OMBSC_TOOL} de >>${LOGFILE} 2>&1
        if [ $? -ne 0 ]
        then
                _err_msg_="Command [${OMBSC_TOOL} de] failed!"
                $TOUCH $ABORT_FILE 2>/dev/null
                abort_script "$_err_msg_" 229
        fi
        if [ -d "/Netbackup/Netbackup" ]
        then
                $RM -fr /Netbackup/Netbackup 2>/dev/null
        fi
 
        return 0
}
 
### Function: start_nbuclient_process ###
# Prepare server for backup
#
# Arguments:
#       none
# Return Values:
#       none
###
start_nbuclient_process()
{
        #-----------------------------------------------------------------------#
        # nbuClient binary tracing: If more tracing is needed, then change      #
        # INFO from false to true in the SCLITRACE Variable below. The extra    #
        # log output created may need knowledge of C-Language/TCP Sockets to be #
        # of great use to the person analysing the inter-socket communitation.  #
        #-----------------------------------------------------------------------#
        SCLITRACE="ERROR=true:INFO=true:INFO_DETAILS=false:SYSTEM_OUT=true:ENTER=false:RETURN=false"
        export SCLITRACE
 
        #------------------------------------------
        # Execute nbuClient for this Policy stream.
        #------------------------------------------
        if [ -d "${OMBSC_HOME}/lib/${OMBSC_PLAT}" ]
        then
                #--- old OMBS3/OMBS4 client ---#
                env LD_LIBRARY_PATH=${OMBSC_HOME}/lib/${OMBSC_PLAT} /bin/ldd ${OMBSC_HOME}/bin/${OMBSC_PLAT}/nbuClient >>${LOGFILE} 2>&1
                echo >>${LOGFILE} 2>&1
                env LD_LIBRARY_PATH=${OMBSC_HOME}/lib/${OMBSC_PLAT} ${OMBSC_HOME}/bin/${OMBSC_PLAT}/nbuClient >>${LOGFILE} 2>&1
        else
                /bin/ldd ${OMBSC_HOME}/bin/${OMBSC_PLAT}/nbuClient >>${LOGFILE} 2>&1
                echo >>${LOGFILE} 2>&1
                ${OMBSC_HOME}/bin/${OMBSC_PLAT}/nbuClient >>${LOGFILE} 2>&1
        fi
        RC=$?
        if [ $RC -ne 0 ]
        then
                echo "(bpstart_notify) nbuClient Daemon failed!" >>${LOGFILE} 2>&1
                echo >>${LOGFILE} 2>&1
                echo "(tip 1) Ensure HA Cluster(s) are in initial state [hastatus -sum]." >>${LOGFILE} 2>&1
                echo "(tip 2) The hostname of backup-server and policy-clientname must be same vlan." >>${LOGFILE} 2>&1
                echo "\tSo when policy is created and Netbackup client software is pushed to the" >>${LOGFILE} 2>&1
                echo "\tclient, the values [bp.conf:SERVER = <xxxx>, CLIENT = <xxxx>] are sane!" >>${LOGFILE} 2>&1
                echo "\t(Particularly critical for HA-CS Backup Policies)." >>${LOGFILE} 2>&1
                echo "(tip 3) For manual backups, ensure JAVA Console is connected to bkup-vlan-hostname." >>${LOGFILE} 2>&1
                echo "(tip 4) Ensure tape-drives are in [UP] status; check syslog messages." >>${LOGFILE} 2>&1
                echo "(tip 5) Try to get [competing policies] to finish-earlier or start-bit-later." >>${LOGFILE} 2>&1
                echo "(tip 6) Increase Multiplexing value on Storage-Unit if required." >>${LOGFILE} 2>&1
                echo "(tip 7) Ensure [nbuserver] service is running on Backup-Server." >>${LOGFILE} 2>&1
                echo "(tip 8) Verify [/etc/hosts] on backup-server/client for valid entries." >>${LOGFILE} 2>&1
                echo >>${LOGFILE} 2>&1
 
                if [ "${I386}" ]
                then
                        ${GREP} "Error returned from server. Execute house_keeping_rollback.sh script" ${LOGFILE} >/dev/null 2>&1
                        if [ $? -eq 0 ]
                        then
                                echo >>${LOGFILE}
                                echo "******************************************" >>${LOGFILE}
                                echo " OMBS on Solaris X86: TRYING TO RECOVER   " >>${LOGFILE}
                                echo "******************************************" >>${LOGFILE}
                                echo >>${LOGFILE}
                                oss_ss_detach_mount_offline_mirror
                                if [ $? -eq 0 ]
                                then
                                        return 0
                                fi
                        fi
                fi
                _err_msg_="(bpstart_notify) nbuClient Daemon failed! Please read above troubleshooting tips. rc=${RC}"
                $TOUCH $ABORT_FILE 2>/dev/null
                abort_script "$_err_msg_" 360
        fi
 
        return 0
}
 
#********************************************************************
# NIQ ENIQ_EVENTS Functions
#********************************************************************
 
### Function: events_prepare_backup ####
# Prepare server for backup
#
# Arguments:
#       none
# Return Values:
#       none
###
events_prepare_backup()
{
        $ECHO "Preparing Events Server for Backup" >> ${LOGFILE}
        /usr/bin/bash ${PREP_ENIQ_EVENTS_BACKUP} -N >>${LOGFILE} 2>&1
        if [ $? -ne 0 ]
        then
                _err_msg_="Failure preparing ENIQ EVENTS server ${BKUP_CLIENT} for backup"
                $TOUCH $ABORT_FILE 2>/dev/null
                abort_script "$_err_msg_" 222
        fi
 
        return 0
}
 
#********************************************************************
# NIQ Statistics Functions
#********************************************************************
 
### Function: statistics_prepare_backup ####
# Prepare server for backup
#
# Arguments:
#       none
# Return Values:
#       none
###
statistics_prepare_backup()
{
 
        /usr/bin/bash ${PREP_ENIQ_STATS_BACKUP} -N -q >>${LOGFILE} 2>&1
        if [ $? -ne 0 ]
        then
                _err_msg_="Failure preparing Statistics server ${BKUP_CLIENT} for backup"
                $TOUCH $ABORT_FILE 2>/dev/null
                abort_script "$_err_msg_" 208
        fi
 
        return 0
}
statistics_prepare_backup_raw()
{
 
        /usr/bin/bash ${PREP_ENIQ_STATS_BACKUP} -a activate >>${LOGFILE} 2>&1
        if [ $? -ne 0 ]
        then
                _err_msg_="Failure preparing Statistics server ${BKUP_CLIENT} for backup"
                $TOUCH $ABORT_FILE 2>/dev/null
                abort_script "$_err_msg_" 208
        fi
 
        return 0
}
 
#********************************************************************
# SNMS Functions
#********************************************************************
 
### Function: snms_prepare_for_backup ####
# Preapare server for backup
 
# Arguments:
#       none
# Return Values:
#       none
###
snms_prepare_for_backup()
{
        #------------------------------------------------------------
        # SNMS: Multiple stream backup after Filesystems fssnap'ed.
        #------------------------------------------------------------
        ${SNMS_BACKUP} -o >>${LOGFILE} 2>/dev/null
        RC=$?
 
        ###Handle RC###
        # If this file exists, set rc=0 #
        if [ -f /snms_backup/.ignore_nonzero_rc ]
        then
                RC=0
        fi
 
        # If nothing mounted, set rc>0 #
        if [ ! -d /snms_backup/mnt/var ]
        then
                RC=555
        fi
 
        ##############
        ###Check RC###
        ##############
        if [ $? -ne 0 ]
        then
                _err_msg_="Command [${SNMS_BACKUP} -o] failed!"
                $TOUCH $ABORT_FILE 2>/dev/null
                ${SNMS_BACKUP} -d >>${LOGFILE} 2>&1
                abort_script "$_err_msg_" 215
                exit 24
        fi
 
        return 0
}
 
 
##########
###MAIN###
##########
 
trap "abort_script" 1 2 3 4 5 6 7 8 10 12 13 14 15
 
# Check that the effective id of the user is root
check_id root
 
DEBUG_MODE=0
DEBUG_FLAG_FILE=/tmp/bpstart_debug_mode
if [ -f "${DEBUG_FLAG_FILE}" ]
then
        DEBUG_MODE=1
fi
 
#**************************
# Pre execution operations
#**************************
TEMP_NODE_DIR=/tmp/.bp_sched_dir
TEMP_SCHED_DIR=${TEMP_NODE_DIR}/${BKUP_CLIENT}
#----------------------------------------#
# VCS - Watch out for failedOver cluster #
#----------------------------------------#
${ECHO} "${POLICY_ID}" | ${EGREP} "OSS_i386|OSS_HACS|VXVM_" >/dev/null 2>&1
if [ ${?} -eq 0 ]
then
 
        failed_over_ha_check
        case ${?} in
        0)
                ha_first_node_check_new || exit 0
                ;;
        1)
               ha_first_node_check || exit 0
               ;;
        esac
fi
ABORT_FILE=${TEMP_SCHED_DIR}/aborted
 
if [ ${DEBUG_MODE} -eq 1 ]
then
        $GREP -w 999 ${DEBUG_FLAG_FILE}
        if [ $? -eq 0 ]
        then
                $MKDIR -p ${TEMP_SCHED_DIR}/bpstart_trace_${POLICY_ID}/
                exec 1>>${TEMP_SCHED_DIR}/bpstart_trace_${POLICY_ID}/bpstart_stream_${STREAM_NUMBER}
                exec 2>>${TEMP_SCHED_DIR}/bpstart_trace_${POLICY_ID}/bpstart_stream_${STREAM_NUMBER}
        fi
fi
 
#---------------------------------------
#--> Which type of ERICSSON Policy? <---
#---------------------------------------
OMBS_POLICY_ID=""
#---------------#
# root policies #
#---------------#
echo "${POLICY_ID}" | $GREP _ROOT >/dev/null 2>&1
if [ $? -eq 0 ]
then
        TAG=_ROOT
else
        for TAG in _FILES _SNMS _VERSANT _Sybase OSS_i386_ OSS_HACS_ OSS_DDS_ OSS_HARS_ OSS_ VXVM_ ENIQ_STATS_MULTIBLADE_CORDINATOR_RAW_ ENIQ_STATS_MULTIBLADE_CORDINATOR_DATA_ ENIQ_STATS_MULTIBLADE_DATA_  ENIQ_STATS_ONBLADE_DATA_ ENIQ_STATS_ONBLADE_RAW_ ENIQ_STATS_ ENIQ_EVENTS_ ENIQ_
        do
                #-------------------------------------#
                # Data Policies or older policy names #
                #-------------------------------------#
                echo "${POLICY_ID}" | grep ${TAG} >/dev/null 2>&1
                if [ $? -eq 0 ]
                then
                        case ${TAG} in
                        OSS_HACS_)              TAG=_HACS
                                                ;;
                        OSS_HARS_)              TAG=_HARS
                                                ;;
                        OSS_DDS_)               TAG=_DDS
                                                ;;
                        VXVM_)                  TAG=_VX
                                                ;;
                        OSS_i386_)              TAG=_OSS_i386
                                                ;;
                        OSS_)                   TAG=_OSS
                                                ;;
                        ENIQ_STATS_MULTIBLADE_DATA_)           TAG=_ENIQ_STATS_MULTIBLADE_DATA
                                                ;;
                        ENIQ_STATS_MULTIBLADE_CORDINATOR_DATA_)           TAG=_ENIQ_STATS_ONBLADE_DATA
                                                ;;
                        ENIQ_STATS_ONBLADE_DATA_)           TAG=_ENIQ_STATS_ONBLADE_DATA
                                                ;;
                        ENIQ_STATS_ONBLADE_RAW_)           TAG=_ENIQ_STATS_ONBLADE_RAW
                                                ;;
                        ENIQ_EVENTS_)           TAG=_ENIQ_EVENTS
                                                ;;
                        ENIQ_|ENIQ_STATS_)      TAG=_ENIQ_STATS
                                                ;;
                        _VERSANT|_Sybase)       break
                                                ;;
                        *)                      TAG=_FILES
                                                ;;
                        esac
                        break
                fi
        done
fi
if [ -z "${TAG}" ]
then
        TAG=_FILES
fi
OMBS_POLICY_ID="${TAG}"
 
#------------------
#------------------
# OMBS Logging  ###
#------------------
#------------------
OMBS_LOGS_HOME=${TPP_HOME}/logs/ombs
if [ ! -d "${OMBS_LOGS_HOME}" ]
then
        $MKDIR -p ${OMBS_LOGS_HOME} 2>/dev/null
fi
 
###SAVE BMR AGENT INFO###
SAVE_CONFIG=${OMBS_COMMON}/bin/save_conf_data.bsh
if [ -s /eniq/bkup_sw/bin/save_conf_data.bsh ]
then
   SAVE_CONFIG=/eniq/bkup_sw/bin/save_conf_data.bsh
fi
 
#########################
# Set General variables #
#########################
case ${OMBS_POLICY_ID} in
 
_HARS|_HACS|_DDS|_OSS*|_VX)     pkginfo ERICombsc >/dev/null 2>&1
                                RC=${?}
                                if [ $RC -ne 0 ]
                                then
                                        pkginfo ERICnbcli >/dev/null 2>&1
                                        RC=${?}
                                fi
                                if [ $RC -ne 0 ]
                                then
                                        _err_msg_="The package [ERICombsc] is not installed!"
                                        abort_script "$_err_msg_" 620
                                fi
 
                                LOGDIR=${OMBS_LOGS_HOME}/data
 
                                ###remove any old stuff###
                                for TIDYUP in ${OMBSC_HOME}/backup/ ${OMBS_LOGS_HOME}
                                do
                                        rm -fr ${TIDYUP}/run_logs >/dev/null 2>&1
                                        rm -fr ${TIDYUP}/oss >/dev/null 2>&1
                                done
 
                                ###Scripts for OSS_<hostname> and OSS_HARS_<hostname> Policies###
                                OMBSC_TOOL=${OMBSC_HOME}/bin/ombsc
                                ;;
 
_VERSANT)                       LOGDIR="${OMBS_LOGS_HOME}/versant"
                                ;;
 
_Sybase)                        LOGDIR="${OMBS_LOGS_HOME}/sybase"
                                ${MV} ${OMBSC_HOME}/logs/sybase/* ${LOGDIR} >/dev/null 2>&1
                                ${RM} -fr ${OMBSC_HOME}/logs/sybase/ >/dev/null 2>&1
                                ;;
 
_SNMS)                          LOGDIR="${OMBS_LOGS_HOME}/snms"
                                SNMS_BACKUP=/snms_backup/backup.sh
                                ;;
 
_ENIQ_EVENTS)                   LOGDIR=${OMBS_LOGS_HOME}/data
                                PREP_ENIQ_EVENTS_BACKUP=/eniq/bkup_sw/bin/prepare_events_bkup.bsh
                                CLEANUP_ENIQ_EVENTS_BACKUP=/eniq/bkup_sw/bin/cleanup_events_backup.bsh
 
                                SAVE_CONFIG=/eniq/bkup_sw/bin/save_conf_data.bsh
                                if [ ! -s ${SAVE_CONFIG} ]
                                then
                                        ALT_SAVE_CONFIG=${OMBS_COMMON}/bin/save_conf_data.bsh
                                fi
                                ;;
 
_ENIQ_STATS_MULTIBLADE_DATA)                    LOGDIR=${OMBS_LOGS_HOME}/data
                                PREP_ENIQ_STATS_BACKUP=/eniq/bkup_sw/bin/prepare_eniq_bkup.bsh
                                CLEANUP_ENIQ_STATS_BACKUP=/eniq/bkup_sw/bin/cleanup_eniq_backup.bsh
 
                                if [ ! -s ${SAVE_CONFIG} ]
                                then
                                        ALT_SAVE_CONFIG=/eniq/admin/backup/bin/save_conf_data.bsh
                                fi
                                ;;
 
_ENIQ_STATS_ONBLADE_DATA)                    LOGDIR=${OMBS_LOGS_HOME}/data
                                PREP_ENIQ_STATS_BACKUP=/eniq/bkup_sw/bin/prepare_eniq_bkup.bsh
                                CLEANUP_ENIQ_STATS_BACKUP=/eniq/bkup_sw/bin/cleanup_eniq_backup.bsh
 
                                if [ ! -s ${SAVE_CONFIG} ]
                                then
                                        ALT_SAVE_CONFIG=/eniq/admin/backup/bin/save_conf_data.bsh
                                fi
                                ;;
_ENIQ_STATS_ONBLADE_RAW)        LOGDIR=${OMBS_LOGS_HOME}/data
                                PREP_ENIQ_STATS_BACKUP="/ericsson/ombss/eniq_stats_blade/bin/activate_snapshots.bsh"
                                CLEANUP_ENIQ_STATS_BACKUP=/ericsson/ombss/eniq_stats_blade/bin/activate_snapshots.bsh
 
                                SAVE_CONFIG=""
                                ALT_SAVE_CONFIG=""
                                ;;
 
_ENIQ_STATS)                    LOGDIR=${OMBS_LOGS_HOME}/data
                                PREP_ENIQ_STATS_BACKUP=/eniq/admin/backup/bin/prepare_eniq_backup.bsh
                                CLEANUP_ENIQ_STATS_BACKUP=/eniq/admin/backup/bin/cleanup_eniq_backup.bsh
 
                                if [ ! -s ${SAVE_CONFIG} ]
                                then
                                        ALT_SAVE_CONFIG=/eniq/admin/backup/bin/save_conf_data.bsh
                                fi
                                ;;
 
_ROOT)                          LOGDIR=${OMBS_LOGS_HOME}/root
 
                                ${ECHO} "${OMBS_POLICY_ID}" | grep ENIQ_EVENTS_ROOT >/dev/null 2>&1
                                if [ ${?} -eq  0 ]
                                then
                                        SAVE_CONFIG=/eniq/bkup_sw/bin/save_conf_data.bsh
                                fi
                                ### Is ERICombsc in use? OSS? ###
                                if [ -d "${OMBSC_HOME}" ]
                                then
                                        if [ ! -d "${OMBSC_HOME}/backup" ]
                                        then
                                                DATA_SAVECFG=${OMBSC_HOME}/bin/house_keeping.sh
                                                DATA_GETVXFS=${OMBSC_HOME}/bin/get_sync_filesystem
                                                ${DATA_SAVECFG} "` ${DATA_GETVXFS} ` /"
 
                                        fi
                                fi
                                ;;
 
*)                              LOGDIR=${OMBS_LOGS_HOME}/clilog/
 
                                if [ ! -s ${SAVE_CONFIG} ]
                                then
                                        ALT_SAVE_CONFIG=/ericsson/ombsx86/bin/save_x86_conf_data.bsh
                                fi
                                ;;
esac
 
#****************************#
# LOGFILE / LOGDIR Handling. #
#****************************#
if [ -d "${LOGDIR}" ]
then
        find ${LOGDIR} \( -name 'bpstart_notify*' \) -atime +3 -exec rm {} \;
else
        $MKDIR -p ${LOGDIR} 2>/dev/null
        ##for events/statistics##
        rm /eniq/admin/backup/log >/dev/null 2>&1
        ln -s ${LOGDIR} /eniq/admin/backup/log >/dev/null 2>&1
fi
 
#**************************#
# Get correct logfile name #
#**************************#
LOGNAME=bpstart_notify_log
BPEND_LOGNAME=bpend_notify_log
CHECK_FOR_PREV_LOGS=false
 
if [ ${STREAM_COUNT} -gt 1 ]
then
        LOGFILE="${LOGDIR}/${LOGNAME}.${STREAM_NUMBER}"
        if [ ! -f ${TEMP_SCHED_DIR}/first_stream ]
        then
                CHECK_FOR_PREV_LOGS=true
        fi
else
        LOGFILE="${LOGDIR}/${LOGNAME}"
fi
 
#***********************************************************************#
# If previous backup failed, make sure we don't overwrite previous logs #
#***********************************************************************#
if [ "${CHECK_FOR_PREV_LOGS}" = true ]
then
        if [ -f "${LOGDIR}/${LOGNAME}" ]
        then
                ###File bpstart_notify_log already exists (prev backup failed?). Archive it!###
                $MV ${LOGDIR}/${LOGNAME} ${LOGDIR}/${LOGNAME}.prev_${RUN_TIME}
        fi
        if [ -f "${LOGDIR}/${BPEND_LOGNAME}" ]
        then
                ###File bpend_notify_log already exists (prev backup failed?). Archive it!###
                $MV ${LOGDIR}/${BPEND_LOGNAME} ${LOGDIR}/${BPEND_LOGNAME}.prev_${RUN_TIME}
        fi
fi
 
 
# Determine absolute path to software
get_absolute_path
 
# Check args
chk_args $*
touch_bpstart_called
#*************************************************#
# Print banner to top of the log-file except for  #
# Sybase (does not use bpstart_notify_logs).      #
#*************************************************#
if [ ${OMBS_POLICY_ID} != _Sybase ]
then
        chk_create_logfile
        printlog_job_header ${*}
fi
 
#************************#
# Sanity check NBU setup #
#************************#
if [ ! -s "${BPCONF}" ]
then
        _err_msg_="File ${BPCONF} not found or empty"
        abort_script "$_err_msg_" 210
fi
 
# Get the client name according to the Netbackup server
if [ ! "${BP_CLIENT_NAME}" ]
then
        _err_msg_="Could not read CLIENT_NAME parameter from ${BPCONF}"
        abort_script "$_err_msg_" 211
fi
 
 
#--------------------------------------------------------------------------
# Execute OMBS code before Netbackup takes over depending on the Policy ###
#--------------------------------------------------------------------------
case ${OMBS_POLICY_ID} in
 
_VERSANT)       #### VERSANT ###
 
                #--------------------------------------#
                # Determine a suitable dump directory. #
                #--------------------------------------#
                case ${OS_VERSION} in
                10)     OMBS_DUMP_DIR=/ossrc/dbdumps/ombs
                        ;;
                *)      OMBS_DUMP_DIR=/export/dbdumps/ombs
                        ;;
                esac
 
                if [ ! -d "${OMBS_DUMP_DIR}" ]
                then
                        mkdir -p ${OMBS_DUMP_DIR} 2>/dev/null
                fi
 
                VDUMP_DIR="${OMBS_DUMP_DIR}/VERSANT"
                if [ ! -d "${VDUMP_DIR}" ]
                then
                        mkdir -p ${VDUMP_DIR} >>${LOGFILE} 2>&1
                fi
 
                chmod 777 ${VDUMP_DIR} >>${LOGFILE} 2>&1
                DUMP=`df -h /ossrc/dbdumps | awk 'NR==2 {print $6}'`
 
                if [ "$VDUMP_DIR" !=  "/export/dbdumps/ombs/VERSANT" ]
                 then
                         if [ "$DUMP" != /ossrc/dbdumps ]
                         then
                                        rm -rf ${OMBS_DUMP_DIR} >>${LOGFILE} 2>&1
                                        rm -rf ${VDUMP_DIR} >>${LOGFILE} 2>&1
 
                                         if [ ! -d  /export/dbdumps/ombs/VERSANT ]
                                         then
                                                        mkdir -p  /export/dbdumps/ombs/VERSANT >>${LOGFILE} 2>&1
 
                                        fi
                                ln -s /export/dbdumps/ombs /ossrc/dbdumps
                                chmod 777 /export/dbdumps/ombs/VERSANT >>${LOGFILE} 2>&1
                                VDUMP_DIR=/export/dbdumps/ombs/VERSANT
                        fi
                fi
 
                chown nmsadm:nms ${VDUMP_DIR} >>${LOGFILE} 2>&1
 
                #-----------------------------------------------------#
                ### only want databases on this host see TR HF67753 ###
                ### two cuts are necessary to get rid of the blanks ###
                #-----------------------------------------------------#
                DBLIST=/opt/versant/ODBMS/bin/dblist
                VBACK=/opt/versant/ODBMS/bin/vbackup
                DB_NAMES="` ${DBLIST} | grep name | cut -d'=' -f2 | cut -d' ' -f2 `"
 
                ######################
                #->Try masterservice #
                ######################
                DB_NAMES="` ${DBLIST} | grep name | cut -d'=' -f2 | cut -d' ' -f2 | nawk -v thishost=masterservice '{ dbname=$0;t=split(dbname,list,"@"); if(list[t]==thishost ){ print list[t-1];} }' `"
 
                ######################
                #->else try hostname #
                ######################
                if [ -z "${DB_NAMES}" ]
                then
                        DB_NAMES="` ${DBLIST} | grep name | cut -d'=' -f2 | cut -d' ' -f2 | nawk -v thishost=$THIS_HOST '{ dbname=$0;t=split (dbname,list,"@"); if(list[t]==thishost ){ print list[t-1];} }' `"
                fi
 
                ######################
                # Dump each Database #
                ######################
                for dbname in ${DB_NAMES}
                do
                        DUMP_VERSANT_DB="${VBACK} -dev $VDUMP_DIR/level0_$dbname -level 0 -backup $dbname"
                        printf "\n\nBacking up database [$dbname]\n"    >>${LOGFILE} 2>&1
                        printf "\n\n[${DUMP_VERSANT_DB}]\n\n"           >>${LOGFILE} 2>&1
 
                        su - nmsadm -c "${DUMP_VERSANT_DB}"             >>${LOGFILE} 2>&1
                        if [ $? -ne 0 ]
                        then
                                printf "\n${RUN_TIME} Command [${DUMP_VERSANT_DB}] failed!\n" >>${LOGFILE} 2>&1
                                exit 22
                        fi
                        echo "[done]"
                done
                ;;
 
_Sybase)        #-------------------------------------------------------------------------
                # Sybase: IF Policy is <hostname>_Sybase_<DB> or <hostname>_Sybase_FILES
                #------------------------------------------------------------------------
                ;;
 
_ENIQ_STATS_MULTIBLADE_DATA|_ENIQ_STATS_ONBLADE_DATA|_ENIQ_STATS_ONBLADE_RAW|_ENIQ_STATS)       ###ENIQ-Statistics DATA###
                if [ ! -s "${PREP_ENIQ_STATS_BACKUP}" ]
                then
                        _err_msg_="File ${PREP_ENIQ_STATS_BACKUP} not found or empty"
                        abort_script "$_err_msg_" 212
                fi
                streamed_data_policy
                ;;
 
_ENIQ_EVENTS)   ###ENIQ-Events DATA###
                if [ ! -s "${PREP_ENIQ_EVENTS_BACKUP}" ]
                then
                        _err_msg_="File ${PREP_ENIQ_EVENTS_BACKUP} not found or empty"
                        abort_script "$_err_msg_" 213
                fi
                streamed_data_policy
                ;;
 
_SNMS)          ###### SNMS ######
                if [ ! -s "${SNMS_BACKUP}" ]
                then
                        _err_msg_="File ${SNMS_BACKUP} not found or empty"
                        abort_script "$_err_msg_" 214
                fi
                ### OK ###
                streamed_data_policy
                ;;
 
_OSS*|_HARS|_HACS|_DDS|_VX)
                if [ ! -s "${OMBSC_TOOL}" ]
                then
                        _err_msg_="File [${OMBSC_TOOL}]  not found or empty"
                        abort_script "$_err_msg_"  211
                fi
                ### OK ###
                streamed_data_policy
                ;;
_FILES)
                streamed_data_policy
                ;;
 
*)              ###Some other Policy###
                solaris_client_checks
                ;;
esac
 
exit 0
#