Video Screencast Help

NDMP Backup terminated: backup configuration failure (2) on RPC Derror: ERROR: BAD ENV OP

Created: 29 Oct 2013 | 14 comments
rk1074's picture

hello all,

We are witnessing the backup failure for few volumes of our FAS 2040  filer. Error code says:

"Backup terminated: backup configuration failure (2) on RPC Derror: ERROR: BAD ENV OP ( for <volume name>)

 

Master Server is NBU 7.5.0.5

HARDWARE LINUX_RH_X86
 

 

We have checked for the existence of the volumes on the Filer and they exist. Other NDMP backups are going good

 

Any Advise?

 
 
Operating Systems:

Comments 14 CommentsJump to latest comment

Marianne's picture

Please show us your policy:

bppllist <policy-name> -U

 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

rk1074's picture

Hello Marianne,

 

please find the required detail:

 bppllist NDMP_abcdSDCSAN03_Daily_Monthly_Yearly -U
------------------------------------------------------------

Policy Name:       NDMP_abcdSDCSAN03_Daily_Monthly_Yearly

  Policy Type:         NDMP
  Active:              yes
  Effective date:      07/17/2013 22:42:35
  Mult. Data Streams:  yes
  Client Encrypt:      no
  Checkpoint:          no
  Policy Priority:     40000
  Max Jobs/Policy:     Unlimited
  Disaster Recovery:   0
  Collect BMR info:    no
  Residence:           abcdnbk01-hcart2-robot-tld-0-abcdsdcsan03.root.com
  Volume Pool:         DataStore
  Server Group:        *ANY*
  Keyword:             (none specified)
  Data Classification:       -
  Residence is Storage Lifecycle Policy:    no
  Application Discovery:      no
  Discovery Lifetime:      28800 seconds
ASC Application and attributes: (none defined)

  Granular Restore Info:  no
  Ignore Client Direct:  no
Enable Metadata Indexing:  no
Index server name:  NULL
  Use Accelerator:  no
  HW/OS/Client:  NDMP          NDMP          abcdsdcsan03.root.com

  Include: 
            /vol/CTX_PVS01_s02/.snapshot/daily.0
            /vol/CTX_PVS02_s02/.snapshot/daily.0
     /vol/esx_ds_app2_t020_vol_1/.snapshot/daily.0
     /vol/cifs_AD_Backups_t140_p_s02/.snapshot/daily.0
          

  Schedule:              Weekly_Kept_for_1_Year
    Type:                Full Backup
    Maximum MPX:         1
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     8 (1 year)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Calendar sched: Enabled
      Allowed to retry after run day
      Saturday, Week 1
      Saturday, Week 2
      Saturday, Week 3
      Saturday, Week 4
      Saturday, Week 5
      EXCLUDE DATE 0 - 01/04/2014
      EXCLUDE DATE 1 - 01/03/2015
      EXCLUDE DATE 2 - 01/02/2016
      EXCLUDE DATE 3 - 01/07/2017
      EXCLUDE DATE 4 - 01/06/2018
      EXCLUDE DATE 5 - 01/05/2019
      EXCLUDE DATE 6 - 01/04/2020
      EXCLUDE DATE 7 - 01/02/2021
      EXCLUDE DATE 8 - 01/01/2022
      EXCLUDE DATE 9 - 01/07/2023
      EXCLUDE DATE 10 - 01/06/2024
      EXCLUDE DATE 11 - 01/04/2025
      EXCLUDE DATE 12 - 01/03/2026
      EXCLUDE DATE 13 - 01/02/2027
      EXCLUDE DATE 14 - 01/01/2028
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:
          Saturday   01:00:00  -->  Saturday   23:00:00

  Schedule:              Annual_Kept_for_7_Years
    Type:                Full Backup
    Maximum MPX:         1
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     6 (7 years)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Calendar sched: Enabled
      Allowed to retry after run day
      SPECIFIC DATE 0 - 01/04/2014
      SPECIFIC DATE 1 - 01/03/2015
      SPECIFIC DATE 2 - 01/02/2016
      SPECIFIC DATE 3 - 01/07/2017
      SPECIFIC DATE 4 - 01/06/2018
      SPECIFIC DATE 5 - 01/05/2019
      SPECIFIC DATE 6 - 01/04/2020
      SPECIFIC DATE 7 - 01/02/2021
      SPECIFIC DATE 8 - 01/01/2022
      SPECIFIC DATE 9 - 01/07/2023
      SPECIFIC DATE 10 - 01/06/2024
      SPECIFIC DATE 11 - 01/04/2025
      SPECIFIC DATE 12 - 01/03/2026
      SPECIFIC DATE 13 - 01/02/2027
      SPECIFIC DATE 14 - 01/01/2028
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:
          Saturday   01:00:00  -->  Saturday   23:00:00

  Schedule:              Daily_Kept_for_1_Month
    Type:                Cumulative Incremental Backup
    Maximum MPX:         1
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     3 (1 month)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Calendar sched: Enabled
      Sunday, Week 1
      Monday, Week 1
      Tuesday, Week 1
      Wednesday, Week 1
      Thursday, Week 1
      Friday, Week 1
      Sunday, Week 2
      Monday, Week 2
      Tuesday, Week 2
      Wednesday, Week 2
      Thursday, Week 2
      Friday, Week 2
      Sunday, Week 3
      Monday, Week 3
      Tuesday, Week 3
      Wednesday, Week 3
      Thursday, Week 3
      Friday, Week 3
      Sunday, Week 4
      Monday, Week 4
      Tuesday, Week 4
      Wednesday, Week 4
      Thursday, Week 4
      Friday, Week 4
      Sunday, Week 5
      Monday, Week 5
      Tuesday, Week 5
      Wednesday, Week 5
      Thursday, Week 5
      Friday, Week 5
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:
          Sunday     01:00:00  -->  Sunday     23:00:00
          Monday     01:00:00  -->  Monday     23:00:00
          Tuesday    01:00:00  -->  Tuesday    23:00:00
          Wednesday  01:00:00  -->  Wednesday  23:00:00
          Thursday   01:00:00  -->  Thursday   23:00:00
          Friday     01:00:00  -->  Friday     23:00:00

[root@abcdsdcnbk01 var]#

Marianne's picture

Have a look at this post:

https://www-secure.symantec.com/connect/forums/snapshot-backups-1

 

See the extract from the manual about valid Backup Selections.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

rk1074's picture

Hello Marianne,

I believe  backup selection is fine as the backup for same volumes were  running good just few days backup.

Yasuhisa Ishikawa's picture

Please try backup of primary data(path without .snapshot/*) with same NDMP host and same STU to confirm if there are not connectivity issue in this configuration.

 Also tell us Data ONTAP version and how long this backup job ran.

Authorized Symantec Consultant(ASC) Data Protection in Tokyo, Japan

rk1074's picture

Hello Ishi,

 

Our backup selection resided on the Snapvault filer and we have only one NDMP license. so we cannot check the primary data backup from the production filer.

 

ONTAP version is   8.1.2 7-Mode

 

the job runs for around 30 secs and then fails

watsons's picture

The storage unit "abcdnbk01-hcart2-robot-tld-0-abcdsdcsan03.root.com" name looks like a tape storage on the NDMP filer itself, as it has the hostname of the NDMP host you backup, can you confirm?

Can you switch it to either a disk storage unit or another tape storage unit (outside of NDMP host) to see if it works? That could tell us if the tape drive on NDMP host is working. "storage show tape" in NDMP host would also tell you if there is any tape drive issue on it.

Yasuhisa Ishikawa's picture

In NetApp site and Google, I could not find anything related to this message other than irregal file selections and network issues.
Primary data I called is not that of SnapVault. Can you try to backup each volumes on abcdsdcsan03.root.com in stead of snapshot paths?

Authorized Symantec Consultant(ASC) Data Protection in Tokyo, Japan

rk1074's picture

@watsons

Yes the tape drive is directly attached to the filer as its a local NDMP backup . And I can confirm that drives are good  as the backup fo other volumes on the same filer is going good. Its just few volumes for which we are facing this issue.

 

@Ishi

 

we are trying with the same. Will let u know the result.

 

 

rk1074's picture

Volume level backup is running fine for the same selection.

its still failing for snapshot level though.

wr's picture

>>I believe  backup selection is fine as the backup for same volumes were  running good just few days backup

 

are you sure it used to work? 

per the manual

The following backup selections capabilities are NOT supported for an NDMP policy:

Individual file names. Only directory or volume names are allowed.

Will Restore -- where there is a Will there is a way

rk1074's picture

Hi WR,

 

yes it was working fine earlier and we are not taking individual file backups but  the snapshot level. my backup selection is :

/vol/<volumename>/.snapshot/daily.0/

And this is working fine for all other volumes .sharing screen shot for one such successfull backup

 

 

Capture.PNG
wr's picture

OK, you say the volume level is fine.  Is the snapshot there when you browse the volume level backup?

Will Restore -- where there is a Will there is a way

smurphy's picture

The backup paths look legal.  You cannot back up the .snapshot area but you can do the individual snapshots underneath.  Perhaps try them one at a time to narrow down the problem volume?

It is highly likely the error is coming from the filer.  This article has logging instructions:

http://www.symantec.com/docs/TECH56492

The key here would be the ndmpd log and possibly the /etc/messages system log.
If you require assistance parsing through these logs, please open a case with Support.
Have all the logs and the nbsu output from the media server ready.

Steve Murphy
NetBackup Technical Support
_________________________________
http://go.symantec.com/nb