Video Screencast Help

Exchange backups are failing with error code 26

Created: 25 May 2012 | 11 comments

Hello,

The exchange backups (DAG) are failing with error code 26

Exchange 2010 sp1

netbackup master : 7.0.1 , Media server and both the exchange nodes are : 7.0.1

 

Job log info:

 

25-May-2012 10:20:24 - requesting resource UK_WDR_WDR-PD
25-May-2012 10:20:24 - requesting resource xpback.NBU_CLIENT.MAXJOBS.dag02
25-May-2012 10:20:24 - requesting resource xpback.NBU_POLICY.MAXJOBS.OneIS_DB_PD_WDR_EXCH_DAG
25-May-2012 10:20:24 - requesting resource EXCHANGE_RESOLVER.xpback.OneIS_DB_PD_WDR_EXCH_DAG.dag02
25-May-2012 10:20:24 - granted resource  xpback.NBU_CLIENT.MAXJOBS.dag02
25-May-2012 10:20:24 - granted resource  xpback.NBU_POLICY.MAXJOBS.OneIS_DB_PD_WDR_EXCH_DAG
25-May-2012 10:20:24 - granted resource  EXCHANGE_RESOLVER.xpback.OneIS_DB_PD_WDR_EXCH_DAG.dag02
25-May-2012 10:20:24 - estimated 0 kbytes needed
25-May-2012 10:20:24 - begin Parent Job
25-May-2012 10:20:24 - started process RUNCMD (pid=27615)
25-May-2012 10:20:24 - ended process 0 (pid=27615)
Operation Status: 0
25-May-2012 10:20:24 - end Parent Job; elapsed time 0:00:00
Operation Status: 0
Operation Status: 0
25-May-2012 10:20:26 - Info bpbrm (pid=27625) from client dag02: TRV - BPRESOLVER has executed on server (UK-EX023)
25-May-2012 10:20:24 - started process bpbrm (pid=27625)
25-May-2012 10:20:36 - Error bpbrm (pid=27625) socket read failed: errno = 104 - Connection reset by peer
Operation Status: 26
Operation Status: 0
25-May-2012 10:20:36 - started process RUNCMD (pid=27947)
25-May-2012 10:20:36 - ended process 0 (pid=27947)
Operation Status: 0
Operation Status: 26
client/server handshaking failed  (26)

Comments 11 CommentsJump to latest comment

Yogesh9881's picture

Is it new implementation or was it working before  ??

have you tested file level backup for all DAG member hosts ??

 

If this post has helped you, please vote or mark as solution.

Before break-up, make sure you have a good backup.....  ;-)

rizwan84tx's picture

The client name must be DAG virtual name.

Are you backing up the Active or passive nodes?

Make sure that all Mailbox servers in the DAG is added in the preffered server list.

 

Can you please comment about this host "UK-EX023" ?

Best Regards,

Rizwan

Marianne's picture

100% sure that all DAG nodes are on 7.0.1?

Problem supposed to be solved with 7.0.1 : http://www.symantec.com/docs/TECH125427

 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

captain jack sparrow's picture

agree with marianne... as from logs yo posted..   BAR is showing at 7.0GA..

please verify you have installed 7.0.1 patch on client...

If patch is successfully deployed two additional services would have been added under NBU

NetBackup Legacy client

NetBackup Letgacy Network

 

pls Check

 Cheers !!!

CJS

 

premchandgeetha's picture

Hello,

Yes. Both the DAG nodes are in 7.0.1 . The main thing is that some times the exchange backups for both the nodes(uk-ex023 and uk-ex022) are getting successful and sometimes they are not  ( faiing with EC 26) and the OS backups also same. Sometimes both the nodes' OS backups are getting succesful, sometimes only uk-ex023's OS backups are getting failed. ( ensured that both the services " Netbackup Legacy client service / Netbackup Legacy Network service are available)

and as per your tech note http://www.symantec.com/docs/TECH125427

I am not getting the below error as mentioned in the tech note

For an Exchange 2010 DAG backup attempt, the following messages are logged in the Activity Monitor. (See the detailed status for the backup job.):

Error bpbrm(pid=20332) This type of backup is not supported on this version of Exchange
Error bpbrm(pid=20332) cannot run bpresolver or it failed on client DAG2010001 Status 26

I am getting the below error in the activity monitor :(highlighted below)

05-Sep-2012 19:20:26 - requesting resource UK_WDR_WDR-PD
05-Sep-2012 19:20:26 - requesting resource xpback.NBU_CLIENT.MAXJOBS.dag02
05-Sep-2012 19:20:26 - requesting resource xpback.NBU_POLICY.MAXJOBS.OneIS_DB_PD_WDR_EXCH_DAG
05-Sep-2012 19:20:26 - requesting resource EXCHANGE_RESOLVER.xpback.OneIS_DB_PD_WDR_EXCH_DAG.dag02
05-Sep-2012 19:20:26 - granted resource  xpback.NBU_CLIENT.MAXJOBS.dag02
05-Sep-2012 19:20:26 - granted resource  xpback.NBU_POLICY.MAXJOBS.OneIS_DB_PD_WDR_EXCH_DAG
05-Sep-2012 19:20:26 - granted resource  EXCHANGE_RESOLVER.xpback.OneIS_DB_PD_WDR_EXCH_DAG.dag02
05-Sep-2012 19:20:26 - estimated 49299642 kbytes needed
05-Sep-2012 19:20:26 - begin Parent Job
05-Sep-2012 19:20:26 - started process RUNCMD (pid=12845)
05-Sep-2012 19:20:26 - ended process 0 (pid=12845)
Operation Status: 0
05-Sep-2012 19:20:26 - end Parent Job; elapsed time 0:00:00
Operation Status: 0
Operation Status: 0
05-Sep-2012 19:20:26 - started process bpbrm (pid=12854)
05-Sep-2012 19:20:46 - Error bpbrm (pid=12854) socket read failed: errno = 104 - Connection reset by peer

Operation Status: 26
Operation Status: 0
05-Sep-2012 19:20:46 - started process RUNCMD (pid=13257)
05-Sep-2012 19:20:46 - ended process 0 (pid=13257)
Operation Status: 0
Operation Status: 26
client/server handshaking failed  (26)

sometimes when i tried to open the BAR on DAG02 ( virtual name of uk-ex023), it says the error

like : " Netbackup has stopped working" ( pls refer the attached screen shot for the same)

also when i find the application logs, i get the Application error as below in DAG02 ( virtual name of uk-ex023)

--------------------------------------------------------------------------------------------------------------------------------------

Faulting application name: nbwin.exe, version: 7.0.2010.707, time stamp: 0x4c35851f

Faulting module name: MSVCP80.dll, version: 8.0.50727.4940, time stamp: 0x4ca2b550

Exception code: 0xc0000005

Fault offset: 0x000000000002730a

Faulting process id: 0x2a04

Faulting application start time: 0x01cd8cdb76bf32bd

Faulting application path: C:\Program Files\Veritas\NetBackup\bin\nbwin.exe

Faulting module path: C:\Windows\WinSxS\amd64_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.4940_none_88df89932faf0bf6\MSVCP80.dll

Report Id: bef71dbb-f8ce-11e1-9beb-78e7d1e1aabc

--------------------------------------------------------------------------------------------------------------------------------------------

Faulting application name: bpresolver.exe, version: 7.0.2010.707, time stamp: 0x4c3584ca

Faulting module name: MSVCP80.dll, version: 8.0.50727.4940, time stamp: 0x4ca2b550

Exception code: 0xc0000005

Fault offset: 0x000000000002730a

Faulting process id: 0x27fc

Faulting application start time: 0x01cd8cdac0201c4d

Faulting application path: C:\Program Files\Veritas\NetBackup\bin\bpresolver.exe

Faulting module path: C:\Windows\WinSxS\amd64_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.4940_none_88df89932faf0bf6\MSVCP80.dll

Report Id: 050639a1-f8ce-11e1-9beb-78e7d1e1aabc

    -----------------------------------------------------------------------------------------------------------------------------------------------

Faulting application name: bpbkar32.exe, version: 7.0.2010.707, time stamp: 0x4c475004

Faulting module name: MSVCP80.dll, version: 8.0.50727.4940, time stamp: 0x4ca2b550

Exception code: 0xc0000005

Fault offset: 0x000000000002730a

Faulting process id: 0x1c8

Faulting application start time: 0x01cd8cdc156a5f0d

Faulting application path: C:\Program Files\Veritas\NetBackup\bin\bpbkar32.exe

Faulting module path: C:\Windows\WinSxS\amd64_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.4940_none_88df89932faf0bf6\MSVCP80.dll

Report Id: 59eee3af-f8cf-11e1-9beb-78e7d1e1aabc

-------------------------------------------------------------------------------------------------------------------------------------------------

 

Let me know if you need any other logs / info to resolve this issue.

 

  

   

AttachmentSize
screen shot - error when opening up BAR in DAG node.docx 23.61 KB
premchandgeetha's picture

Forgot to mention that some times the OS backups for UK-EX023 are getting failed with EC 13. Also Netbackup client version has been reinstalled on DAG02 . still I am facing the same issue.

as below

06-Sep-2012 00:58:00 - requesting resource UK_WDR_WDR-PD
06-Sep-2012 00:58:00 - requesting resource xpback.NBU_CLIENT.MAXJOBS.uk-ex023
06-Sep-2012 00:58:00 - requesting resource xpback.NBU_POLICY.MAXJOBS.OneIS_OS_PD_BDR_EXCH
06-Sep-2012 00:58:02 - granted resource  xpback.NBU_CLIENT.MAXJOBS.uk-ex023
06-Sep-2012 00:58:02 - granted resource  xpback.NBU_POLICY.MAXJOBS.OneIS_OS_PD_BDR_EXCH
06-Sep-2012 00:58:02 - granted resource  MediaID=@aaaa9;DiskVolume=PureDiskVolume;DiskPool=UK-WDR1-PD_Pool;Path=PureDiskVolume;StorageServer=uk-pd-wdr1-01;MediaServer=uk-bk001
06-Sep-2012 00:58:02 - granted resource  UK_WDR_WDR-PD
06-Sep-2012 00:58:02 - estimated 15301700 kbytes needed
06-Sep-2012 00:58:03 - started process bpbrm (pid=1236)
06-Sep-2012 00:58:06 - connecting
06-Sep-2012 00:58:08 - connected; connect time: 0:00:00
06-Sep-2012 01:00:38 - Error bpbrm (pid=1236) socket read failed, An existing connection was forcibly closed by the remote host.  (10054)
06-Sep-2012 00:58:17 - end writing
file read failed  (13)
 

 

captain jack sparrow's picture

Not sure if it helps, But worth give try

https://www-secure.symantec.com/connect/videos/exc...

Refer the transcript from above silent video, though it's for NBU 7.5 and ex2k10 but still applies to NBU 7.x family

Make sure you get it through

i had recently encountered similar of yours, where everything was LOOKING AS IT SHOULD BE (same as mentioned under transcript) but still backups  were failing with status 13. everylogs was still pointing to same reason access issue though had all the required privileges. Issue solved after weekend and turned out it was AD replication which took time to update the changes (privilege updates)

There after we tried changing account to local system account that too did worked.

during entire backup process this simply slipped of us that customer had 8 Domain controllers which must be at same level hence no choice till wait to get it replicated. Check if it's ur case too

 

 Cheers !!!

CJS

 

premchandgeetha's picture

Hello Speedfreak,

 

 Actually this issue started 3 months before. Symantec created a work around policy , where in the policy they mentioned : Database backup source: "Active copy only" and the server UK-EX023 ( problematic) is included in the preferred server list. refer the attached screen shot ( uk-ex022 exchange backups were working fine at that time, 3 dbs are mounted in ex022 and the rest of 3 dbs are mounted in ex023). so the exchange backups was getting successful for ex022 from the original policy and the exchange backups were getting successful for ex023 from the work around policy, which was created  by symantec.

Now some two weeks before there was three security patches were installed on both the nodes, from that day even the work around policy also started to failed on both the nodes.

I changed the account to local system and tried still backups are failing with EC 13 for OS and EC 26 for exchange backups.

Let me know if you need any other info / logs

 

AttachmentSize
screen shot for work around policy.docx 38.13 KB
captain jack sparrow's picture

did you tried isolating it , by removing thos security patches.. were thee patches tested in UAT before prod ?

Try if it helps

 

 Cheers !!!

CJS

 

premchandgeetha's picture

No. We did not try isolating it, since it needs so many procedure ( reverting the security patch changes), also did not test in UAt before prod.  Any EEB available for this issue ?

captain jack sparrow's picture

can u reconfirm with your messaging team on SP level on exchange , is it SP1 or SP2.....

 Cheers !!!

CJS