Video Screencast Help

BE2010 Remote Windows Agent Crashes on Server 2008R2 - Faulting Module bedsmbox.dll

Created: 07 Feb 2010 • Updated: 21 May 2010 | 36 comments
This issue has been solved. See solution.

Clean install of BE2010 with Hotfix. Pushed remote agents to server 2008R2 box (DC, Exchange 2010). When accessing the system, Beremote.exe crashes with faulting module bedsmbox.dll. I have tried to do a complete uninstall, reboot, install and it is inconsistant. The backup job is data only job, as Exchange is backed up in a seperate job. The Exchange job does not seem to crash beremote. Only data backup.

No issues on other 2008SP2 and 2003R2 servers.

Any help would be appreciated. I have all the dump files if needed and have attached one of the several.

 

Comments 36 CommentsJump to latest comment

jnelson2000's picture

I should also note that after taking out the system state, shadow copy and system partition (Z), it seems not to crash.

jnelson2000's picture

Faulting application name: beremote.exe, version: 13.0.2896.0, time stamp: 0x4b184ba1
Faulting module name: bedsmbox.dll, version: 13.0.2896.0, time stamp: 0x4b184b99
Exception code: 0xc0000005
Fault offset: 0x0000000000040bf0
Faulting process id: 0x%9
Faulting application start time: 0x%10
Faulting application path: %11
Faulting module path: %12
Report Id: %13

jnelson2000's picture

It also seems to happen only on the Server 2008R2 box during the system snapshot on the mail volume.

James McKey's picture

 This may be an issue we have recently identified. I or another Symantec Employee will provide you an update on this thread.

James McKey
Social Media Support Manager
Support Services

Matthijs in 't Anker's picture

I reported the same issue in this forum yesterday. Strangefully my message has disappeared from the forum.

What is the solution for this problem?

Matthijs in 't Anker's picture

I found the original message:

Hi there,

We installed Backup Exec 2010. When the backup starts, the Remote Agent stops (beremote.exe). We tried to restart it immediately. That helped a bit (not very practical offcourse :). But the next task in the job made the beremote.exe crash again.
We use Windows 2008R2 + Exchange 2010. In de applicatielog we got:

Fault bucket , type 0

Event Name: APPCRASH

Response: Not available

Cab Id: 0

Problem signature:

P1: beremote.exe

P2: 13.0.2896.0

P3: 4b184ba1

P4: bedsmbox.dll

P5: 13.0.2896.0

P6: 4b184b99

P7: c0000005

P8: 0000000000040bf0

P9:

P10:

Attached files:

These files may be available here:

C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_beremote.exe_ffadddcabfeb4469114686cbd2efa05fc7b87_082a2b23

Analysis symbol:

Rechecking for solution: 0

Report Id: 10cb880b-156d-11df-ad61-002481d1e400

Report Status: 4
----------------------------------------

When we debugged the .dmp file, we got:

This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(1e30.78c): Access violation - code c0000005 (first/second chance not available)
ntdll!NtWaitForSingleObject+0xa:
00000000`771ffefa c3              ret

------------------------------------------

Any ideas or fixes?

With kind regards,

Matthijs in 't Anker
2Eenheid

thegoolsby's picture

There is now a public KB for this issue as well:

Beremote.exe crashes when backing up file data from a 2008 R2 server with Exchange 2010 installed
support.veritas.com/docs/337862

Please make sure to try the workaround in the KB article before calling into support.

SOLUTION
Server-tech's picture

I am a bit puzzled on how a issue can be marked as having a solution when 1 week and 3 days later a patch is not available.  What is the ETA of this patch that is for a Product that is supposed to work with Windows Server 2008 R2 and Exchange Server 2010.  Unfortunately I did not see this thread before I upgraded from Backup Exec 12.5 (which I knew did not cover Exchange Server 2010) yesterday and spent the past 25 hours on and off working on it.

Again:

 What is the ETA of this patch that is for a Product that is supposed to work with Windows Server 2008 R2 and Exchange Server 2010?

r2k's picture

Hi!

we have two customers which are waiting for this patch.

Please provide us an ETA or a temoprary solution.

Best regards

Server-tech's picture

Collin Goolsby:
   Is there any update on when a hotfix will be available?

Our mutual customer needs this now.

Matthijs in 't Anker's picture

Great news.
Well, the weirdest thing is that it appears to happen on a server which has Exchange 2010 installed. The backup of Exchange 2010 itself goes perfectly, though.

Well, we'll wait for working and selling Backup Exec 2010 until this works. 90% of our customers have Exchange running.

 

ArnetNZ's picture

I'm not sure whether this is exactly the same issue as you were/are all having,  same symptoms, job commences then crashes straight away, same thing happens when trying to validate the credentials whilst configuring the backup job.

First of all I got a .NET Runtime Event ID:0, followed by Backup Exec Event ID:34113 regarding a communication failure. I also discovered regular SceCli warnings event ID:1202, I discovered  Microsoft KB977695 and resolved the SceCli warnings, which in turn resolved the BeRemote agent crash when trying to backup my server.

Server-tech's picture

Thanks for trying to help.  I just took a quick look and my server is not experiencing the SceCli warnings at least on the backup/exchange server.  I do see the error on the Domain Controller for which the BE Remote Agent is not crashing. 

Dandav's picture

I have created a seperate Exchange 2010 job and that is working.  The job to bakup data is failing every time.  Is there another workaround.  This client is not getting backups of their data.

Ben L.'s picture

Per the technote Collin posted, there is a beta hotfix available for this issue but you will need to call into support and open a case for the issue to get the hotfix.  Right now there is not an ETA on when the hotfix will be released to the general public as it is still in testing.

If this response answers your concern, please mark it as a "solution"

Server-tech's picture

How long does it take to get the hot fix and more information about this hot fix?  I opened a case last Sunday.  I had been in contact with Symantec Technical support on Tuesday and Thursday.  I still do not have the patch.  I had been using Backup Exec System Recovery 2010 to at least cover the Server OS.  The trial version ha expired.  I originally had Backup Exec 12.5 cover the OS and Backup Exec System Recovery 2010 took care of Exchange 2010.  This delay has cost me a major window when the server can go down.

jnelson2000's picture

Server-tech, PM me your email and I will send you the hotfix. It is one file that must be applied on the media server and remote server 2008 R2 Exchange server. Tested for almost a week now and so far so good.

dirisit's picture

can you also send me the beta fix, I too have been waiting almost a week for support to get back to me with this fix, symantec support is horrible!!
thanks!

Ben L.'s picture

dirisit,
Did the workaround of seperating the exchange and OS backup not work for you?  Is the service still crashing after seperating the backups?

If this response answers your concern, please mark it as a "solution"

dirisit's picture

No, I separated the jobs and it did not work, some work around, huh??

I have 2 tape drives and run 2 backup jobs at night simultaneously and backed up the exchange db on 1 job, which ran fine, always did,
and backed up the file systems (C:/E:/system state/shadow copy) on the other job, and that job crashed the beremote.exe just like it did
when it was all in one job.

How long does it take to get this beta fix?? does anybody have it to give to me? I can not wait 3-4 weeks for symantec to finally get around to
me to get me this fix. Backups are not a luxury, they are a NECESSITY, tell symantec that.!

Ben L.'s picture

If you're having a hard time with the technician you are currently working with, then my best suggestion would be to call in and ask to speak to the Duty Manager. They can normal help with getting your case moved along.

If this response answers your concern, please mark it as a "solution"

Engineer System's picture

Before the fix, I coudl see the remote disks and the Exhange Information Store.
Pre-patch, i could backup ONLY the Information Store and the server crash if i tried to back up any disk drive.
Post-patch I can now backup any drive with no problem, BUT the Exchange Information Store is no longer visable in the selcetion list.
Yes, great, i can now backup disk drives, but errr. How about my Inormation Store.
For me its was better post-patch as at minimum i had my store backed up.
Anyone else loaded these dll and has it worked or not worked for Information Store

Kingbee's picture

You have to show your extensions of all file types. Symantec has given you the new DLL's with a .txt extension.

Engineer System's picture

I have replcaed the txt for dll.
And is has made chnages to my system as i  can now backup \c:  \d:  \g:  etc.  
But with the new .dll i have lost the visability of my InformationStore.
It is as if the .dll hides the fact that the server is a Exchange in order to allow disk backup.

MicrocompTech's picture

Hi Guys,

I have the same issue and I open a ticket with Symantec but the guy gave me some instructions related to HP Insight - after I test all steps the agent crush every time. Also I create only one4 job for exchange 2010 and it worked ok for 75GB (HP test with 128GB worked well.)
I reply to your tech but no answerr yet.
Case no# 411502864
Windows server 2008 R2 with exchange 2010 installed - HW - HP ML350G6 - with HP insight software uninstalled.
Any news regarding the release of this patch?
Can I have also this test patch to test it with my server?

Ben L.'s picture

MicrocompTech,

Ask the technician assigned to your case to escalate it so that you can get the Beta hotfix.

If this response answers your concern, please mark it as a "solution"

Server-tech's picture

Ben:
  I am a little disappointed with Symantec's handling of cases.  I finally got the hot fix via email 10 days after I posted earlier trying to get the hotfix.  Why should it take so long.  My client could have been without backups all this time but someone else helped me out.  How long normally does it take to get replies?  My case had a note for someone to contact me 3/1. 

KJW281's picture

My case on this issue "was escalated directly to our Advanced Team" on 2/26/10.  I still have no resolution or hot fix.  As a Symantec reseller I really need to find another vendor.  Symantec as a company is embarrassingly bad.

Normal
0

false
false
false

MicrosoftInternetExplorer4

teiva-boy's picture

 You can ALWAYS ask for a duty manager to raise the issue.  

Alternatively, you partners/resellers, You SHOULD be in the TAPP program so that you get immediately to level 2 support on the first call using your own support ID number and not the customers.  You get better support this way.

There is an online portal, save yourself the long hold times. Create ticket online, then call in with ticket # in hand :-) http://mysupport.symantec.com "We backup data to restore, we don't backup data just to back it up."

CrEOF's picture

Could someone send me this hotfix? I am evaluating BE2010 so calling support isn't really an option.

Thanks!

MicrocompTech's picture

Still waiting after one week...- REALY BAD SUPPORT
Now we speak to someone :
XXXXXXXX   MCP, VCP, CSP
Senior Technical Support Engineer
Backup Exec for Windows Servers
Symantec Corporation

MicrocompTech's picture

Ok Guys,

Finally we got the patch.
Good thing - the backup is working smooth.
The bad thing took long time to get it.
Note from the readme.txt:
1th->""This orphan has not been fully qualified by the
     Symantec Quality Assurance group.  As such, its use should be
     limited to the environments where information is being
     gathered or where the workaround provided by this orphan
     outweighs the risks of using untested code.  """

Guys, really don't know what the Quality Assurance Group is doing but the release version pass that group and it is failing- so they do a really GOOOOOD JOB - so that group should be fired!!!!

2th->"The user of this
     software agrees to assume the risks inherent in the use of an
     orphan build and further agrees not to distribute this code. "
GUYS, when someone is offering his server to test for FREE your software try to be more responsive - not every day and everyone can do that ...so please be faster in your responses.

ANYWAY – THANK YOU and I am waiting for the real patch that it will PASS the QC Group...

Ben L.'s picture

I don't know if this was clear in the technote or not, but if the Exchange backup and the File system backup were seperated into two seperate jobs, this issue does not occur.   

Was anyone seeing this issue after seperating the backup jobs or was there a problem seperating the Exchange and File system backups?

If this response answers your concern, please mark it as a "solution"

Dandav's picture

I did create 2 seperate jobs and the Exchange job ran successfully, the data job still failed.  I managed to get the hotfix and that seems to have corrected the issue, working fine now. This is a single server, Windows Server 2008 R2, domain controller with Exchange 2010 installed.

Ben L.'s picture

Did the job fail or did the service crash during the data job?  Also where you running the jobs at the same time or at different times?

If this response answers your concern, please mark it as a "solution"

Matthijs in 't Anker's picture

Could anyone mail me the fix? I will not try to get it from Symantec as I read here that it will take weeks for it to arrive. Strange btw that the fix is still in Beta. It has been weeks since we reported this problem.

I 2sc Dandav when he says:
I did create 2 seperate jobs and the Exchange job ran successfully, the data job still failed.  I managed to get the hotfix and that seems to have corrected the issue, working fine now. This is a single server, Windows Server 2008 R2, domain controller with Exchange 2010 installed.

-- end quote

We have the same thing. Splitting Exchange and File backup is an old trick which we do since Backup Exec 12. It didn't help for this issue.

So, can I have the fix as a beta or distributed by Liveupdate, please?

With kind regards,

Matthijs in 't Anker