Video Screencast Help

Backup Exec appears to have spontaneously executed a restore... TWICE!!!

Created: 03 Nov 2012 • Updated: 11 Feb 2013 | 12 comments
This issue has been solved. See solution.

Hiyas,

I am in desparate need of some assistance please.  Twice in the past two weeks, Backup Exec appears to have spontaneously decided to execute a restore instead of a backup and has complete restored the computer to an earlier point in time.  When I say completely restored, when looking at any system log, the backup log, and even data in our database, all items are showing lapses from the time of the tape to the current day.

We are a small operation with a single server (Windows SBS 2003) and 5 client devices (Windows XP).  I use Backup Exec 2010 do to a GFS tape rotation on which the full backup occurs on Mondays.  The job is configured to happen at 8pm nightly and has a timeout set for 1pm the following day.  I also have System Recovery 2010 configured to take nightly snapshots of the server and store them on a NAS device.  The SR snapshots start at 12AM. I am the Sys Admin but the Office Administrator manages the tape rotation because I am not a full time employee.

Since the system is getting restored, I have no logs I can use to try and peice together what has happened.  The user account for the first event is follows:

The tape is provided at the end of the day before the Office Administrator (OA) leaves.  She comes in the next morning and checks on the tape and sees that the tape has ejected but there is also a prompt to supply a tape for a job that appears to be trying to execute.  The tape was supplied and ran for about an hour or two and then the user was provided a message that the system was going to reboot.  After reboot, data was missing.  Log examination showed a complete disappearance of data for a 2 week span.   I restored the computer from the System Recovery images and the computer was fine for about 2 weeks.

This morning I was called and notified that "it happened again".  I came in and found that the log entries in the System application log read 10/19 and then 11/2 - nothing in between.  I get email notifications of the backups and the last backup notification is 10/24 which appears to have completed successfully with exceptions ( unable to obtain minimum quiet time for a drive).  I have been told the shut the server down on 10/29 in preparation for the hurricane and it was down till 10/31.  After restore... the earliest log entries I have are 11/2 at 8:43:19 AM.  Coincidentally, both accounts seemed to happen on a thursday night into friday morning (10/11 and 11/1).

As I do not have any logs to examine or any restore jobs configured, is there any advice you can give me about how this possibly could have happened?  Admittedly, the OA is not a very technical person, so if she was prompted to supply a tape for restore instead of backup, she might not have realized that is was a different request.  However, what really concerns me is why she would have been given such a prompt to begin with? 

Thank you.

Keith

 

Comments 12 CommentsJump to latest comment

Jaydeep S's picture

Keith this is very very strange. I have not seen this happen. Unfortunately, Backup Exec does not have the ability to create new jobs on its own and execute them.

I would suggest you to investigate the the BE console if there are any scheduled restore jobs setup. Also these jobs would have to be restore from disks as if it was from a tape, it would need that perticular tape to be inserted for the job to run.

Also, investigate the possibility of a System Restore being performed from windows.

tafcoi's picture

Jaydeep,

Thanks for the prompt reply.  That is what I thought.  I can confirm that there were no restore jobs scheduled as I have never had to execute a restore from tape.  We only have the 3 GFS rotation backup jobs listed.

Do you have any recommendations on where I could start to investigate the possiblity of a windows system restore happining?  I don't have any windows backups configured but is this something that could be along the lines of a recovery point?  The one thing that stikes me as odd is that the last log times seem to coindide the w/ last used times of the tape.

One thing I should note.  Last night I found that the S.M.A.R.T is reporting an impending failure of one of my hard drives.  I have a RAID 1 (Mirrored) system set up.  Is there anyway this could have triggered something?  Could it have been a faulty tape?

Thanks.

Keith

CraigV's picture

...since when have automated restores been possible in Backup Exec...? AFAIK this has been considered but never implemented, and never been part of BE 2010 R3 and any previous version.

If so, then there is NO way the OP's version of BE can just decide to restore data on its own as it is a manual process.

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

CraigV's picture

Hi Keith,

Could be someone else restoring your data if they have admin rights to the media server and then deleting the data out of Windows! If you turn on auditing for that media server it will show user logins for instance.
Since you can't schedule restores, this alone should take BE out of the equation!
Check or any tasks that have been scheduled from within windows...delete them if not required!

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

CraigV's picture

Hi Keith,

Could be someone else restoring your data if they have admin rights to the media server and then deleting the data out of Windows! If you turn on auditing for that media server it will show user logins for instance.
Since you can't schedule restores, this alone should take BE out of the equation!
Check or any tasks that have been scheduled from within windows...delete them if not required!

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

tafcoi's picture

Craig,

We thought of that... we are a small business with a single server and one administrator account to which 3-4 people total know the password.  All people who know the password are trusted longstanding employees.  They are also not technology savvy users and probably wouldn't even know how to initiate a restore job.

I myself am the main technological mind in the org and find this truly baffling.  I don't even know where to begin looking...  The computer has not undergone extensive customization.  We use a basic Windows SBS 2003 setup with a few applications that depend on the SQL Server installation (Sharepoint and SBS which are natively installed and dESCO's ESC which is a thrid-party app for the service industry.)  The computer primarily functions as a db and file server and PDC.  Email is served through exchange using the POP3 connector...

I don't really have any scheduled processes/services other than Backup Exec.  Unless there is a way that a restore could be automatically triggered (i.e. some sort of fallback option - which I am hearing there is nothing like that in the program), I am not sure where else to look.  If you guys have any recommendations, they would be much appreciated.

Thanks.

Keith

CraigV's picture

Hi Keith,

 

That is weird, but BE has NO way of automating a restore currently. I haven't seen anything in any documents around this. So the fact that BE is automatically restoring data doesn't seem right. Nor does checking for an automated job when this isn't possible.

Have you checked into repairing BE through Add/Remove Programs, and then doing a repair of the BEDB using the TN below?

http://www.symantec.com/business/support/index?page=content&id=TECH67239

Also make sure that your version of BE is fully patched, and that if you have any remote servers, that these patches have been push-installed to them!

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

SOLUTION
tafcoi's picture

Craig,

Thanks.  I haven't thought of that but I will look into that.  

Thank you.

Keith

Nabil polska's picture

i Advise you to run backup exec  install repair,

reboot ur server and run ur AV scan

this happend only with scheduled backups or the manually one as well ?

CraigV's picture

read carefully guys...this has nothing to do with scheduled backups at all!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

Colin Weaver's picture

I am wondering is something outside of Backup Exec is creating snapshot (shadow) copies of your disks and then deciding (for whatever reason) to revert the snapshot. 

As advised (I think by Craig V) earlier BE does not have any direct ability to automate restores they have to be manual.

Although BE in conjunction with the Symantec HA product range can automate a restore as a last resort if all other HA recovery methods have failed. I doubt you are running this kind of setup as it is typically a large enterprise solution.