Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

VM backups pilot job gives error 196

Created: 13 Sep 2012 • Updated: 22 Feb 2013 | 15 comments
This issue has been solved. See solution.

VM backups pilot job gives error 196. The policy is set to run about 8 hours. All jobs complete in about 4 hours. But often the pilot job errors out with 196.

 

Anyone seen this before?

Comments 15 CommentsJump to latest comment

Stuart Green's picture

Do you get the 196 error code when running a manual backup of this policy?

Please include version of netbackup software and platform. And possibly vsphere version etc.

 

Tip: Get overview/document your NBU environment. Run 'nbsu' and review the output.

• If this provides help, please vote or mark appropriate solution.

Fabrice P.'s picture

I get a lot of those unexpected196 errors with my "EnterpriseVault" backups, only with them.

My Schedules are Calendar type with a backup window opening fromt 8pm to 6am. During the week, we are doing incrementals and most of the time it only takes 1-2 hours.

However, from time to time, there is still 196 error coming during the night, even if the backup window is still open. This is VERY weird and looks like a nasty bug, somewhere, in the scheduler. I have the issue since the v7 (we are now in 7.5.0.4) and so far, the Symantec support did not helped much.

I have the feeling that this is related to the parent/childs jobs, when the parent job (performing the snapshots) is starting before midnight but when the chids are in queue and start only after midnight.

Few days ago, I switched the "daily incremental" to a "frequency" type of schedule (rather than calendar) and so far the issue is gone. Unfortunatly i can't really use this type of shedule for the weekly or monthly backups since we needs them to start on very specific dates...

Authorised Symantec Customer ;)

Mark_Solutions's picture

There was a bug at 7.5.0.3 :

 http://www.symantec.com/docs/TECH192104

but does sound more like a schedule issue, would need to see what you window was, whether it was calendar based and if so if it crossed midnight

For example, calendar based with a window 23:50 to 06:00 only actually gives a window for the initial back of 23:50 to 00:00 so the parent job kicks in but by the time the snapshots have been prepared the window has closed

Hope this helps

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Fabrice P.'s picture

For me yes, the EnterpriseVault policy is very simple: calendar based with a backup window from 20:00 to 6:00am. However I'm using 7.5.0.4 (master/media/client) and according to the tech note, this bug seems to be corrected in that version...

Mark, when you say:

"For example, calendar based with a window 23:50 to 06:00 only actually gives a window for the initial back of 23:50 to 00:00 so the parent job kicks in but by the time the snapshots have been prepared the window has closed."

This is an explanation of the 7.5.0.3 bug or is it a "normal" behavior, even in 7.5.0.4 (which means that I need to re-think/change my schedules...) ?

 

Authorised Symantec Customer ;)

Mark_Solutions's picture

If you are using calendar scheduling you cannot cross midnight with you open backup window ...

You say you are using calendar scheduling and your window is 20:00 to 06:00, but 20:00 is on one day and 06:00 is on another day.

So if you have Monday selected that only runs from 20:00 to 00:00 (or 23:59) after that it is Tuesday so your window up to 06:00 does not count.

You need to use either frequency based or start your backups at say 00:10 with a Window on Tuesday and not Monday

This is normal behavoir as the calendar is based on days and the day changes at midnight

So your parnet job kicks in at 20:00 and queues lots of jobs. Any that haven't actually started to actively run before midnight will therefore miss their backup window and fail with a 196 error

Hope that clarifies things

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

SOLUTION
Fabrice P.'s picture

Thanks a lot, yes it is very clear now ! I need to find a workaround now... 

During the week it's easy because I can switch to frequency. But problem is for monthly backups that *MUST* occurs the 1rst Friday night of each months... Using first saturday of each month do not works because the first friday/saturday of each month are not always in the same week (typically when the 1 st day of the month is a saturday...).

EDIT: I ticked the option "retries allowed after runday" and will check if it works. Description of that option seems to fit my requirement ("with this attribute enabled, the schedule attemps to run, even after a specified runday has passed") but I remember that I turned it on with no luck in the past (maybe it is working now in 7.5.0.4)...

Authorised Symantec Customer ;)

Tomer Gurantz's picture

I believe you can set the monthly backup using calendar based and the other with frequency. If the calendar and frequency backup OVERLAP (meaning start at the EXACT same time on the same day) than I believe it will run only the one with larger retention, i.e. this is a way you can ensure that only the monthly runs on that specific day.

(And of course with frequency based even though it may say "1 / week" or "1 / day" or whatever, if you set the window to only be open on Monday at 8pm, then it will only run on that day, so it is still limited in that manner, similar to a calendar based schedule).

Senior Principal Technical Education Consultant with Symantec Corporation

Mark_Solutions's picture

Retries Allowed After Run Day is fine but it can lead to things going out of sysc so do be careful

If a Monthly backup fails one week then it will happily run the week after so you start to get a mix of weekly and monthly backups running at the same time, which some people do not like

It will only actually run when a window is open though - not just the next day, so if the window does not open again for a week it will not run until the next week

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Fabrice P.'s picture

Very interesting, maybe I could use this option like this for my monthly :

- using calendar type, check 1 st Friday of each month

- tick the "retires allowed after runday"

- use open window on the saturday 0:00 to 23:55 (no open window at all on the Friday)

Do you think this way the backup would start on the saturday @ 0:00am following the 1st Friday of the month ?

Typically this would instruct NetBackup to start the backup "at the next open window after the 1st Friday of the month".

Authorised Symantec Customer ;)

Mark_Solutions's picture

It may do, but if it failed at any point it would re-try on the 2nd, 3rd, 4th, 5th Saturday - that is where things can get out of sync

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Fabrice P.'s picture

7.5.0.5:

Problems with '196 error' even though backup window is not closed. Calendar schedules

rerun multiple times within the backup window.
 
Interesting !

Authorised Symantec Customer ;)

BTLOMS's picture

"If you are using calendar scheduling you cannot cross midnight with you open backup window ..."

Fabrice P.'s picture

7.5.0.5 release note, pages 46:

Etrack Incident: 2913131

■ Associated Primary Etrack: 2884119

■ Associated Service Request Etrack(s): 2883669
 
■ Description:
 
If a parent job started before midnight, and a child job needed to run or retry
after midnight, it either failed with a status code 196 or did not run at all.
 
BTLOMS, your issue (and mine) was due to a bug which is (claimed to be) fixed on this release. I don't see any reason/logic why a backup window could not cross midnight, even in calendar type schedules.

Authorised Symantec Customer ;)

Mark_Solutions's picture

I dont believe that ETrack relates to calendar scheduling, it relates to a bug causing frequency based scheduling to have that failure.

Calendar based scheduling does impose a midnight restriction as i have covered earlier as after midnight it is a different day and sio the window will not be open.

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Fabrice P.'s picture

I've installed this v7.5.0.5 update and put back my Enterprise Vault policy in "calendar-crossing-midnight" shedule type. I will let you know if I still have 196 issues.

Authorised Symantec Customer ;)