Video Screencast Help

EV 10.0.1; Monitoring; Days Since Last Backup

Created: 27 Mar 2013 • Updated: 05 Aug 2013 | 19 comments
Kai Schröer's picture
This issue has been solved. See solution.

Hello together,

 

the value of "Days Since Last Backup" jumps from 0 days after the backup to 126 days in just a few hours. some days it also jump to 17 or 150 days. Totaly without any direction:

0 Days

0days.jpg

10 hours later: 126 days

126days.jpg

 

We find this value also in the SQL-Database

we try to set the backupmode and restart the storage service but nothing happens.

They have further locations on the NetApp without this issue.

 

Any ideas?

Thanks for suggestions.

Kai

 

Edit: 27th june 2013:

Thread is locked. Great Stuff!

Short abstract of Symantecs findings:

The View $EV-Pending in Domino has elements from 21st of Nov.

The Store for journaling is set to after backup

The engine looks for the creation date.

New eTrack is generated: 3240619 "In EVOM for Domino the 'Days Since Last Backup' for Journal locations might be erroneously calculated"

 

Edit: 5th August 2013:

Technote regarding the issue:

Enterprise Vault for Domino:

In EVOM the Domino Server Monitoring the counter 'Days Since Last Backup' for Journaling locations might be wrongly calculated

http://www.symantec.com/docs/TECH209195

 

Operating Systems:

Comments 19 CommentsJump to latest comment

TonySterling's picture

Check the journal maibox for old items or spam:

Days Since Last Backup' in the Enterprise Vault Operations Manager (EVOM) has a value even though the vault store backup was performed.

Article:TECH55673  |  Created: 2007-01-21  |  Updated: 2012-03-30  |  Article URL http://www.symantec.com/docs/TECH55673

 

Kai Schröer's picture

Hi Tony,

thanks for the answer, but unfortunately this is not the issue:

elements.jpg

 

 

PMCS.helpLine Software Gruppe
www.pmcs.de
Twitter: https://twitter.com/PMCS_EV

TonySterling's picture

So remember that EVOM updates every 15 minutes, is it possible the old message already moved through?

This query should give you the date of the messages archive in the last 24 hours.  Can you run in on the Journal Vault Store database?

select "ItemDate" = left (convert (varchar, IdDateTime,20),14),
"Hourly Rate" = count (*),
"Av Size" = sum (itemsize)/count (*)
from saveset
where archiveddate > dateadd("hh", -24, getdate ())
group by left (convert (varchar, archiveddate,20),14)
order by "ItemDate" desc

Kai Schröer's picture

Hmm good idea, thanks for the query. I have to check this with customer.

I will send the feedback after the test.

Thanks!

 

Kai

PMCS.helpLine Software Gruppe
www.pmcs.de
Twitter: https://twitter.com/PMCS_EV

Kai Schröer's picture

Hi Tony,

we had to modified the script:

select left (convert (varchar, IdDateTime,20),14) as itemdate,
"Hourly Rate" = count (*),
"Av Size" = sum (itemsize)/count (*)
from saveset
where archiveddate > dateadd("hh", -24, getdate ())
group by left (convert (varchar, IdDateTime,20),14)
order by  left (convert (varchar, IdDateTime,20),14) desc

 

The oldest archived element in the last 24 hours is 6 days old.

Today the "Days Since Last Backup" - Value is 56 days

 

Further Ideas? Thanks!

Kai

 

PMCS.helpLine Software Gruppe
www.pmcs.de
Twitter: https://twitter.com/PMCS_EV

TonySterling's picture

I would suggest you look in the failed to copy\failed to store folders.  It could be the items were attempted to be archived and failed but were pending with EVOM looked at the mailbox.

Other than that, this is a very minor issue.  If you are really concerned with it though I would recommend opening a call with SYMC support.

 

TS

Old Man Jensen's picture

I see that you're doing queries in the Saveset table. From my experience, however, EV uses the JournalArchive table to keep track of records that haven't been backed up.

In particular, it's looking at the 'BackupComplete' column, which would be marked as '0' if that record is not registered as being backed up. I'm not precisely sure what EVOM is querying, but I'd imagine it would be going for 'BackupComplete', and then the oldest 'RecordCreationDate' to get its measurement.

Perhaps if you try something like:

SELECT * from JournalArchive
WHERE BackupComplete = '0'
ORDER BY RecordCreationDate

Just to see what the oldest record is that hasn't been marked as backup complete. Sometimes these records don't get cleared, particularly when we're talking EV 8 and earlier.

Jason Jensen

Director – Archiving & eDiscovery | Verizon Terremark

YouTube | LinkedIn

Tremaine's picture

The Days since last backup is determined by running a query against the Databases:

If could please confirm your EV version and also run the following two queries and provide the results when you see this discrepency:

SELECT MAX(backup_Start_date) FROM msdb.dbo.backupset
WHERE (database_name = 'Your_DB_NAME') AND type = 'D'

And

SELECT MAX(backup_Start_date) FROM msdb.dbo.backupset
WHERE (database_name = 'Your_DB_NAME') AND type IN ('D', 'I')

TonySterling's picture

He is talking about in EVOM.  Here is a more up to date TN:

 

Exchange Server Monitoring Summary tab

Article:HOWTO37336  |  Created: 2010-12-24  |  Updated: 2012-06-27  |  Article URL http://www.symantec.com/docs/HOWTO37336

It says:

The number of days since a backup of the vault store was performed.

This value is calculated from the age of the oldest item in a pending archive state. It assumes that the Remove Safety Copies property of the vault store in the Administration Console is set to "After Backup". If Remove Safety Copies is set to another value, you need to interpret the value of Days Since Last Backup accordingly. For example, if Remove Safety Copies is set to "After backup (immediate for journaling)" or "Immediate", the value is typically always 0.

Old Man Jensen's picture

Ghost, I think you're talking about the database backups, whereas this is dealing with the backing up of the vault store partition itself.

I'm still curious as to what the query of the JournalArchive table will turn up. This is where EV tracks what items are backed up, so I'm sure that's where the issue will lie.

Jason Jensen

Director – Archiving & eDiscovery | Verizon Terremark

YouTube | LinkedIn

TonySterling's picture

No, this is not dealing with database backups or the partition!  It is dealing with items in the Exchange Journal mailbox in EV Operations Manager.

You can see from the two technotes I linked to that the way EVOM is determining this is by the oldest pending message in the journal mailbox.  Additionally, in the EV 10 Administrators Guide it repeats:

  1. The number of days since a backup of the vault store was performed.

    This value is calculated from the age of the oldest item in a pending archive state. It assumes that the Remove Safety Copies property of the vault store in the Administration Console is set to "After Backup". If Remove Safety Copies is set to another value, you need to interpret the value of Days Since Last Backup accordingly. For example, if Remove Safety Copies is set to "After backup (immediate for journaling)" or "Immediate", the value is typically always 0.

So I would doubt quering the JournalArchive table would show any different.  If it does the Technotes and Documentation are wrong.

 

Regards,

 

Tremaine's picture

Right, my mistake here. As per Tony, this is based on the items in an ArchivePending state by looking at the last modified date of the item (which should technically be when EV last touched the item) and then getting a diff based on the current time. If you run  a Dtrace on Monitoring and look for the following line: "CExchangeHelper::GetInboxStats - DaysSinceLastBacup" that should reflect what is being seen. It might also be useful to actually interrogate the Journal mbx itself and view the items as they exist at that point in time to see what they indicate. You will need to add the Modified Date/Time Field to the view in order to establish what is set and sort by that.

Kai Schröer's picture

Hi!

Thanks a lot for all the suggestions.

In the first step I will aks the customer about the „failed to copy\failed to store“ - folders.

In the next step I try to trace the monitoring.

Unfortunately the customer is out of office till next week. So it needs some time till the next feedback.

Thanks and Kind Regards

Kai

PMCS.helpLine Software Gruppe
www.pmcs.de
Twitter: https://twitter.com/PMCS_EV

Michael Bilsborough's picture

Hi,

The interesting thing through is the report is showing Failed Operations as 0 in both cases. So if there were items in failed to copy/store then surely failed operations would be >0....

So my guess is there are items flipping between pending and being cancelled and then being attempted archiving again in the Inbox.

Mike

Old Man Jensen's picture

Mr. Bilsborough brings up an interesting point there.

It has me wondering what the 'Pending Shortcut Timeout' property is set to in the archiving policy? Perhaps that needs to be increased, or disabled, at least until the culprit is found.

Jason Jensen

Director – Archiving & eDiscovery | Verizon Terremark

YouTube | LinkedIn

TonySterling's picture

If it is really concerning then the customer should open a case.  However, thing to remember is that EVOM isn't for real-time monitonring, it is always going to be up to 15 minutes behind.  This seems to me like a possible bug that should be logged but it really is cosmetic in nature.  Any how, that's my opinon. smiley

Kai Schröer's picture

In the meanwhile it looks like this:

Days since last backup: 18

The oldest item in Journaling MBX: 40 Days

The Delay Reports keep in Journal-MBX

The oldest regular items are 2 days old.

Further the failed to copy\failed to store folders are empty.

But please keep in mind that the customer use Domino Journaling instead of Exchange.

I will check further suggestion from this thread and come back.

Thanks and Regards

Kai

PMCS.helpLine Software Gruppe
www.pmcs.de
Twitter: https://twitter.com/PMCS_EV

Kai Schröer's picture

We try this:

SELECT * from JournalArchive
WHERE BackupComplete = '0'
ORDER BY RecordCreationDate

The oldest record is nearly 2 days ago and we have now 164 Days since last backup.

The Policy "Pending Shortcut Timeout" doesn't exist for Domino.

Next and last try is the Dtrace...

:-)

 

PMCS.helpLine Software Gruppe
www.pmcs.de
Twitter: https://twitter.com/PMCS_EV

TonySterling's picture

I would recommend opening a case with support if it bothers the customer that much.  Seems like there might be a bit of a flaw in how it being determined.

SOLUTION