Video Screencast Help
Protect Your POS Environment Against Retail Data Breaches. Learn More.

EV causing Exchange log growth?

Created: 18 Oct 2012 • Updated: 15 Apr 2013 | 10 comments
Per Vaihinen Enter's picture

Our customer have Enterprise Vault 9.0.1 and Exchange 2010 SP2 RU2.

Two weeks ago we installed September, August and July MS updates, after the weekend the exchange server stopped because the exchange log volume had filled up.
We noticed that logs had accumulated 20-30GB a day, which is much higher than normal. While investigating the cause the log rate came back to normal.
The exchange tracking log showed no looping messages or spam attack, eg at night we took a 5 min sample of logs that was 197MBytes and at that time tracking log explorer showed now activity.

This week we noticed that EV hadn't archived since the MS updates were run. So we restarted EV server and the next day we noticed that logs started again to grow at 20-30GB a day rate.
We stopped EV server and now the log rate is back to normal.
We are suspecting that some messages start to loop when EV tries to archive them. I have some evidence pointing to that direction.

So my question is does EV:s archiving cause exchange tracking log event?

Have anybody else noticed similar behavior after september, august or july MS updates.

Comments 10 CommentsJump to latest comment

JesusWept3's picture

I haven't heard of anything, usually the biggest cause of mass log growth like that is RestoreShortcutBody, where it rebuilds every shortcut in the mailbox.

It could also be that the policies might have changed to be more agressive?
I guess you'd really have to look at what the EV server is doing at that time when the logs start to become that active.

 

I have also seen other times a looping issue where EV tries to process a corrupt calendar item over and over and over and over and over again and never finishes, you open up the calendar item and it has some weird property where each attachment has another attachment that has another attachment that has another attachment etc..... that was on EV Journaling though, but would still cause the exchange logs to exploder

KarlW's picture

Please could you help me understand the details and setup a little more by answering the following:

1. How are you measuring the log growth? (For example is this the log growth where the DB is active, or have you accumulated all log growth across the DAG)

2. Do you have a measurement for the amount of data archived during the time the logs grew?

3. Do you have any deletion policies set?  (Storage expiry, shortcut deletion)

4. Do you have any MAC Entourage/Outlook users (and have set the simple mail client - sync shortcuts setting)

5. How new is the mailbox database?  It's not something we were expecting but I'm wondering if the whitespace fix is having an impact.  Do you have a DB that has been created since RU2 was installed that you could validate on?

Regards

Karl

Per Vaihinen Enter's picture

1. There is no DAG so only one mailbox database log grows

2. Our EV guy is busy with other projects today so I will come back to this on monday

3. Same as 2.

4. No MAC Entourage/Outlook users, I know of a few iPhone users

5. About 2 months old so it's a new one

Chau Tran's picture

I have seen a similar issue where MSMQ had loads of messages stuck in the A3, A5 queues causing the Exchange logs to blow up. Normally this should not cause a problem with exchange or EV. What I discovered was there were corrupted messages in the A3 queue and caused a looping consequently caused the problem with exchange logs. This was EV8 and exchange 2007 and it was a while ago whilst working in support at Symantec.

I disabled the corrupted mailbox(es) for EV, cleared the A3 and A5 queues and restarted EV services. Ran manual archiving and it was fixed. Well, for that particular environment anyway. 

 

Regards,

Chau Tran

ASC, STS, SCS and MCITP

John Santana's picture

Hi Chau,

Is it safe to purge the MSMQ queue ? I wonder what's lost when you did that ?

How did you know that particular mailbox which cause the issue, can you please share some insight in this  ?

Kind regards,

John Santana
IT Professional

--------------------------------------------------

Please be nice to me as I'm newbie in this forum.

Chau Tran's picture

Hi John,

I had to enable dtrace and we saw the a couple of particular mailboxes that got stuck i.e. it never got processed correctly (i can't remeber the exact error unfortunately).

It is safe to purge A3 and A5 queues, A5 will get processed again once the schedule archive kick off again. A3 is the manual archive (when you perform a Run Now on the archiving task). Also watch out for other queues such as A6 and A1. A6 normally blows the SQL logs, I have seen a few of those too :-). Becareful not to purge the A1 queues. If A1 queue get stuck you need to troubleshoot why and not to purge it.

Hope that helps.

Regards,

Chau Tran

ASC, STS, SCS and MCITP

John Santana's picture

Thanks for sharing the knowledge Chau Tran, I really appreciate your assistance and help here.

  • A1 -  queue get stuck you need to troubleshoot why and not to purge it.
  • A2
  • A3 safe to purge - manual archive (when you perform a Run Now on the archiving task)
  • A4
  • A5 safe to purge - will get processed again once the schedule archive kick off again
  • A6 - wathc out - normally blows the SQL logs

Kind regards,

John Santana
IT Professional

--------------------------------------------------

Please be nice to me as I'm newbie in this forum.

Chau Tran's picture

Hi John,

Rule of thumb is to NOT purge the A1 as this is the post-processing queue for Pending items in the user's mailboxes. If you purge this queue, potentially you'll have more issues down the line. If you get lots of messages stuck in the queue then dtrace and troubleshoot it (get Symc Tech Support involve if need be).

The A6 queue was the Synchronisation queue which became A7 queue from EV8 onwards. The A6 queue is the Shortcut Updates queue, it is being managed by the "Moved Items" tab in the Mailbox Archiving Policy. This queue usually causes an issue and can cause the SQL log to grow when people upgrade from older version of EV to EV8 and then enable the entire environment to updated all the moved shortcuts within the user's mailboxes.

Hope that helps. Also I am sure others on here will explain this better than I can. :-)

Thanks.

Regards,

Chau Tran

ASC, STS, SCS and MCITP

John Santana's picture

Many thanks Chau Tran !

this is the most comprehensive post knowledge that I know about EV.

Kind regards,

John Santana
IT Professional

--------------------------------------------------

Please be nice to me as I'm newbie in this forum.

Chau Tran's picture

I asked the customer to do the normal exchange/mailbox troubleshooting for the affected mailbox (from the dtrace), e.g. Move to different database, move to another exchange server and recreate mailbox etc. after that they never reported the issue again and yours is the second time I have come across. We weren't able to troubleshoot further from EV point of view as exchange server was down as soon as the log files filled the disk space.

Regards,

Chau Tran

ASC, STS, SCS and MCITP