Video Screencast Help

PST import when only journaling is used

Created: 01 May 2013 • Updated: 10 May 2013 | 24 comments
This issue has been solved. See solution.


I used a third party tool to migrate data from a legacy archive (Mimosa NearPoint) into an Exchange  journaling mailbox that gets archived by EV 10.0.2. It is taking a very long time for EV to pull the archived messages out of Exchange, probably because of the poor format of some of the migrated messages. I was wondering if I create PSTs can I import them into the journal archive? Or does this only work with Mailbox archiving?


Operating Systems:

Comments 24 CommentsJump to latest comment

Rob.Wilcox's picture

You can't import to a journal archive from the VAC.
You can import to a journal archive using EVPM.

How are you going to get the items in to PSTs though? Or do you already have them in PSTs?

JesusWept3's picture

If you import via PST then you are going to have issues if there is a journal report attached to it as the PST will process it like a regular message and will not perform the same actions as the journal archiving task

Rob.Wilcox's picture

And, usually, for the reasons that JW3 mentioned, many people import 'old' data into a shared archive.

JesusWept3's picture

If you import in to a shared archive you will be doubly messed up

The journal report will be treated as the main message, the real message which is the P2 will be treated as an attachment and the attachments won't be indexed

Rob.Wilcox's picture

That is IF these mails are proper journal mails with a P1 and P2

JesusWept3's picture

Yeah I guess it depends on the third party tool, so if it was TVS for instance it will do envelope reconstruction etc

But if there is an envelope do not push in to a shared archive or import via EVPM

Set up a new mailbox, target it as a journal mailbox in EV and then import the PST's in to that mailbox so you keep your production journal seperate from your historic import journal

TheEmptyMind's picture

I would additionally add something that I wish I knew when I was doing the migration.  Set "Expand Distribution Lists" to Off in your Journal archiving policy, especially if P1 is being constructed by the migration software. Otherwise, should the DL still exist, EV will add additional recipients (that exist in the DL but not in P1) and archive the message, which makes the message non-compliant.

Also, set "Failed DL expansion behavior" to Archive, if it isn't already.  Otherwise, EV will simply move the message to "Failed DL Expansion" folder under the journal mailbox, should the DL doesn't exist in GAL.

JesusWept3's picture

Agreed 100%
So if you go that route, create a secondary journal policy and apply it to the historic import

That way you don't cause issues for your production journals

Rob.Wilcox's picture

I agree with these comments so far... but back to the OP.  Do you already have the items in a PST file? Creating PSTs, then ingesting them should surely be slower than just letting EV archive them?

Are you doing them via journal archiving, or mailbox archiving?

How much data? What sort of archiving rate are you getting?

rnt123's picture

WOW! Lots of great input here, but I think I have more questions than when I started is actually TransVault The items are not in PST, but I could put future items into PSTs, as I paused the migration operation at the 30% mark. I could also move the already migrated items to a PST easily via Outlook of course. They are being put into EV via Journal archiving. There are no mailboxes being archived at all. TransVault cannot put data directly into EV, only Exchange or PSTs. The amount of data is about 10GB with about 2.7 already migrated. The funny thing is, items coming into the journal the normal way (by capturing a copy from an Exchange mailbox or transport get archived at a rate of about 100 items per minute. The stuff from TransVault only gets put in about 1 item per minute.

Rob.Wilcox's picture

So the slow bit is getting the data from tansvault into the mailbox, ready for it to be journal archived by EV?

rnt123's picture

Nope. Transvault gets the mail out of Mimosa into Exchnage very quickly. The slow part is rom The Exchange journal mailbox to EV.

TonySterling's picture

Hmm, that normally isn't an issue.  Are you using multiple Journal mailboxes so you can load one with TVS whilst EV journals the other?

It would be worth dtracing the process to find out what is going on.

JesusWept3's picture

I've seen this happen when something like a third party filter is used on the journal such as DCS or Orchestria

Do you have any custom filtering or selective journaling configured?

Like rob said it may be worth a DTrace and looking at the time to process

But yeah, if you haven't, set up a new journal policy and a new journal mailbox, and a new journal task for that mailbox and the. Import the PST cos otherwise you will interfere with production and it could be something like DL expansion causing issues etc

rnt123's picture

Running a Dtrace now. One thing I notice is that the "stuck" messages have a message class of Enterprise Vault Pending Archive Part, and then quickly disappear after changing to Enterprise Vault Pending Archive before they are removed from Exchange. I wonder if "Part" is a clue to the problem.

Rob.Wilcox's picture

If I remember correctly they turn in to .part messages and wait 5 minutes (it's the journaldelay setting in the policy) before they get made pending, and then archived.

So that part, I think, might well be normal. Having lost my demo environment today whilst I rebuild my machine I can't tell you for 110% sure.

JesusWept3's picture

Rob is absolutely correct
It will go from IPM.Note -> IPM.Note.EnterpriseVault.PendingPart -> IPM.EnterpriseVault.Pending -> Delete

The pending part is controlled by the JournalDelay set in the registry.
It waits for 5 minutes by default to see if any new journal reports come in and merges them if they do so you dont have duplicates. After the 5 minutes it changes it to pending, sends to storage and once its been archived correctly, it will go back and delete the item

it should be doing more than 1 item at a time though

rnt123's picture

Do you have the path to this key? i would love to speed things up! thanks JesusWept3!

JesusWept3's picture

Well heres the thing, it should only delay it by 5 minutes at most, it should only appear to be slow right at the beginning....the thing is though with that registry key though is that it affects any of the journal tasks for that EV Server

So first question,
1. Do you have a seperate Mailbox for these imports, or are you importing them in to the production journal mailbox?
2. Did you create a new Journal Archive Policy and apply it to the mailbox that you're importing the PST in to?
3. Did you disable the expansion of Distribution Lists etc that were suggested above?

TonySterling's picture

Just FYI,

Journal Delay can now be set in the Exchange journal policy in the advanced settings

JesusWept3's picture

LOL, yeah, errr i knew that!!! :)
Thanks Tony.

So yeah IF you have a seperate mailbox for the PST imports thats targeted by its own policy, you can set the journalDelay setting via the policy as not to interrupt the production email etc

But to be honest, i don't think its going to speed up anything, but worth a try

TonySterling's picture

wink Yeah, I agree.  The journal delay shouldn't be causing this unless TVS is assigning the same message-id but I doubt that would be the issue.

Enterprise Vault (EV) Journaling experiences a performance degradation when multiple different messages shares the same 'Message-Id'
padding: 1px;padding-bottom: 3px ;font: 12px Arial; text-align: left;">Article:TECH181078 padding: 1px;font: 12px Arial; text-align: left;"> |  padding: 0px;font: 12px Arial; text-align: left;">Created: 2012-02-09 padding: 1px;font: 12px Arial; text-align: left;"> |  padding: 1px;font: 12px Arial; text-align: left;">Updated: 2013-02-13 padding: 1px;font: 12px Arial; text-align: left;"> |  padding: 1px;font: 12px Arial; text-align: left;">Article URL
Rob.Wilcox's picture

Anything else needed for this issue, or is it resolved?

rnt123's picture

I found that if I moved the items into separate folders in amounts of less than 5,000 EV could archive them faster, and archiving would not stop randomly. It is still not as fast as regular archiving from the Exchange database, but then a lot of these messages have deleted recipients, corrupt envelops, etc. Thanks everyone for the help!