Video Screencast Help
Search Video Help Close Back
to help
Not able to make it to Vision this year? Get a sampling in the Best of Vision on Demand group.

ShortCut Processing Error in 9.0 SP1

Updated: 16 Mar 2011 | 50 comments
Weasel's picture
0 0 Votes
Login to vote
This issue has been solved. See solution.

Hello,

we are running EV 9.0 SP1 with Exchange 2007 SP3.

Recently I upgraded from 9.0.0 to 9.0.1. Nothing else changed and everything worked fine before this. But now the shortcut processing produces a lot of errors:

Eventid 6578:
Abnormal error occurred
Object: CRetentionCategoryCache
Reference: LE/RE
 

Eventid 6578:
Abnormal error occurred
Object: CRetentionCategoryCache
Reference: RE(1)/fe

Eventid 2270:
A queued operation exceeded the retry count and has been discarded
m_pIBackgroundArchivingAgent->ProcessMovedItemsInFolder2(ExchangeMailboxDn = "/o=ardenne-at/ou=Erste administrative Gruppe/cn=Recipients/cn=OFFICE",
ArchiveID = "1266E621CC231C34B9318C4CEFC9CB4381110000EVSite.2a-net.intern",
ArchiveFolderID = "1F6E0CE30F4A03D419D01C4BA3091E8C11110000EVSite.2a-net.intern",
FolderRetCat = "",
SourceFolderID = "ID",
FolderPathName = "�Posteingang�Ablage�01_Organisation�Strategie",
CompressedItemRequestXML = "some XML",
EVPMFolder = "FALSE")
HRESULT: 0x80004005

I testet this by a manual run of the archive task against one mailbox with shortcut processing only. Pure Archiving works ok and I disabled shortcut processing now. I can see that the A6-Queue is full with all folders the mailbox contains, even if there are definitivly never moved any shortcuts.

I opened a case but did not recieve any answer from symantec yet. Anyone seen this effect too?

 

Regards

Comments

JesusWept2's picture
29
Dec
2010
0 Votes 0
Login to vote

I have the same

I have the same errors
Create a case with support, cross reference case #413-396-627

GertjanA's picture
29
Dec
2010
0 Votes 0
Login to vote

expiry

The below link mentions something about shortcut expiry.

This one: http://www.symantec.com/docs/TECH73137

Might this be the issue?

Thank you, Gertjan
MCSE, MCITP (2x), MCTS, SCS7.5/8.0/9.0, STS9/10
Company: www.t2.nl
The EV-Dashboard. Get it here: http://evdashboard.so

Weasel's picture
30
Dec
2010
0 Votes 0
Login to vote

My Case is

My Case is #413-430-097

@GertjanA: I don't think that storage expiry is a problem. The errors occure during "process moved items" when processing the a6-queue. As far as i know this queue reflects to the folders containing moved shortcuts.

Nick White's picture
30
Dec
2010
0 Votes 0
Login to vote

I think a dtrace of the

I think a dtrace of the Archiving Task is going to be the best step forwards here as we will need to see what is happening in the background when the event is logged

Weasel's picture
30
Dec
2010
0 Votes 0
Login to vote

I will do a dtrace next

I will do a dtrace next monday.

I turned off the option to update moved items in the mailbox policies and archiving is working in normal way.

For a test I activated this option again and let the archiving task run. The a6-queue was filled with about 12000 elements! Never seen this before the upgrade. Every folder of every mailbox is now processed. This takes hours of time. In 9.0 this did not happen, a mailbox without moved itmes never appeared in the a6 queue and moveditems_updatesummary-logs.

This seems to be a bug for me.

BigPhil's picture
30
Dec
2010
0 Votes 0
Login to vote

I also have the same errors.

I also have the same errors. EV 9.0.1 on Windows 2008 R2, Exchange 2007 SP3. I traced the Archiving Task when the errors are logged and sent it to a Symantec Engineer. I'll see what he says about it. If another symantec employee wants to look at the archiving trace, it should be in case 413-350-451. that case is a separate issue with Journaltask.exe failing that should be fixed with a hotfix on monday.

Philip Nunn EV 10.0 on Windows 2008 R2 SP1, Exchange 2007 SP3 RU 5 CCR on Win2008 x64, ESXi 5, Mircosoft TMG, NetApp/EqualLogic storage

If this response answers your question, please mark it as solution

zmcmanus's picture
02
Jan
2011
0 Votes 0
Login to vote

another one as well

My MSMQ A6 queues all maxed out, for each Archive Task... I'd have to say that 9.0.1 introduced this error since my sites have never had this issue prior.

I have unchecked the default maximum size limit for each A6.  The queues are continuing to grow now, at least until I can get an answer from Symantec on it...

JesusWept2's picture
02
Jan
2011
0 Votes 0
Login to vote

turn it off, if you let the

turn it off, if you let the queues grow and grow and grow, you will just cause instability by the MSMQ breaching its quota, and plus the amount of time EV will take to go through all those items just aren't worth it

So consider the fact that you have for exampe 100,000 items in your queue, that will take a long time to process to begin with, but then lets say that normally it would have taken a few minutes to process after its been reported, but now with this delay its taken a week.

What will happen is, if you wait 1 or 2 weeks to start processing that queue, and those users have moved or deleted the items again, you are now going to see errors from the archiving task where it couldn't update because the shortcut has been moved or deleted.

You are best of purging the queue, turning the feature off, working with symantec and then when it is fixed, re-enabling it, and then it will go through and process the items again.

Also what is happening in our case is because of these bugs we're actually getting a lot of blacklisted shrotcuts, when it attempts to update, but fails, and then tries again, and then after the 3rd attempt if it cannot update it, it will mark it so that it does not attempt to process it again.

There is no use in letting the queues fill up, will just cause more issues

Weasel's picture
03
Jan
2011
0 Votes 0
Login to vote

I sent the dtrace-log to

I sent the dtrace-log to symantec support. It takes about 24 hours to process about 12000 items in the a6 queue, so I turned this feature off until this is solved. I will not change the Queue Limit, this makes no sense.

FreKac's picture
03
Jan
2011
0 Votes 0
Login to vote

Seems that one of my

Seems that one of my customers have the same event.

Will check if it's exactly the same behaviour but the event looks the same.

Authorized Symantec Consultant

zmcmanus's picture
03
Jan
2011
0 Votes 0
Login to vote

When you guys are saying you

When you guys are saying you just turn the feature off, are you talking specifically about shortcut processing?  I didnt think you could just do archiving and not shortcut processing, unless you are triggering them manually?

FreKac's picture
03
Jan
2011
0 Votes 0
Login to vote

You can disable the "Moved

You can disable the "Moved Items" features in the mailbox policy (Mbx Policy->Moved Items tab).

Authorized Symantec Consultant

zmcmanus's picture
03
Jan
2011
0 Votes 0
Login to vote

Fantastic! I had completely

Fantastic! I had completely forgotten to check the policy settings for that.  Thank you very much.

Weasel's picture
03
Jan
2011
0 Votes 0
Login to vote

ShortCut-Processing can be

ShortCut-Processing can be deactivated in the mailbox policy. Uncheck the option to update archive location for items moved in the mailbox on the Moved Items tab.

James Herbert's picture
17
Jan
2011
0 Votes 0
Login to vote

Morning All,   Did you find

Morning All,

 

Did you find the long term fix for this?  We have just upgraded to 9.0.1 and are experiencing this issue

Weasel's picture
17
Jan
2011
0 Votes 0
Login to vote

I spoke to technical support

I spoke to technical support last week. Did not hear something about any solution.

vmds's picture
17
Jan
2011
0 Votes 0
Login to vote

We have the same error in our

We have the same error in our environment. After opening a case, the workaround I got was to disable archiving and then to enable update moved shortcuts + shortcut processing bit by bit for each archiv, so the A6 queue process the items without getting brimmed. 

 

According to the supporter, that update moved item feature never worked and must have just started working in EV9 SP1.

So far, doing it one by one works ok. The A6 queue doesnt get crowded anymore and the errors went away.

 

GertjanA's picture
17
Jan
2011
0 Votes 0
Login to vote

uhm?

Don't know which supporter, but this feature works fine in 8...

Thank you, Gertjan
MCSE, MCITP (2x), MCTS, SCS7.5/8.0/9.0, STS9/10
Company: www.t2.nl
The EV-Dashboard. Get it here: http://evdashboard.so

Mohawk Marvin's picture
17
Jan
2011
0 Votes 0
Login to vote

Yea this worked without issue

Yea this worked without issue in all the v8 installs I saw too

vmds's picture
17
Jan
2011
0 Votes 0
Login to vote

According to him it didnt for

According to him it didnt for us. No clue why.

But in fact we have thousands of items in the A6 queue until it fails if we just "let it go" by normal processing.

James Herbert's picture
17
Jan
2011
0 Votes 0
Login to vote

Hi, Have you got a link to

Hi,

Have you got a link to this work around or the instructions from the Symantec enginner?  I'm not sure I understand the process of 'disable archiving and then to enable update moved shortcuts + shortcut processing bit by bit for each archive'

James Herbert's picture
17
Jan
2011
0 Votes 0
Login to vote

Thanks.  What are you

Thanks.  What are you currently doing as a work around?  Just disabling the ShortCut-Processing

vmds's picture
17
Jan
2011
0 Votes 0
Login to vote

Here is what i have

Here is what i have done:

 

1. Disable "normal" archiving via shedule, else it would run into the same problem everytime the shedule starts. As the queues are processed from 1 to 6, the A6 queue will always be AFTER normal archiving process so the A6 queue takes longer and longer to process.

2. Enable update moved shortcuts. 

3. Start processing some of the mailboxes via Exchange Task --> Run now --> Shortcut processing for selected mailboxes. 

Watch the A6 queue. It should build up and then go down again. 

 

Takes a while to process that... enable normal archiving from time to time (disable update moved shortcuts meanwhile) if you like.

Weasel's picture
17
Jan
2011
0 Votes 0
Login to vote

This is not a solution. The

This is not a solution. The feature to update shortcut moves in the archive worked fine from 8 to 9.0. Imagine, you have 1000 or more archives and you do what this supporter sayd - not applicable.

I turned the update moved Items - feature in mailbox policy off and let scheduled archiving run without this until symantec provides us with a propper solution.

vmds's picture
17
Jan
2011
0 Votes 0
Login to vote

Thats why its labeled a

Thats why its labeled a workaround, not a solution :)

 

Its doable for us, as we are just implementing archiving for every user. Had it run as a POC for 2 years now with limited user mailboxes.

Weasel's picture
19
Jan
2011
0 Votes 0
Login to vote

Just spoke to Symantec

Just spoke to Symantec support, seems to be an issue with sp1.

Until it is not solved the moved item processing should be disabled in mailbox policy.

vmds's picture
19
Jan
2011
0 Votes 0
Login to vote

Is there any official

Is there any official document on this issue? Is it first introduced with SP1? 

James Herbert's picture
24
Jan
2011
0 Votes 0
Login to vote

Symantec have acknowledged

Symantec have acknowledged that this is a issue and said a patch should be release by the end of the week, has only heard of it's release?

JamaliP's picture
31
Jan
2011
0 Votes 0
Login to vote

Yeah we have turned off Moved

Yeah we have turned off Moved Items. it was killing our exchanged servers. Hope they release a hotfix soon.

JesusWept2's picture
31
Jan
2011
0 Votes 0
Login to vote

well to be fair, as far as

well to be fair, as far as exchange activity goes and the stress on the exchange servers, especially transaction logs, there is NO hotfix symantec can release for that, its an intensive process, and if you move 1000 items , you will have 1000 transactions in your exchange logs, there is no getting around it

So in your case i would suggest rolling out Virtual vault if the exchange environment is that sensitive to load, as that would actually take stress off the exchange servers and still allow you to move items in a PST like environment within outlook

JamaliP's picture
31
Jan
2011
0 Votes 0
Login to vote

No its the way EV 9.0.1

No its the way EV 9.0.1 processes moved items. like the other posters everything was ok before the upgrade from 8.0.3 or proir to 9.0.1.

EV Director's picture
31
Jan
2011
0 Votes 0
Login to vote

df

Hi,

I'd be grateful if those who think they have this issue and are dealing with support could post their case numbers here so we can look at what data you've sent it and make sure we have a full understanding of the issue.

 

Cheers,

Mike

Mike Bilsborough
Director,Enterprise Vault Engineering Support 

JamaliP's picture
31
Jan
2011
0 Votes 0
Login to vote

Case #413-651-164 - this one

Case #413-651-164 - this one is pending closed, but we were about to update it, because we tested it again over the weekend and had to turn it off due to the high cpu usage on the exchange servers and logs.

Littler-Er's picture
31
Jan
2011
0 Votes 0
Login to vote

My A6 Queue is full too.

I can't seem to make mine shrink with either of these methods.  I am calling support Tomorrow.  This is quite unacceptable.

vmds's picture
01
Feb
2011
0 Votes 0
Login to vote

  Case 413530848

 

Case 413530848 here.

Noticed that even with manual processing single archives the A6 queue will shrink but i can process the archives over and over again and it always "finds" new shortcuts. 

vmds's picture
02
Feb
2011
0 Votes 0
Login to vote

Just wanted to add this: I

Just wanted to add this:

I checked a customer environment where we recently installed the SP1 for EV9. They dont have any Problems with the A6 queue with the moved item feature turned on.

Difference between this customer and our environment is, that we have 2 Ex2010 MBX (4 Databases) and 2 CAS, the customer has 1 EX 2010 that functions as MBX (1 Database) and CAS.

Weasel's picture
02
Feb
2011
0 Votes 0
Login to vote

Got new message from Symantec

Got new message from Symantec Support telling me they are researching the issue.

I sent them a dtrace of archiving against a single mailbox with the Update Moved Items - Feature on.

I can see in the trace log, that every folder is marked as moved, even folders like Sent Items or Tasks. These are not moved and do not contain moved shortcuts! But they where put in the A6 and this happens every time when shortcut processing runs. This is new since SP1 and seems to be the issue.

vmds: Was it a fresh install or an upgrade in your customers environment?

vmds's picture
02
Feb
2011
0 Votes 0
Login to vote

Customer had EV 8.0 before.

Customer had EV 8.0 before. It got updated to EV9.0 and migrated to a 2008 Server. SP1 got installed a week ago. For us we are still running EV9.1 on a 2003 Server (Upgrading since EV 2007)

What is your exchange setup?

Weasel's picture
02
Feb
2011
0 Votes 0
Login to vote

EV is on 2008 SP2. We have

EV is on 2008 SP2.

We have Exchange 2007, two servers with Windows 2003. Both have MBX, CAS and HT roles. One holds all 4 Mailbox databases and the second is configured as a standby machine. We use standby continiouos replication.

I think it does not matter how exchange is configured. It seems that the archiving task picks every folder in the mailboxes regardless if they are moved or contain moved shortcuts. Perhaps there is something wrong in the database.

Nick White's picture
02
Feb
2011
0 Votes 0
Login to vote

We're actively working on

We're actively working on this and have a fix designed for an instance where the top of information store appears to us to have been moved when it hasn't actually been moved at all.

Please keep providing information as we would like to make sure that this is definitely the root cause of all the issues that are being seen and not just one part of it. Thanks to all who have responded so far

Littler-Er's picture
02
Feb
2011
0 Votes 0
Login to vote

Thanks, my A6 Queue shrunk

Thanks, my A6 Queue shrunk after disabling shortcut processing, but that is a feature that we actually need.  Especially with our 3rd party integration with our DMS.  I hope we can get a solution soon.

midlandruss's picture
03
Feb
2011
0 Votes 0
Login to vote

We're experiencing this issue

We're experiencing this issue as well:  case # 413-555-358

EV 9.01 for Exchange 2007 SP2

Basically, Outlook takes forever to load after an archiving run if we have moved-items processing turned on.  Once we turn it off, post-archiving downloads from Exchange function as normal.  The length of load time is directly proportional to the number of emails within the folder.  

JB22's picture
04
Feb
2011
0 Votes 0
Login to vote

Here also

We are also experiencing the issue.

EV9.0.1 on a windows 2003 STD SP2 - Exchange 2003 SP2

Updated from ev 8.0 SP3

 

Case: 413-708-873

GertjanA's picture
07
Feb
2011
0 Votes 0
Login to vote

Case 413-850-303

Please provide an eta on the fix. We're upgrading 7 sites. This needs to be resolved.

Upgraded from EV80SP4 on W2008R2 standard to EV90SP1

Exchange 2007SP2, SQL2005SP3

Thank you, Gertjan
MCSE, MCITP (2x), MCTS, SCS7.5/8.0/9.0, STS9/10
Company: www.t2.nl
The EV-Dashboard. Get it here: http://evdashboard.so

Nick White's picture
09
Feb
2011
0 Votes 0
Login to vote

The only ETA I can give right

The only ETA I can give right now is as soon as possible. We ran through a first pass of testing and found another problem so the fix is being reworked.

olmit's picture
08
Feb
2011
0 Votes 0
Login to vote

Case ID 413-653-943

We have the same problem since updating from 8.0.3 to 9.0.1. Our Environment: Exchange 2003 SP2, Windows 2003. SP2. Case ID 413-653-943.

JamaliP's picture
09
Feb
2011
1 Vote +1
Login to vote

Technote

Anybody with this issue may want to subscribe to the below technote if you havn't already.

 

http://www.symantec.com/docs/TECH152423

Weasel's picture
23
Feb
2011
2 Votes +2
Login to vote

Hi all, I have got a message

Hi all,

I have got a message from sym support with this link to a hotfix for this and other issues:

http://www.symantec.com/docs/TECH153457

I installed the hotfix and at first look the moved items feature is working now. I will let run an archiving with shortcut processing enabled this night and monitor the a6.

ImAlwaysSmiling's picture
04
Mar
2011

Still getting errors after applying hotfix post 9 upgrade.

We too applied the hotfix and were excited to see that finally our exchange servers were not taking the hit that they were taking prior to the hotfix.  However, we have errors that we never had prior to the upgrade that caused all these problems.  We still continue to get the 6578 error and the 2270 error when the archive task is run.  We ran the SQL script to which is in the HOTFIX and it shows us that 1461 mailboxes are affected.  This number does not go down after each archive task.  The 2270 error refers to FolderPathName = Deleted Items which in our exchange environment is a managed folder.  We currently have Archive Deleted Items set to ON and the Archive Exchange Managed Folders set to OFF which is how it was prior to the upgrade.  Which one takes precedence?  We feel that since the upgrade we need to change both of these settings to OFF.  If not the hotfix, then what is going to get rid of these errors that we have been getting since the upgrade???  Any help is appreciated.

Christopher Sorel's picture
16
Mar
2011

same error event 6578

I have opened a case but have not heard back.  all my user complain when outlook opens of long process and repeated 10gb update when outlook opens after my upgrade to v9.01