ShortCut Processing Error in 9.0 SP1
Hello,
we are running EV 9.0 SP1 with Exchange 2007 SP3.
Recently I upgraded from 9.0.0 to 9.0.1. Nothing else changed and everything worked fine before this. But now the shortcut processing produces a lot of errors:
Eventid 6578:
Abnormal error occurred
Object: CRetentionCategoryCache
Reference: LE/RE
Eventid 6578:
Abnormal error occurred
Object: CRetentionCategoryCache
Reference: RE(1)/fe
Eventid 2270:
A queued operation exceeded the retry count and has been discarded
m_pIBackgroundArchivingAgent->ProcessMovedItemsInFolder2(ExchangeMailboxDn = "/o=ardenne-at/ou=Erste administrative Gruppe/cn=Recipients/cn=OFFICE",
ArchiveID = "1266E621CC231C34B9318C4CEFC9CB4381110000EVSite.2a-net.intern",
ArchiveFolderID = "1F6E0CE30F4A03D419D01C4BA3091E8C11110000EVSite.2a-net.intern",
FolderRetCat = "",
SourceFolderID = "ID",
FolderPathName = "�Posteingang�Ablage�01_Organisation�Strategie",
CompressedItemRequestXML = "some XML",
EVPMFolder = "FALSE")
HRESULT: 0x80004005
I testet this by a manual run of the archive task against one mailbox with shortcut processing only. Pure Archiving works ok and I disabled shortcut processing now. I can see that the A6-Queue is full with all folders the mailbox contains, even if there are definitivly never moved any shortcuts.
I opened a case but did not recieve any answer from symantec yet. Anyone seen this effect too?
Regards
Comments
I have the same
I have the same errors
Create a case with support, cross reference case #413-396-627
expiry
The below link mentions something about shortcut expiry.
This one: http://www.symantec.com/docs/TECH73137
Might this be the issue?
Thank you, Gertjan
MCSE, MCITP (2x), MCTS, SCS7.5/8.0/9.0, STS9/10
Company: www.t2.nl
The EV-Dashboard. Get it here: http://evdashboard.so
My Case is
My Case is #413-430-097
@GertjanA: I don't think that storage expiry is a problem. The errors occure during "process moved items" when processing the a6-queue. As far as i know this queue reflects to the folders containing moved shortcuts.
I think a dtrace of the
I think a dtrace of the Archiving Task is going to be the best step forwards here as we will need to see what is happening in the background when the event is logged
I will do a dtrace next
I will do a dtrace next monday.
I turned off the option to update moved items in the mailbox policies and archiving is working in normal way.
For a test I activated this option again and let the archiving task run. The a6-queue was filled with about 12000 elements! Never seen this before the upgrade. Every folder of every mailbox is now processed. This takes hours of time. In 9.0 this did not happen, a mailbox without moved itmes never appeared in the a6 queue and moveditems_updatesummary-logs.
This seems to be a bug for me.
I also have the same errors.
I also have the same errors. EV 9.0.1 on Windows 2008 R2, Exchange 2007 SP3. I traced the Archiving Task when the errors are logged and sent it to a Symantec Engineer. I'll see what he says about it. If another symantec employee wants to look at the archiving trace, it should be in case 413-350-451. that case is a separate issue with Journaltask.exe failing that should be fixed with a hotfix on monday.
Philip Nunn EV 10.0 on Windows 2008 R2 SP1, Exchange 2007 SP3 RU 5 CCR on Win2008 x64, ESXi 5, Mircosoft TMG, NetApp/EqualLogic storage
If this response answers your question, please mark it as solution
another one as well
My MSMQ A6 queues all maxed out, for each Archive Task... I'd have to say that 9.0.1 introduced this error since my sites have never had this issue prior.
I have unchecked the default maximum size limit for each A6. The queues are continuing to grow now, at least until I can get an answer from Symantec on it...
turn it off, if you let the
turn it off, if you let the queues grow and grow and grow, you will just cause instability by the MSMQ breaching its quota, and plus the amount of time EV will take to go through all those items just aren't worth it
So consider the fact that you have for exampe 100,000 items in your queue, that will take a long time to process to begin with, but then lets say that normally it would have taken a few minutes to process after its been reported, but now with this delay its taken a week.
What will happen is, if you wait 1 or 2 weeks to start processing that queue, and those users have moved or deleted the items again, you are now going to see errors from the archiving task where it couldn't update because the shortcut has been moved or deleted.
You are best of purging the queue, turning the feature off, working with symantec and then when it is fixed, re-enabling it, and then it will go through and process the items again.
Also what is happening in our case is because of these bugs we're actually getting a lot of blacklisted shrotcuts, when it attempts to update, but fails, and then tries again, and then after the 3rd attempt if it cannot update it, it will mark it so that it does not attempt to process it again.
There is no use in letting the queues fill up, will just cause more issues
I sent the dtrace-log to
I sent the dtrace-log to symantec support. It takes about 24 hours to process about 12000 items in the a6 queue, so I turned this feature off until this is solved. I will not change the Queue Limit, this makes no sense.
Seems that one of my
Seems that one of my customers have the same event.
Will check if it's exactly the same behaviour but the event looks the same.
Authorized Symantec Consultant
When you guys are saying you
When you guys are saying you just turn the feature off, are you talking specifically about shortcut processing? I didnt think you could just do archiving and not shortcut processing, unless you are triggering them manually?
You can disable the "Moved
You can disable the "Moved Items" features in the mailbox policy (Mbx Policy->Moved Items tab).
Authorized Symantec Consultant
Fantastic! I had completely
Fantastic! I had completely forgotten to check the policy settings for that. Thank you very much.
ShortCut-Processing can be
ShortCut-Processing can be deactivated in the mailbox policy. Uncheck the option to update archive location for items moved in the mailbox on the Moved Items tab.
Morning All, Did you find
Morning All,
Did you find the long term fix for this? We have just upgraded to 9.0.1 and are experiencing this issue
I spoke to technical support
I spoke to technical support last week. Did not hear something about any solution.
We have the same error in our
We have the same error in our environment. After opening a case, the workaround I got was to disable archiving and then to enable update moved shortcuts + shortcut processing bit by bit for each archiv, so the A6 queue process the items without getting brimmed.
According to the supporter, that update moved item feature never worked and must have just started working in EV9 SP1.
So far, doing it one by one works ok. The A6 queue doesnt get crowded anymore and the errors went away.
uhm?
Don't know which supporter, but this feature works fine in 8...
Thank you, Gertjan
MCSE, MCITP (2x), MCTS, SCS7.5/8.0/9.0, STS9/10
Company: www.t2.nl
The EV-Dashboard. Get it here: http://evdashboard.so
Yea this worked without issue
Yea this worked without issue in all the v8 installs I saw too
According to him it didnt for
According to him it didnt for us. No clue why.
But in fact we have thousands of items in the A6 queue until it fails if we just "let it go" by normal processing.
Hi, Have you got a link to
Hi,
Have you got a link to this work around or the instructions from the Symantec enginner? I'm not sure I understand the process of 'disable archiving and then to enable update moved shortcuts + shortcut processing bit by bit for each archive'
Thanks. What are you
Thanks. What are you currently doing as a work around? Just disabling the ShortCut-Processing
Here is what i have
Here is what i have done:
1. Disable "normal" archiving via shedule, else it would run into the same problem everytime the shedule starts. As the queues are processed from 1 to 6, the A6 queue will always be AFTER normal archiving process so the A6 queue takes longer and longer to process.
2. Enable update moved shortcuts.
3. Start processing some of the mailboxes via Exchange Task --> Run now --> Shortcut processing for selected mailboxes.
Watch the A6 queue. It should build up and then go down again.
Takes a while to process that... enable normal archiving from time to time (disable update moved shortcuts meanwhile) if you like.
This is not a solution. The
This is not a solution. The feature to update shortcut moves in the archive worked fine from 8 to 9.0. Imagine, you have 1000 or more archives and you do what this supporter sayd - not applicable.
I turned the update moved Items - feature in mailbox policy off and let scheduled archiving run without this until symantec provides us with a propper solution.
Thats why its labeled a
Thats why its labeled a workaround, not a solution :)
Its doable for us, as we are just implementing archiving for every user. Had it run as a POC for 2 years now with limited user mailboxes.
Just spoke to Symantec
Just spoke to Symantec support, seems to be an issue with sp1.
Until it is not solved the moved item processing should be disabled in mailbox policy.
Is there any official
Is there any official document on this issue? Is it first introduced with SP1?
Symantec have acknowledged
Symantec have acknowledged that this is a issue and said a patch should be release by the end of the week, has only heard of it's release?
Yeah we have turned off Moved
Yeah we have turned off Moved Items. it was killing our exchanged servers. Hope they release a hotfix soon.
well to be fair, as far as
well to be fair, as far as exchange activity goes and the stress on the exchange servers, especially transaction logs, there is NO hotfix symantec can release for that, its an intensive process, and if you move 1000 items , you will have 1000 transactions in your exchange logs, there is no getting around it
So in your case i would suggest rolling out Virtual vault if the exchange environment is that sensitive to load, as that would actually take stress off the exchange servers and still allow you to move items in a PST like environment within outlook
No its the way EV 9.0.1
No its the way EV 9.0.1 processes moved items. like the other posters everything was ok before the upgrade from 8.0.3 or proir to 9.0.1.
df
Hi,
I'd be grateful if those who think they have this issue and are dealing with support could post their case numbers here so we can look at what data you've sent it and make sure we have a full understanding of the issue.
Cheers,
Mike
Mike Bilsborough
Director,Enterprise Vault Engineering Support
Case #413-651-164 - this one
Case #413-651-164 - this one is pending closed, but we were about to update it, because we tested it again over the weekend and had to turn it off due to the high cpu usage on the exchange servers and logs.
My A6 Queue is full too.
I can't seem to make mine shrink with either of these methods. I am calling support Tomorrow. This is quite unacceptable.
Case 413530848
Case 413530848 here.
Noticed that even with manual processing single archives the A6 queue will shrink but i can process the archives over and over again and it always "finds" new shortcuts.
Just wanted to add this: I
Just wanted to add this:
I checked a customer environment where we recently installed the SP1 for EV9. They dont have any Problems with the A6 queue with the moved item feature turned on.
Difference between this customer and our environment is, that we have 2 Ex2010 MBX (4 Databases) and 2 CAS, the customer has 1 EX 2010 that functions as MBX (1 Database) and CAS.
Got new message from Symantec
Got new message from Symantec Support telling me they are researching the issue.
I sent them a dtrace of archiving against a single mailbox with the Update Moved Items - Feature on.
I can see in the trace log, that every folder is marked as moved, even folders like Sent Items or Tasks. These are not moved and do not contain moved shortcuts! But they where put in the A6 and this happens every time when shortcut processing runs. This is new since SP1 and seems to be the issue.
vmds: Was it a fresh install or an upgrade in your customers environment?
Customer had EV 8.0 before.
Customer had EV 8.0 before. It got updated to EV9.0 and migrated to a 2008 Server. SP1 got installed a week ago. For us we are still running EV9.1 on a 2003 Server (Upgrading since EV 2007)
What is your exchange setup?
EV is on 2008 SP2. We have
EV is on 2008 SP2.
We have Exchange 2007, two servers with Windows 2003. Both have MBX, CAS and HT roles. One holds all 4 Mailbox databases and the second is configured as a standby machine. We use standby continiouos replication.
I think it does not matter how exchange is configured. It seems that the archiving task picks every folder in the mailboxes regardless if they are moved or contain moved shortcuts. Perhaps there is something wrong in the database.
We're actively working on
We're actively working on this and have a fix designed for an instance where the top of information store appears to us to have been moved when it hasn't actually been moved at all.
Please keep providing information as we would like to make sure that this is definitely the root cause of all the issues that are being seen and not just one part of it. Thanks to all who have responded so far
Thanks, my A6 Queue shrunk
Thanks, my A6 Queue shrunk after disabling shortcut processing, but that is a feature that we actually need. Especially with our 3rd party integration with our DMS. I hope we can get a solution soon.
We're experiencing this issue
We're experiencing this issue as well: case # 413-555-358
EV 9.01 for Exchange 2007 SP2
Basically, Outlook takes forever to load after an archiving run if we have moved-items processing turned on. Once we turn it off, post-archiving downloads from Exchange function as normal. The length of load time is directly proportional to the number of emails within the folder.
Here also
We are also experiencing the issue.
EV9.0.1 on a windows 2003 STD SP2 - Exchange 2003 SP2
Updated from ev 8.0 SP3
Case: 413-708-873
Case 413-850-303
Please provide an eta on the fix. We're upgrading 7 sites. This needs to be resolved.
Upgraded from EV80SP4 on W2008R2 standard to EV90SP1
Exchange 2007SP2, SQL2005SP3
Thank you, Gertjan
MCSE, MCITP (2x), MCTS, SCS7.5/8.0/9.0, STS9/10
Company: www.t2.nl
The EV-Dashboard. Get it here: http://evdashboard.so
The only ETA I can give right
The only ETA I can give right now is as soon as possible. We ran through a first pass of testing and found another problem so the fix is being reworked.
Case ID 413-653-943
We have the same problem since updating from 8.0.3 to 9.0.1. Our Environment: Exchange 2003 SP2, Windows 2003. SP2. Case ID 413-653-943.
Technote
Anybody with this issue may want to subscribe to the below technote if you havn't already.
http://www.symantec.com/docs/TECH152423
Hi all, I have got a message
Hi all,
I have got a message from sym support with this link to a hotfix for this and other issues:
http://www.symantec.com/docs/TECH153457
I installed the hotfix and at first look the moved items feature is working now. I will let run an archiving with shortcut processing enabled this night and monitor the a6.
Still getting errors after applying hotfix post 9 upgrade.
We too applied the hotfix and were excited to see that finally our exchange servers were not taking the hit that they were taking prior to the hotfix. However, we have errors that we never had prior to the upgrade that caused all these problems. We still continue to get the 6578 error and the 2270 error when the archive task is run. We ran the SQL script to which is in the HOTFIX and it shows us that 1461 mailboxes are affected. This number does not go down after each archive task. The 2270 error refers to FolderPathName = Deleted Items which in our exchange environment is a managed folder. We currently have Archive Deleted Items set to ON and the Archive Exchange Managed Folders set to OFF which is how it was prior to the upgrade. Which one takes precedence? We feel that since the upgrade we need to change both of these settings to OFF. If not the hotfix, then what is going to get rid of these errors that we have been getting since the upgrade??? Any help is appreciated.
same error event 6578
I have opened a case but have not heard back. all my user complain when outlook opens of long process and repeated 10gb update when outlook opens after my upgrade to v9.01
Would you like to reply?
Login or Register to post your comment.