Video Screencast Help

Agent service crashing

Created: 22 Mar 2011 • Updated: 30 Mar 2011 | 24 comments

Hey all,

We have been having a random problem with 7.0 agents on our network wherin the Symantec Management Agent service just simply shuts down without warning and then doesn't restart.  It will restart if you manually kick the service OR if you set the recovery options to "restart the service".

In Event viewer we are getting EventID 7034, which is basically just the service control manager logging the fact that the service stopped.  This is the only note we are getting.

We have started running the diagnostic agent on a few key machines, but haven't collected any data on a crash yet.

All of our systems are XP SP3 machines, running the latest 7.0.x agent.

Has anyone seen this issue?

Comments 24 CommentsJump to latest comment

luke.s's picture

Hi Mmoney,

Could you please attach the Agent.log for some of these computers where you're getting the EventID 7034?

Regards,

If the suggestion has helped to solve your problem, please mark the post as a solution.

Fábio Sanches
IT Technical Manager | WTR Services | www.wtrservices.com.br

mmooney's picture

I have agent(#).log numbers 1 - 20 available, and the just plain agent.log

luke.s's picture

The agent.log is the current Altiris Agent log. If is posible, zip all the logs and send to us.

Did you also verified the itens suggested by mclemson?

Regards,

If the suggestion has helped to solve your problem, please mark the post as a solution.

Fábio Sanches
IT Technical Manager | WTR Services | www.wtrservices.com.br

mmooney's picture

Not really sure what you mean by that.  Mclemson indicated that App Metering (Through Altiris, I assume) and AV might be problems.  Ok, we have both.

I don't mean to sound unappreciative, but I need more information.  Like, how and why are these things a problem?  What should I be looking for to potentially isolate one or the other or both as a problem?  If they are an issue, what do I do about it?  (Removing either is a non-starter, btw)  If they are known issues, is there a symwise article about it?  I need more information.

Also, I have sent some logs in on the ticket I have open on this issue.  I can attach them here if you want, just let me now.

mclemson's picture

Logs would be good.  Looking for last few things to happen before it crashes.  What files does it access?  What tasks does it perform?  What's similar between different crash incidents?

And of course the typical, "What might have changed around the time this issue began?" is important.

Mike Clemson, Senior Systems Engineer, ASC
Intuitive Technology Group -- Symantec Platinum Partner
intuitivetech.com

mmooney's picture

I have attached 2 log files from 2 different machines that are known to have the "Agent dissapears" problem.

Most of the rest of the questions should be answerable by the log files.

As far as system changes go, the change was installing Altiris.  We've been having trouble with this, to greater or lesser extent, since original install day.  There doesn't appear to be any rhyme or reason for the issue either.  It has been happening on machines that had been unchanged since the Altiris install last year, and on other machines that have radically changed.  One machine that is used by another of the IT personnell here has NEVER had a functional Altiris client.  It hasn't worked since day 1.

All machines are from the same basic image (with variances for drivers)

All machines use the same Antivirus - McAffee

All machines have the same basic software, with some departmental differences.  (I should note that the failures cross departmental lines, so unlikely to be related to a specific department's software)

That's all I can think of right now.  Hopefully the log files will be more revealing.

AttachmentSize
agentlog.zip 16.57 KB
luke.s's picture

Hi mmoney,

Seconds before the crash error is logged the following messages:

<event date='Mar 11 09:16:32' severity='2' hostName='MCCOURTMDT' source='Agent' module='AeXNSAgent.exe' process='AeXNSAgent.exe' pid='228' thread='180' tickCount='199392593' >

<![CDATA[Object Altiris.ProcessMonitor is not installed: Invalid class string (-2147221005). This situation should be resolved once the agent rollout has been completed.]]>

To solve this first problem there are two posible ways:

1) The Client Task Agent.dll must be re-registered with the system.  This is done by using the following command at a command prompt or script: regsvr32 <filename>.  The default path to the file is: C:\Program Files\Altiris\Altiris Agent\Agents\Client Task Agent\ Client Task Agent.dll.

2) The Server Settings Notification Server Infrastructure Task Server Agent Configuration policy is associated with the All Computers collection by default, which is why all resources are receiving configuration information for the Client task Agent in their Client Settings Policies, which is causing the NSAgent to record these warnings. So replace the All Computers collection association with the With Client Task Agent Installed collection. (Article HOWTO7166)

Please could you apply this steps on some computers and let us know about the results?

Regards,

 

If the suggestion has helped to solve your problem, please mark the post as a solution.

Fábio Sanches
IT Technical Manager | WTR Services | www.wtrservices.com.br

mmooney's picture

I'm not seeing :Server Settings > Notification Server Infrastructure > Task Server > Agent Configuration anywhere in my interface.  Should I be looking under Settings for this, or under Manage ->Policies?

luke.s's picture

Hey mmooney,

The correct path to search at NS 7.1 version (and I think that is the same at 7.0) is:

Settings -> Notification Server -> Task Settings -> Task Agent Settings.

Regards,

If the suggestion has helped to solve your problem, please mark the post as a solution.

Fábio Sanches
IT Technical Manager | WTR Services | www.wtrservices.com.br

mmooney's picture

My Task agent is set to check for new tasks every 30 minutes.  I think that's the default as I don't remember changing this.  Should it be more or less often?

luke.s's picture

No problem with this configuration. Including is recommended that check interval do not be so frequent like 1 minute, 2 minutes, or something like that.

Did you re-registered the Client Task Agent.dll at that workstation?

Thank you for your patience in provide the details and to are following the troubleshooting process.

Regards,

If the suggestion has helped to solve your problem, please mark the post as a solution.

Fábio Sanches
IT Technical Manager | WTR Services | www.wtrservices.com.br

mmooney's picture

We re-registered that DLL.  Do you need me to pull logs from it again?

mclemson's picture

A few thoughts after looking through the logs.

I see that a bunch of detection checks happen for managed software deliveries at 9 a.m.  Although this probably isn't the cause, don't run these unnecessarily often.  I have limited data in this small log, but if it's happening every 30 minutes or every hour, move to something more reasonable like weekly, daily, every 4-8 hours, etc.

Application Metering is referenced right after the stack trace:
C:\Program Files\Altiris\Altiris Agent\Agents\Application Metering Agent\AMAgent.dll

What application metering policies are active?

What do these GUIDs reference?  They are used just before the crash:
D23D8F52-2C9D-4D08-9E13-B0D9D7CD6BF4
83377032-df91-4738-ac2c-bdc7a343efb6
Use the item browser on the NS within the Altiris Log Viewer to find out.

Is your task interval check set to 1 minute?  You may want to change this to something more reasonable.

Agent12.log also shows a reference at Mar 08 07:53:37 to AMAgent.dll before the crash.

These lines are especially interesting:
  <![CDATA[Could not find a matching application in the App metering summary file for application details:    gravitixservice.exe   C:\Program Files\PatchLink\Update Agent\GravitixService.exe SYSTEM LOCALMACHINE.]]>
</event>
<event date='Mar 22 09:06:55' severity='4' hostName='REBMANNJDTN' source='CAeXEventManager::DoRemoveProcess' module='AMAgent.dll' process='AeXNSAgent.exe' pid='8412' thread='9852' tickCount='-1875451703' >
  <![CDATA[Could not find a matching application in the App metering summary file for application details: Microsoft Corporation svchost.exe 5.1.2600.5512 (xpsp.080413-2111) svchost.exe Microsoft® Windows® Operating System 5.1.2600.5512 C:\WINDOWS\system32\svchost.exe SYSTEM LOCALMACHINE.]]>
</event>

Do you have policies for these .exe's?  They're the final lines in Agent12.log just before the next crash.

It seems Application Metering is the issue.  What policies are enabled?

Mike Clemson, Senior Systems Engineer, ASC
Intuitive Technology Group -- Symantec Platinum Partner
intuitivetech.com

mmooney's picture

 

As far as policies turned on go, I turned on ALL of the default 7.0.x policies right out of the box.  I would list them but it is rather extensive.  Needless to say, it's all the defaults, NO customization except in the blacklist, where we have blocked iTunes.

That patchlink thing is part of the old patching system.  We removed the old server, but never got around to removing the old client.  (We were rushed and forgot to send out a "remove the client" job from the old server before we pulled it.  Doh!)  It is possible that client could be interfering with Altiris.  Some systems have had it removed, some haven't.

I can easily build a task to remove the client as I still have a copy of the client installer msi.  I'll do that and run it on a couple of the affected systems and see if it helps.

mclemson's picture

Good plan with the patchlink app.  I would disable all of your App Metering policies as well, unless you need them.  If you kept all default settings for App Metering it shouldn't be an issue to leave Office and such enabled.  Keeping video game-based policies enabled if the games are not installed anywhere does not make sense, and I would disable them.

Mike Clemson, Senior Systems Engineer, ASC
Intuitive Technology Group -- Symantec Platinum Partner
intuitivetech.com

mmooney's picture

Ok, I just checked for gravitixservice.exe on both of the machines that i sent log files from, and BOTH have had the Patchlink agent uninstalled and the program files folders deleted.  So the "gravitixservice.exe" file doesn't exist on either system anymore.

The other executable you note is the Windows Service Host, isn't it?  Why would that be a problem?

mclemson's picture

It wouldn't.  But Application Metering was the last thing running.  That's important to know.

How quickly will these agents crash?  If it's predictable, I'd consider disabling McAfee.  Maybe it doesn't like App Metering putting hooks into the apps when they launch, thinks it's a virus.  McAfee has been bad news for us in other ways, but I don't have any environments that include McAfee and Altiris, so I can't comment.

As a temporary test, of course.

Mike Clemson, Senior Systems Engineer, ASC
Intuitive Technology Group -- Symantec Platinum Partner
intuitivetech.com

mmooney's picture

Just an update to this issue:

 I've removed the old patchlink agent from all systems, no changes.

If it is the Application Metering plugin in conflict with the McAfee AV that is causing this, is there a specific service or software name for the plugin?  I ask because Mcafee will allow us to add software and/or services to an "ignore" list.  I could simply add the App Metering service and program into this list and that should fix the problem.

mclemson's picture

It should just be AMAgent.dll.  But if McAfee is objecting to it interacting with the program .exe, that won't help.  I'd recommend the following:

  1. Find repeatable way to crash system OR determine how long a system will take to crash the agent
  2. Disable McAfee completely and/or uninstall
  3. See if crash problem is resolved
  4. Reinstall McAfee and determine what exclusion might be necessary

I'd consider opening a support case as well.

Mike Clemson, Senior Systems Engineer, ASC
Intuitive Technology Group -- Symantec Platinum Partner
intuitivetech.com

mclemson's picture

Two quick items to look at would be App Metering and antivirus.  They've interfered in the past with agent operation.

Mike Clemson, Senior Systems Engineer, ASC
Intuitive Technology Group -- Symantec Platinum Partner
intuitivetech.com

ADH's picture

We have same problem here.  We discovered it in the last few days because an msi rollout failed to deploy to some PCs.

Multiple XP SP3 PCs running v 7.0.8641.1641 agents but not all.  Agent service will not start and error logged in system log:

 

Event ID:  7034

The Symantec Management Agent Service terminated unexpectedly

 

-------------------------------------------

Agent.log output:

 

Exception - Access Violation: C0000005

Bit Flag: 00000000 - Memory Address Accessed: 54746E65 : 08E6EDEC

EAX=08E6F12C EBX=00000200 ECX=54746E65 EDX=00000004 ESI=08E6F4C0
EDI=08E6F4B4 EBP=08E6F0E4 ESP=08E6F0BC EIP=021C9699 FLG=00010206
CS=001B DS=001B SS=0023 ES=0023 FS=0023 GS=003B
Stack dump:

Stack Trace
]]>
</event>
<event date='Mar 23 09:42:48' severity='2' hostName='PC129' source='Altiris Agent' module='AeXNSAgent.exe' process='AeXNSAgent.exe' pid='4768' thread='836' tickCount='-1697698812' >
  <![CDATA[Unexpected exit C0000005]]>
</event> 

ADH's picture

I have physically visited around 20 PCs and manually uninstalled the Symantec Management Agent and reinstalled it.  Everything works fine until I notice that there are many PCs that need a Deployment Plug-in upgrade.  One of them is my own PC.  So I enable the deployment plug-in upgrade and as soon as it runs, I notice the agent icon disappears, the agent service crashes and will not start again, requiring another uninstall of the symantec management agent.

So one thing that is causing a problem is installation or upgrade of the deployment plug-in.  It kills the management agent service and it will not start again.

Same error is logged:  ID 7034

The Symantec Management Agent Service terminated unexpectedly 

luke.s's picture

Hey ADH, who are you?

Could you please verify which DS version are you using?

There is a similiar issue with Deployment Solution 7.1.1374.

Regards,

If the suggestion has helped to solve your problem, please mark the post as a solution.

Fábio Sanches
IT Technical Manager | WTR Services | www.wtrservices.com.br

ADH's picture

Hi luke.s,

I am not absolutely sure of the versions numbers we were using at the time we had the problem as I have been downloading a considerable number of updates through the installation manager of late .

However, originally I think we were using DS 7.1.607.  I have now downloaded an update to version 7.1.643 but I haven't yet deployed this to all workstations.  I have deployed 7.1.643 to about 5 workstations.

 

Our older config was as follows:

 

Symantec Management Agent:  7.0.7436

Software Update:  7.0.4210

Application Metering:  7.0.1255

Base Task Handlers:   7.0.7416

Client Task Agent:  7.0.7418

Depolyment Solution:  7.1.607

Inventory Agent:  7.0.1255

Inventory Rule Agent:  7.0.7416

Out of Band:  7.0.1541

pcAnywhere:  12.5.539

Software Management Framework:  7.0.7417

Software Management Solution:  7.0.1866

 

Our newer config is as follows:

 

Symantec Management Agent:  7.0.8641.1641

Software Update:  7.0.4409

Application Metering:  7.0.1295

Base Task Handlers:   7.0.8641

Client Task Agent:  7.0.8641

Depolyment Solution:  7.1.643

Inventory Agent:  7.0.1295

Inventory Rule Agent:  7.0.8641

Out of Band:  7.0.1541

pcAnywhere:  12.5.539

Software Management Framework:  7.0.8641

Software Management Solution:  7.0.1914