Symantec Management Platform (Notification Server)

I am seeing the altiris agent continually crash on new installs of the agent 

This is what event viewer is showing.  Any thoughts?

Faulting application name: AeXNSAgentHostSurrogate32.exe, version: 7.5.3300.0, time stamp: 0x5481b7f7
Faulting module name: OLEAUT32.dll, version: 6.1.7601.17676, time stamp: 0x4e58702a
Exception code: 0xc0000005
Fault offset: 0x0000b13a
Faulting process id: 0xc8c
Faulting application start time: 0x01d0aead8facaf8e
Faulting application path: C:\Program Files\Altiris\Altiris Agent\x86\AeXNSAgentHostSurrogate32.exe
Faulting module path: C:\Windows\syswow64\OLEAUT32.dll
Report Id: f9ee1e0f-1aa0-11e5-b3e2-c4d987e6f22a

Hi SonicGT,

Is it possible to share crash dump file and logs from problematic computer?

On WIndows7 default paths for dump and logs are:

C:\ProgramData\Symantec\Symantec Agent\CrashDumps

C:\ProgramData\Symantec\Symantec Agent\Logs

Thank you,

Alex.

I'll see what I can grab, it may not be as widespread as I thought, just ive seen more then a few on machines brought to my attention.  reinstalling seems to fix.  I'll post the logs when i get another one

Thank you, SonicGT.

I have found couple of similar crashes in our database but they all related to 7.1.x codebase.

Probably you know about reports that might help to locate computers with crashes, but just in case, here they are:

Thank you,

Alex.

Was this ever resolved?

I have the exact same issue on some tablets we are piloting. They are HP Elite X2 1011.

I have a case open with Symantec right now. There is no crash logs at C:\ProgramData\Symantec\Symantec Agent\CrashDumps. Not that the path does not exist, but there are no logs being created by this.

I have attached my logs to see if they will help.

I have already walked through installing each agent/plug-in individually with the initial theory that it was one of the installations that was causing the agent to crash. It was not. It happens after a short period of time after all agents are installed.

With just the SMA installed, everything is fine. It is one of the plug-ins that causes this with that dll.

We are running Windows 8.1

Attachment(s)

Agent_4.zip   46 KB 1 version

Hi Donald,

Looks like SMA crash dump was unable to gather dump file. These is a chance that windows error reporting was able to gather something. May you try to run command line:

C:\>dir /s *.hdmp

This cmd should find all dump files gathered by windows (normally they are located in system temp directory). Please find one which timestamp is similar to the time when SMA has crashed and share it?

Which version of product you have in your environment?

Thank you,

Alex.

(an example of running mentioned cmd below):

Yeah, when I check the management service faults report, I see only like 20 a day, and we have 20,000 clients which leads me to believe it is not too large of a concern as I initially thought, and if a reinstall solves it and our technicians haven't been reporting it to me in mass I'll live with it for now. 

We did have a larger issue when we first went to https and implimented CEM where the report was showing 6-7000, but that was resolved with an agent upgrade in one of the hotfix's back with early 7.5 if I remember.

Does not look like anything we didnt know already. Maybe you can pick something else out of it. Here is the WER file for that crash.

We are currently running 7.5 SP1 HF5.

I was recently informed that this is happening on another model as well. What do the 2 of them have in common?...Windows 8.1.

We are not having any issues on any of our other OS's.

Attachment(s)

Report_0.zip   1 KB 1 version

Hi Donald,

Thank you for the web report file. I see several altiris dll running in paths like "C:\Program Files (x86)\Altiris\Altiris Agent\Agents" - that means they are quite old all plugins should be in native %ProgramFiles% path on x64 OS since 7.1.SP2 (or maybe even earlier) - is it true and you have new SMA with old plugins on this OS?

May you export and attach registry key "HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Plugin Objects\Agents" ?

Also, .wer file is just overview of the problem, but to dig  further we need dump file created by windows. Ordinary they are in the same path as report.wer with extensions .mdmp or .hdmp (hdmp is preferable).

Thank you,

Alex.

Alex,

I wish it was as easy as old versions, but I am 100% sure it is not. Our 7.5 environment was not an upgrade. It was a brand new implementation seperate from the clients 7.1 environment. So, there should be no residual 7.1 and below files hanging around. I have attached the registry export as you asked.

Regarding the HDMP, I second-guessed myself and did not send you this one because it was 2 minutes earlier than the event. There were 4 with the same name at C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_AeXNSAgentHostSu_6bca88f4e3a9777cbf73a223d1b8fc8bc765c0_0d2ddacc_cab_0afaa1c7. Only one of them had an HDMP file though. All of them had WER files. The timestamp on the first one I sent you though was the same time as the event in the Event Viewer. Here is the correct HDMP file as well.

Attachment(s)

Reg_and_HDMP.zip   334 KB 1 version

Hi Donald,

Thank you very much for the hdmp file!

Looks like the problem is with "Altiris Software Update Agent": by some reason (i.e. by some bug) 32-bit version of this plugin installed on x64 OS and that causes issues.

Re-install of this plugin should help (as SonicGT has already mentioned). 

Please verify path of PatchManagementAgent after the reinstall - it should point to the "Program Files", not to the "Program Files (x86)".

Thank you,

Alex.

Hi SonicGT,

Please take a look at my reply to Donald - we believe the issue is with incorrect version of Patch plugin on 64-bit OS.

Question: are all "20 crashes/day" specific to Oleaut32.dll or there are also other issues? If there are other crashes and it will be possible for you to share AeXCrashHistory.log files for them it will be good - I wish to check if these failures are known to us.

Thank you,

Alex.

I believe that is it Alex. I looked back at my script and sure enough, it was installing the 32bit plug-in no matter what the OS architecture was. I have all of our agent/plug-ins scripted for our remote location installations. This way, they dont have to leave their machines online for a period of time while they wait for all the agents/plug-ins to be installed through traditional methods.

Oddly enough, we would have never noticed this if it was not for Windows 8.1. I have a Windows 7 machine, sitting next to me, that has the 32bit plugin on a 64bit OS and the SMA is behaving normal. Yet, once we do this on a 8.1 machine, SMA crashes.

I used the following query, in a filter, to find all of our machines that have the plug-in installed incorrectly.

select id._ResourceGuid
from Inv_AeX_AC_Identification id
left join Inv_AeX_AC_Client_Agent ca
	on ca._ResourceGuid = id._ResourceGuid
where [System Type] = 'Win64'
	and [Agent Name] = 'Altiris Software Update Agent'
	and [Install Path] = 'C:\Program Files (x86)\Altiris\Altiris Agent\Agents\PatchMgmtAgent'

I am now able to target all of our "improperly" installed Software Update agents with the Uninstall policy since many of them are over CEM. This should allow them to uninstall and receive the correct install policy after Resource Membership Updates run.

Again, Thank you Alex for your help.

Donald, no prob.

I will pass thanks to the dev who was involved to the research :)

Thank you,

Alex.