Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Seeing a lot of basexmlhttpcallback exceptions

Created: 09 Nov 2010 • Updated: 07 Jul 2011 | 72 comments

I see this one:
 <event date="Nov 09 15:54:09 +00:00" severity="1" hostName="DEVICEMGMT" source="Altiris.TaskManagement.ClientTask.*" module="w3wp.exe" process="w3wp" pid="7032" thread="129" tickCount="606113671"><![CDATA[BaseXmlHttpCallback Exception: System.Exception: An exception occurred while executing the request ---> System.Exception: System.Xml.XmlException: The 'statusEvt' start tag on line 1 does not match the end tag of 'st辮'. Line 1, position 532.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.Throw(String res, String[] args)
   at System.Xml.XmlTextReaderImpl.ThrowTagMismatch(NodeData startTag)
   at System.Xml.XmlTextReaderImpl.ParseEndElement()
   at System.Xml.XmlTextReaderImpl.ParseElementContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at System.Xml.XmlTextReader.Read()
   at Altiris.DotNetLib.Xml.XmlNodeSetReader.Next()
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(String statusXml)
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatus(Hashtable input, Hashtable& output)
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingServer.ProcessRequestObject(Object work)
   --- End of inner exception stack trace ---
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequestI(String service, Hashtable input)
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequest(String service, Hashtable input)
   at Altiris.ClientTask.Server.HttpHandlers.WebClientApiManager.ProcessPostClientStatus(XmlTextWriter wr, NameValueCollection queryValues, Stream requestStream, Int32 contentLength)
   at Altiris.ClientTask.Server.HttpHandlers.PostStatus.WriteResponse(XmlTextWriter wr)
   at Altiris.TaskManagement.Common.XmlHttp.BaseXmlXmlHttpCallback.WriteResponseRaw(XmlTextWriter xwr)
   at Altiris.TaskManagement.Common.XmlHttp.BaseXmlHttpCallback.ProcessRequest(HttpContext context)]]></event>

And this one over and over:

<event date="Nov 09 15:54:09 +00:00" severity="1" hostName="DEVICEMGMT" source="Altiris.TaskManagement.ClientTask.*" module="w3wp.exe" process="w3wp" pid="7032" thread="78" tickCount="606113531"><![CDATA[BaseXmlHttpCallback Exception: System.Exception: An exception occurred while executing the request ---> System.Exception: System.Xml.XmlException: Unable to find node named "statusEvt".  Found node "" instead
   at Altiris.DotNetLib.Xml.XmlTextReaderHelper.MoveToExpectedNode(XmlReader r, String nodeName)
   at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.OnFromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
   at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.FromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(AltirisResourceGuid resourceGuid, String statusType, XmlTextReader r)
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(String statusXml)
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatus(Hashtable input, Hashtable& output)
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingServer.ProcessRequestObject(Object work)
   --- End of inner exception stack trace ---
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequestI(String service, Hashtable input)
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequest(String service, Hashtable input)
   at Altiris.ClientTask.Server.HttpHandlers.WebClientApiManager.ProcessPostClientStatus(XmlTextWriter wr, NameValueCollection queryValues, Stream requestStream, Int32 contentLength)
   at Altiris.ClientTask.Server.HttpHandlers.PostStatus.WriteResponse(XmlTextWriter wr)
   at Altiris.TaskManagement.Common.XmlHttp.BaseXmlXmlHttpCallback.WriteResponseRaw(XmlTextWriter xwr)
   at Altiris.TaskManagement.Common.XmlHttp.BaseXmlHttpCallback.ProcessRequest(HttpContext context)]]></event>

Discussion Filed Under:

Comments 72 CommentsJump to latest comment

Ludovic Ferre's picture

is receiving http requests that do not contain the expected information, in some cases bad characters and in other empty one if I see the logs correctly.

Can you check on the IIS log files what the exact parameters are?

May be post the files here or mail them to me for a quick review.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Ludovic Ferre's picture

I looked at the log files and it doesn't provide any insights, so this points to a problem in the posted data.

The data is getting corrupted somehow so it could comes from one of the various elements in the chain:

  • The agent sending back bad information
  • Network issue causing the posted data to become corruption
  • Compression issue causing the posted data to become corrupt
  • File system / OS issue

I have seen network corruption occur in one of my customers with an ACNS environment in place, with strange char sets being inserted in odd location similar to what you are seeing now.

But I have also seen problems with the agent sending invalid data or with compression going wild...

So we can't rule problems on the agent or server out yet, so the best course of action right now would be to run the Altiris profiler to trace operations and see which IP the "bad" posted data is coming from, so we can see if the problem is rooted at a single managed computers or if the error is wiedly seen (pointing to a likely communication / server / setting issue).

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

camaroguy's picture

All of these clients are out in the wild and the server is in the DMZ.   So my guess would be network....  Thoughts

Ludovic Ferre's picture

This should be a quick way to see some of the problems and qualify whether it is serious or something you can live with ;).

But if you are unfamiliar with the profiler this could be a little tricky to setup. I think I wrote an article on using the profiler a while back... and I am sure you should be able to find some help here on Connect.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

EugeneDisc's picture

Sup Camaroguy.

I am having the same issue at a client.

Did you manage to resolve this issue?

Eugene

Ludovic Ferre's picture

Hi Camaroguy, EugeneFT,

One option that could be worth check is Wireshark or any other network capture tool.

The invalid data is sent to the server via the post method, so it doesn't appear on the IIS log or agent log, so only the network and application / IIS can get access to it.

What would be worth is to check from both a client and the server whether the invalid string are crafted by the agent or made up during transmission (unlikely but not totally unheard off).

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Richard J. Tracy II's picture

I'm recieving the exact same problem. At first I though it was an issue with agent installed incorrectly. I have reloaded Altirs CMS 3 times (tried with old and new databases) and still recieve the "statusEvt" error constantly (about every other second). I think it may have something to do with site servers. Does anyone know how to resolve this?

Ludovic Ferre's picture

Hello Richard,

The problem seem to come from the posted data (data send to NS via the post method from the Client Task Agent) that doesn't contain correctly formatted xml (invalid end tag in the first example, or missing event tag in the second one).

As posted data is not recorded on the IIS logs the only way to see what is sent to the server would be either to run a profiler traceon the NS or to do a wireshark (network) trace.

I would favor the network trace as it will tell us exactly what is sent to the NS and from which computer / ip. Wireshark is very nice to run and you can specify filters to only capture http traffic or only display post method call.

From that you should be able to inspect the xml data sent to the NS and check what uis actually invalid.

Then if you can do the same from one of the faulty agent (if the messages appear to come from only a few agents rather than many random ones) to check whether the outgoing data is correct (indicating a network corruptioon) or not (indicating an agent problem).

I suspect it'll be an agent issue so you may want to start tracing from there.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

noodleNT's picture

I get a ton of these in my log too!

"3/3/2011 1:08:56 PM","BaseXmlHttpCallback Exception: System.Exception: An exception occurred while executing the request ---> System.Exception: System.Xml.XmlException: Unable to find node named ""statusEvt"".  Found node """" instead
   at Altiris.DotNetLib.Xml.XmlTextReaderHelper.MoveToExpectedNode(XmlReader r, String nodeName)
   at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.OnFromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
   at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.FromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(AltirisResourceGuid resourceGuid, String statusType, XmlTextReader r)
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(String statusXml)
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatus(Hashtable input, Hashtable& output)
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingServer.ProcessRequestObject(Object work)
   --- End of inner exception stack trace ---
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequestI(String service, Hashtable input)
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequest(String service, Hashtable input)
   at Altiris.ClientTask.Server.HttpHandlers.WebClientApiManager.ProcessPostClientStatus(XmlTextWriter wr, NameValueCollection queryValues, Stream requestStream, Int32 contentLength)
   at Altiris.ClientTask.Server.HttpHandlers.PostStatus.WriteResponse(XmlTextWriter wr)
   at Altiris.TaskManagement.Common.XmlHttp.BaseXmlXmlHttpCallback.WriteResponseRaw(XmlTextWriter xwr)
   at Altiris.TaskManagement.Common.XmlHttp.BaseXmlHttpCallback.ProcessRequest(HttpContext context)","Altiris.TaskManagement.ClientTask.*","w3wp","24" 

Rhys Paterson's picture

I heard something about it being resolved in 7.1?

We get it logged as an error and I'm pretty sure someone said that it was just classed incorrectly (should be informational or such).

That said, it doesn't resolve the issue. I just hate the way it fills up the Log Viewer errors.

Ludovic Ferre's picture

But the engineering team never managed to reproduce the issue so this was target for 7.1.

However this doesn't mean it's solved in 7.1, just that it's not known to occur yet...

I have a couple of customer with this issue now on various use case, so I'm looking into the details and I'm also trying to repdoruce or cpature interesting data (wireshark, taskmanagement logs).

BTW, defect ID is etrack #1827643.

I'll try to get this in a KB as I did not find anything on Symwise (may be my search term or brain at fault here).

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

noodleNT's picture

Not sure if this helps, but I have removed BESR and Mobile Management Suite from my NS.

Ludovic Ferre's picture

Did that solve the problem?

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

noodleNT's picture

No, I think it caused the problem. I am wondering of anyone else started seeing this issue after removing a solution.

Ludovic Ferre's picture

http://www.symantec.com/docs/TECH47665

Probably not the best answer, but it makes sense.

In short, these problem are seen when other (worse) issues arise on the clients. And fixing the other issues (the real problem) stops the flow of errors on the server.

The KB marks these events to be downgraded (from error to another logging level) albeit right now I find them useful (it's clear from the red logs that something is wrong, so action must be taken to find out what and why).

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Ludovic Ferre's picture

I am on a remote session with a customer having these problems.

We checked the client and server side and one thing we found was 4 xml entries under "%programfiles%\Altiris\Altiris Agent\TaskManagement\status". These xml files should exist only during the excution of a task, until the status is sent to the server.

The files not only happened to be there, but they were all blank (full of blank spaces), despite their size of 2.7KB.

So, we stopped the Altiris Agent, deleted the files and the broken client task agent started to work properly again (it was running in a loop before that time).

No, for information, I traced the http request (get and post) whilst running a taks on a client computer of mine.

Here's an xml string that is sent to the server in the post method, from the client agent:

<status>
    <event resourceGuid="e1d4aaf1-7157-4ead-958b-464ebbf50a61" type="empty">
        <statusEvt type="empty" tig="bc5c45a5-e40e-4e76-9925-590734701c72" pig="bc5c45a5-e40e-4e76-9925-590734701c72" jng="00000000-0000-0000-0000-000000000000" tvg="759f10aa-bdb9-4f39-bb29-2be0d1213ad5">
            <output>
                <param name="@ReturnValue" type="System.Int32">
                    <int>0</int>
                </param>
                <param name="@Success" type="System.Boolean">
                    <boolean>true</boolean>
                </param>
                <param name="PDS" type="System.String">
                    <string>Package already downloaded on 2011-03-14 15:21:32</string>
                </param>
                <param name="TRS" type="System.String">
                    <string> launched successfully</string>
                </param>
            </output>
        </statusEvt>
    </event>
</status>

Which contrast from the string xml sent when the server was throwing the BaseXmlHttpCallback errors:

<status>
    <event resourceGuid="e1d4aaf1-7157-4ead-958b-464ebbf50a61" type="empty">
    </event>
</status>

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

noodleNT's picture

We checked the client and server side and one thing we found was 4 xml entries under "%programfiles%\Altiris\Altiris Agent\TaskManagement\status.

Did you find these files on the clients? I don't see anything on my NS server. The folder is empty.  

Ludovic Ferre's picture

The problem found on the server log are caused by the information posted by the Client Task Server.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

noodleNT's picture

I see an empty file called <GUID>.type.ClientCreated.xml on a few of my clients. I am thinking about creating a task to just find these and remove them from all my machines and then hope these errors go away.

Should I go ahead and do that?

Ludovic Ferre's picture

If you are encountering the same problem as my customer then you could find that sending tasks to the computer will not work.

In the case I worked on the CTA was looping and constantly sending empty status message, thus causing the error on the server.

I hope you are on the lucky side and can do set the agent back to work form Altiris. I'll suggest running a BAT or VBS to stop the agent and delete the status files.

Finally, I can comment on the KB and defect: these error messages look perfectly legitimate and downgrading them seem like a bad idea, like sweeping dust under the carpet. I'll go and create a case to pass data to the Task Server team now.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

noodleNT's picture

I created the following task and ran it against my environment. The amount of errors have reduceded dramatically. Is there a way to hunt down what clients are generating the error?

@ECHO OFF

IF EXIST "C:\Program Files (x86)\Altiris\Altiris Agent\TaskManagement\status" (
   CD "C:\Program Files (x86)\Altiris\Altiris Agent\TaskManagement\status"
   DEL *.XML /Q
)

IF EXIST "C:\Program Files\Altiris\Altiris Agent\TaskManagement\status" (
   CD "C:\Program Files\Altiris\Altiris Agent\TaskManagement\status"
   DEL *.XML /Q
)

Ludovic Ferre's picture

We'll try to see form my customer whether it's possible to run something on the affected clients or if they are <<zombies>> as the client we worked on today (the task agent was looping sending empty status events to the NS).

It's good that you can run this from the CTA without stopping the agent or restarting.

Actually this is something I noted on another case with the CTA, as the agent fails to refresh updated packages (including the SP5 HOWTO36013 version). Our solution there was to clean the TaskManagement cache of all the xml files in the directory (they are only a compress or encrypted - I doubt that, given it's only a subset of the client policy file - yet they have an xml extension).

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Rhys Paterson's picture

%ProgramFiles%\Altiris\Altiris Agent\TaskManagement\cache

When you open them in notepad, it's just a whole bunch of crazy characters.

I'll look at cleaning them up and see what happens.

Ludovic Ferre's picture

The xml files under the cache folder seem to be compressed entries. Their are legitimate and will never go away, as they are the local CTA copy of the policies it handles (Managed SWD etc).

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Rhys Paterson's picture

I will refrain from deleting them.

We get a bunch of the errors too, but I've yet to see a client with bad XML files in the status folder.

Maybe I've just been unlucky so far.

noodleNT's picture

I was working with a Support Tech and he pointed me to this Tech article..

http://www.symantec.com/business/support/index?page=content&id=TECH144662

He told me to run the client cleanup script on all my computers. I had to do this with DS since the task services was having issues on some machines. After running this all my errors went away. However it did add increased load onto my SQL and NS server since inventory tasks started to run again and report in.

Run a Cleanup Script on Clients

Create a new run script task with the following syntax:

---------------------------------------------------------
REM Get the Altiris Agent install path
FOR /F "tokens=2*" %%A IN ('REG.EXE QUERY "HKLM\Software\Altiris\Altiris Agent" /V "installdir"') DO SET AgentDir=%%B
set tempbat=%temp%\AgentClean.bat"

REM Create temporary batch file to execute while the agent restarts
echo "%AgentDir%\aexagentutil" /stop > %tempbat%
echo rmdir "%AgentDir%\TaskManagement" /s /q >> %tempbat%
echo ping localhost -n 30 >> %tempbat%
echo "%AgentDir%\aexagentutil" /start >> %tempbat%
echo exit >> %tempbat%

REM Executes temporary batch file
start "" /MIN %tempbat%
---------------------------------------------------------

Ludovic Ferre's picture

Hello NoodleNT,

The script does a full clean up of the directory, so to me it looks like we are treating the symptoms rather than the root cause.

I'm creating an etrack (defect) today as I received many files from my customer to pass on to engineering, so they can try to prevent the problem from happening.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

noodleNT's picture

I am still seeing these come in but it isn't as bad as before. Has anyone figured out how to pin the error message to the client that is causing it?

Ludovic Ferre's picture

It's called aila (which stands for the Altiris IIS Log Analyzer) and version 0.5.0 (just released) contains code to handle exactly this (as you know, my customers are affected by this problem).

Here are some docs to get started on it:

I'm working on a second sample using the query shell that will address this specific issue.

It should be on Connect by tomorrow, if all goes well ;-).

Here's a sample output that shows the list of ip and guids causing the workload on a server:

Ip cache top 20 entries:
    14763, 192.168.10.51
    10845, 192.168.20.58
    10350, 192.168.2.134
    10162, 192.168.14.154
    10158, 192.168.72.154
    <<data truncated here>>

Guid cache top 20 entries:
    14762, 965c234c-9e45-4d19-8f83-bb9970f401d5
    10845, 62bd92f5-b1bb-41eb-899d-90e08ee2dbcb
    10350, 6a52a043-1594-41d5-bd17-669533ff6e83
    10162, 83cbe1f5-7c2d-4071-976f-fb522f699f23
    10158, 5377e478-49ee-40b3-8c42-e744c5873fc3
    <<data truncated here>>

Enter a command (help or ?) or exit to quit:

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Ludovic Ferre's picture

The sample (or is it a tutorial?) with this problem in mind was made public earlier today.

Here it is:

https://www-secure.symantec.com/connect/articles/c...

As usual, let me know if you have any issues with the tool or if the articles need clarifications etc.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Ludovic Ferre's picture

If the client task agent (the machines found using aila above) is operational you can run the clean up task described earlier:

https://www-secure.symantec.com/connect/forums/see...

If the CTA is out-of-order then you can revert to the AClient/DAgent as per earlier again ;):

https://www-secure.symantec.com/connect/forums/see...

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Zac H's picture

I can't thank you guys enough for this.  My logs are finally clean.  For now...

noodleNT's picture

Mine came back (Worse). Which step did you do to clean yours out?

Is the agent getting in a loop because of multiple schedules on a policy?

Zac H's picture

I was hoping this would go away with 7.1.  It didn't.  I can run the tasks to clean it up, but are we any closer to finding an actual SOLUTION?

noodleNT's picture

Did you do a DB upgrade or new install? I am planning on going new because DMC uninstall really messed up my DB.

Ludovic Ferre's picture

...and confirmed last week that they have a code fix for this specific problem.

The next step will be for Engineering Management to decide how the fix is delivered to our customer.

From what I have seen it is unlikely to make it to a point fix, however it is most likely that it will be included in the upcoming SP5 MR5.

I'll post updates on this thread when I have more information on this.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Ludovic Ferre's picture

The 7.1 fix should be in SP2.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

Zac H's picture

Is there possibly a hotfix or rollup that resolves the issue???

noodleNT's picture

Still seeing this in my 7.0 MR4 environment. I can get it to stop for a bit if I run a job on all computers that stops the agent service, deletes the TaskManagement folder the client and then starts the Symantec Management agent service again.

Ludovic Ferre's picture

Check with your local support team, this is an on-demand rollup much like we did earlier this year (it was Pointfix rollup 18, I can't remember what it applied to - may be pure SP5).

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

noodleNT's picture

Is there a KB article pointing to this? I want to see what actually fixes it. Right now in my environment I have a daily task to delete that TASKMANAGEMENT folder else my agents stop working all together.

noodleNT's picture

Do this step only in DS 6.9 ONLY!

My agents got stuck rebooting over and over and over. I have now moved over all our servers to a new CLEAN DB with the 7.1 SP1 NS. Unfortunatly I will have to manually migrate each software package over and move Recovery solution over to the new environment before I can get my clients switched over. At least I don't have my servers going crazzy and patch managment actually works now.

At this point, I suggest all of you with this issue to create a new 7.1 environment. Don't try doing a migration. My logs look clean on the new environment. I still see these basexml errors in my old 7.0 environment.

rpoag's picture

still having this problem on 7.1 with sp2 installed

Ryan Poag
Northrop Grumman
If your question has been resolved, please be sure to "Mark as Solution"! Thank you.

rpoag's picture

I ran this from the post above:

@ECHO OFF

IF EXIST "C:\Program Files (x86)\Altiris\Altiris Agent\TaskManagement\status" (
   CD "C:\Program Files (x86)\Altiris\Altiris Agent\TaskManagement\status"
   DEL *.XML /Q
)

IF EXIST "C:\Program Files\Altiris\Altiris Agent\TaskManagement\status" (
   CD "C:\Program Files\Altiris\Altiris Agent\TaskManagement\status"
   DEL *.XML /Q
)

and it looks like the error has stopped. It may come back but at least I can do something about it. The logs were being deluged with the error. Unfortunately I ran the packet capture after the script so I could not confirm that there were any files in the directory above. If I get the error again I will confirm whether there are files in the status directory prior to running the job to delete them.

Ryan Poag
Northrop Grumman
If your question has been resolved, please be sure to "Mark as Solution"! Thank you.

mbarbaro's picture

In my environment I had this error, but it wasn't frequently. Now after application of SP2 and the relative upgrade of the agent and plug-in I receive a lot of this error.
I have tried to ran the batch for cleaning the "TaskManagement\status"  directory but I have not seen any improvment.

Ludovic Ferre's picture

Could you paste some error message samples here? I'm currious to see to it is exactly the same problem.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

nande1965's picture

Hello,

I too am seeing hundreds of these errors in my log files since upgrading the NS to 7.1 SP2, appearing at a rate of 3-4 per second.  This NS has about 600 clients and is installed on Windows Server 2008 R2 x64

<event date="Jan 25 15:05:33 +00:00" severity="1" hostName="myServerName" source="Altiris.TaskManagement.ClientTask.*" module="w3wp.exe" process="w3wp" pid="10000" thread="230" tickCount="87106824"><![CDATA[BaseXmlHttpCallback Exception: System.Exception: An exception occurred while executing the request ---> System.Exception: System.Xml.XmlException: Unable to find node named "statusEvt".  Found node "" instead
   at Altiris.DotNetLib.Xml.XmlTextReaderHelper.MoveToExpectedNode(XmlReader r, String nodeName)
   at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.OnFromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
   at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.FromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(AltirisResourceGuid resourceGuid, String statusType, XmlTextReader r)
   at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(String statusXml)
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingServer.ProcessRequestObject(Object work)
   --- End of inner exception stack trace ---
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequestI(String service, Hashtable input)
   at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequest(String service, Hashtable input)
   at Altiris.ClientTask.Server.HttpHandlers.WebClientApiManager.ProcessPostClientStatus(XmlTextWriter wr, NameValueCollection queryValues, Stream requestStream, Int32 contentLength)
   at Altiris.ClientTask.Server.HttpHandlers.PostStatus.WriteResponse(XmlTextWriter wr)
   at Altiris.TaskManagement.Common.XmlHttp.BaseXmlXmlHttpCallback.WriteResponseRaw(XmlTextWriter xwr)
   at Altiris.TaskManagement.Common.XmlHttp.BaseXmlHttpCallback.ProcessRequest(HttpContext context)]]></event>

In addition, I see about a dozen errors in Windows Server Management under Roles - Web Server ....  I am not sure if they are related, but thought I'd include them here since they also come from IIS

Log Name:      System
Source:        Microsoft-Windows-WAS
Date:          1/25/2012 3:59:14 AM
Event ID:      5053
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      myServerNameDescription:
The Windows Process Activation Service received a change notification, but was unable to process it correctly.  The data field contains the error number.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-WAS" Guid="{524B5D04-133C-4A62-8362-64E8EDB9CE40}" EventSourceName="WAS" />
    <EventID Qualifiers="49152">5053</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2012-01-25T06:59:14.000000000Z" />
    <EventRecordID>13800</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>System</Channel>
    <Computer>ARKMAASRV747.ark.xstrata.int</Computer>
    <Security />
  </System>
  <EventData>
    <Binary>0D000780</Binary>
  </EventData>
</Event>

Log Name:      System
Source:        Microsoft-Windows-WAS
Date:          1/25/2012 3:59:14 AM
Event ID:      5189
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      ARKMAASRV747.ark.xstrata.int
Description:
The Windows Process Activation Service failed to generate an application pool config file for application pool '*'. The error type is '0'. To resolve this issue, please ensure that the applicationhost.config file is correct and recommit the last configuration changes made. The data field contains the error number.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-WAS" Guid="{524B5D04-133C-4A62-8362-64E8EDB9CE40}" EventSourceName="WAS" />
    <EventID Qualifiers="49152">5189</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2012-01-25T06:59:14.000000000Z" />
    <EventRecordID>13799</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>System</Channel>
    <Computer>ARKMAASRV747.ark.xstrata.int</Computer>
    <Security />
  </System>
  <EventData>
    <Data Name="AppPoolID">*</Data>
    <Data Name="ErrorType">0</Data>
    <Binary>0D000780</Binary>
  </EventData>
</Event>

rpoag's picture

Did you try deleting any files in %program files%\Altiris\Altiris Agent\TaskManagement\status? That worked for me and the errors haven't come back.

I don't think the other errors are related unless IIS is not functional on the server in question. It is hard to wade through that much noise so if you haven't tried deleting the files I would suggest trying that first to see if you can tone down how much is being reported in the log and then choose another error to work on.

Ryan Poag
Northrop Grumman
If your question has been resolved, please be sure to "Mark as Solution"! Thank you.

noodleNT's picture

Did you do that on the clients? It would be great if the log actually had the IP of the machine that caused the error!

rpoag's picture

I ran a script through DS, we are still runing 6.9, which deleted any files in those directories. I have not had an issue with that error since I did that and I too have upgraded to SP2 since then.

Ryan Poag
Northrop Grumman
If your question has been resolved, please be sure to "Mark as Solution"! Thank you.

davidhilling's picture

or I dunno the GUID, anything.... cause hey guess what I did not get this error until sp2 was install but now im getting tons...yet another problem after sp2 lets make this like number 8.

noodleNT's picture

But then was working with Support on an issue with RMS agents crashing. They had me uninstall PPA and then we had to re-install ITMS. After that I am seeing these alerts again.

Patrick Schmid's picture

I had these errors too, they were logged every 2 seconds.

In my environment it was 1 notebook that did problems after the redirection from a 7.0 to a 7.1 Server.

I determined which computer it was by querying against the SQL DB and checked which client had an old Agent. After that it was easy, just hit "Update" on the client and it worked (the Client did that not for 10 hours, settings are every hour once).

Maybe this helps

Sally5432's picture

we also are having the issue... every 2 seconds error in logs.  CMS SP2

---
Don't forget to mark posts as helpful if they are, and mark answers as solutions.

flo85's picture

Hallo,

a Symantec Engineer give me the follow tip to stop the flood of this message.

1. Open IIS console, right click on the NS application pools and click recycle.
2. Restart the following services: Altiris Client Task Data Loader, Altiris Object Host Service.

Sally5432's picture

This help anyone else?  Either I didn't do it right or it didn't help me.  Putting in a support ticket.

---
Don't forget to mark posts as helpful if they are, and mark answers as solutions.

noodleNT's picture

Restarting the data loader service and Host service were key to get mine to stop after cleaning up the status folder on all my computers. I have not recieved anymore base.xml errors since doing this.

noodleNT's picture

I am using this batch file in DS to find the computers that have the status files queued. This helped me ID the comptuers that needed their agents fixed. The script will exit with an error code with the number of files found. If the agent isn't installed on the client computer you will get a -1 error code. I hope this helps you guys as much as it helped me. I found 3 machines causing all my problems.

@ECHO OFF
SET STATUS=0

@ECHO OFF
REM Get the Altiris Agent install path
FOR /F "tokens=2*" %%A IN ('REG.EXE QUERY "HKLM\Software\Altiris\Altiris Agent" /V "installdir"') DO (SET AgentDir=%%B)

FOR /F %%a in ('DIR /B "%AgentDir%\TaskManagement\status\*" ^|Find /V /C""') do (SET /a STATUS=%%a-1)

IF %STATUS% GTR 0 (
 ECHO Files Queued:%Status%
 EXIT %Status%
) ELSE (
 ECHO Files Queued:%Status%
 EXIT %Status%
)

Sally5432's picture

Anyone have tips for running something similar in 7.x?

If I run as a quick delivery or even schedule the task, it seems to run but times out on machines that are powered off.  

I miss the days of 6.x where I could schedule a task and it would queue until a machine came online.

If I run as a managed software policy, I'm not seeing a good way to see results of the script?

Support pointed me to this, which is less than helpful

http://www.symantec.com/business/support/index?page=content&id=TECH47665

---
Don't forget to mark posts as helpful if they are, and mark answers as solutions.

noodleNT's picture

Untill Symantec comes around and splits their DS agent out of the NS agent in 7.X you will need to keep a DS box around. The number of times this thing has saved my butt is astonishing. The NS agent is no where near as stable as the DS agent. Besides you get online and offline status and queued tasks that you can't get in 7.X. (btw..I have heard that online offline status will never come to the new 7 Action Center console) You should check with your rep to see if you already own a license for DS6.9. It's also super easy to get up and running compaired to NS.

Ludovic Ferre's picture

Our engineering team has done some great work correcting the behaviour of the Task Agent. The problem was fixed in 7.0 but only for a fail case, and as you all probably noticed the coming back of this issue in 7.1 was rather unpleasant (and surprising).

The fix this time around is comprehensive, including task server switching if the current TS fails to respond and of course verification of the return code from the TS (so that we don't try to send some empty xml files for ever and ever and...).

This code fix is in the current 7.5 base but I have asked for an inclusion to a future poinntfix rollup in the SP2 codebase.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

rpoag's picture

That is good news. Thank you for the update.

Ryan Poag
Northrop Grumman
If your question has been resolved, please be sure to "Mark as Solution"! Thank you.

noodleNT's picture

I am finding that I have to run the detect and cleanup job on a daily bassis to keep my NS performing at it's best.

Ludovic Ferre's picture

But don't quote me on this ;-).

I'm providing the infog because I know it's a hot topic for you guys, our beloved customer, but I have to had the usual disclaimer here - anything time line or fix inclusion is tentative... you know the drill.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec