Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.
Endpoint Management Community Blog

{CWoc} Getting Started with an Altiris Specific IIS Log File Analyzer

Created: 06 Dec 2009 • Updated: 08 Dec 2009
Ludovic Ferre's picture
+2 2 Votes
Login to vote

Content:

Introduction

One very interesting repository for historical data found on the Notification Server is often the least epxloited one: the IIS log files contian data on the workload recorded per day generally over the entire server life time (I was on a customer server where this folder was taking 50+ GB!!!).

But how can this data be utilised efficiently, as the log files can be very large (from a few tens of MB to 800MB in the worse case I experienced)? I have worked out a few small tools uploaded now on the Google code site related to my Connect Winter of Code {cwoc} initiative ;). See here and there for more details or to add your input.

Back to top

IIS default log files format (W3C Extended)

IIS records requests information on log files (by default under '%windir%\System32\LogFiles\<<sitename>>') and these file can contains a lot of information that pertains to Altiris infrastructure and that is not necessarily analysed in standard log file analyser.

Here is an IIS log file header, with sample data right under it:

#Software: Microsoft Internet Information Services 6.0
#Version: 1.0
#Date: 2009-11-24 00:00:06
#Fields: date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status 

The header defines the various white space seperated columns in the log files. Here's a brief overview of each one of them with special notes on the entries that interest us specially for Altiris servers:

date The date on which the activity occurred.
time The time, in coordinated universal time (UTC), at which the activity occurred.
s-sitename The Internet service name and instance number that was running on the client.
s-ip The IP address of the server on which the log file entry was generated.
cs-method The requested action, for example, a GET method.
cs-uri-stem The target of the action, for example, Default.htm.
cs-uri-query The query, if any, that the client was trying to perform. A Universal Resource Identifier (URI) query is necessary only for dynamic pages.
[This can contain information from the workstation such as IP Adresses that would not be visible in an environment with proxy or content delivery tools.]
s-port The server port number that is configured for the service.
cs-username The name of the authenticated user who accessed your server. Anonymous users are indicated by a hyphen.
c-ip The IP address of the client that made the request.
cs(User-Agent) The browser type that the client used.
sc-status The HTTP status code. [This can indicate error but also return 200 (success) even if the Altiris NS returns an error to the agent]
sc-substatus The substatus error code.
sc-win32-status The Windows status code. [This is good to know if the request fails because of Windows errors.]

A fully detailed list is available on Microsoft technet.

Here is a selected extract from my main test Notification Server with selected highlights [width limited manually to 100 char and extra blank lines are here to ensure highlights are visible]:

2009-11-24 12:21:00 W3SVC1 169.254.29.197 GET /Altiris/NS/Agent/GetClientPolicies.aspx xml=%3CReques
t%20configVersion=%222%22%3E%3CWrkstaGuid%3E%7B8865CA6F-44FC-428C-86CE-4553529EBAC7%7D%3C%2FWrkstaGu
id%3E%3C%2FRequest%3E&compress=1&hash=5662CD961C576286BCEDEF16F9C0279E 80 - 169.254.29.197 -
 200 0 0

2009-11-24 12:21:00 W3SVC1 169.254.29.197 POST /Altiris/InventoryRuleManagement/Agent/InvRuleWebServ
ice.asmx - 80 - 169.254.29.197 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+1.
1.4322.2443) 200 0 0

2009-11-24 12:21:00 W3SVC1 127.0.0.1 GET /Altiris/Console/ItemServices.aspx action=RenameItem&It
emGuid=f4d7a138-dd89-4e2c-93e0-b3678c691241&Name=E1856325_AeXTSAt_6.1.1034_3 80 VBOX-ATRS1\Admin
istrator 127.0.0.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.2;+SV1;+.NET+CLR+1.1.4322;+.NET+C
LR+2.0.50727) 200 0 0

2009-11-24 12:21:16 W3SVC1 169.254.29.197 HEAD /ALTIRIS/NS/Agent/PostEvent.asp - 80 - 169.254.29.193
 - 200 0 0

2009-11-24 12:21:16 W3SVC1 127.0.0.1 GET /Altiris/Console/ItemPage.aspx ItemGuid=ca6ad90b-5620-4e81-
8373-fda9e52d3490&TreeGuid=e7d59624-359c-44c5-be2a-995ee4903800 80 VBOX-ATRS1\Administrator 127.
0.0.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.2;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727)
 302 0 0

2009-11-24 12:21:20 W3SVC1 169.254.29.197 GET /ALTIRIS/NS/Agent/GetPackageInfo.aspx xml=%3Crequest%2
0resource=%22%7B1F744022-BA98-4B66-B0AF-8D4B82718A01%7D%22%20version=%221%22%20type=%22packageServer
s%22%20compress=%221%22%20totalTime=%220%22%3E%0A%3Cpackages%3E%0A%09%3Cpackage%20guid=%22%7B9E796F6
6-B64C-4D67-A912-5FA24E2FAD85%7D%22%2F%3E%0A%3C%2Fpackages%3E%0A%3Caddresses%3E%0A%09%3Caddress%20ip
=%22169.254.29.193%22%2F%3E%0A%3C%2Faddresses%3E%0A%3C%2Frequest%3E%0A 80 - 169.254.29.193 - 200 0 0

2009-11-24 12:21:23 W3SVC1 169.254.29.197 GET /altiris/ns/agent/getpackagesnapshot.aspx PackageId=%7
B9E796F66-B64C-4D67-A912-5FA24E2FAD85%7D&compress=1 80 - 169.254.29.193 - 200 0 0

2009-11-24 12:21:26 W3SVC1 169.254.29.197 GET /Altiris/NS/NSCap/Bin/Win32/X86/ETrack_1856325_Pkg1/Te
st1.vbs - 80 - 169.254.29.193 - 200 0 0

2009-11-24 12:21:28 W3SVC1 169.254.29.197 POST /ALTIRIS/NS/Agent/PostEvent.asp - 80 - 169.254.29.193
 - 200 0 0

The highlighted sections here are mostly contained within the cs-uri-query field. What we can find in there is related to the Altiris Agent or the current operation. Here's a list of information we can retrieve on this field to analyse data or problems:

  • Requesting agent guid
  • Requesting agent ip (not necessarily the c-ip
  • Requested package guid
  • Requested NSCap files

This is the extra data that we want to report on and analyse in conjunction with the other request related data.

Back to top

Design goals

Here are a few goals with regards to implementing a program to help troubleshoot issues or analyse workload and trend over time:

  1. Simplicity (1): the program should be able to run on a NS without installing new software on it
  2. Portablity: the program could be run on Windows and possibly on Linux
  3. Efficiency: the program needs to run efficiently due to the potentially high workload
  4. Scaleability: the program should be able to scrape an entire IIS log files directory
  5. Simplicity (2): it should be easy to run, with enough options to get valuable information out quickly
  6. Extensibility (1): the tools should allow to get output data (directly or indirectly) in a database server
  7. Extensibility (2): the tools may work with a graphical user interface

Back to top

Design overview

DRAFT ! So where do we start? Given the above (1) and (2) point we have only two option: C or C#. Which is just as well. Now I don't think there's any reqiurements to decide on a specific language right now, albeit I can see that c# is most open as it's a simpler language to get started with than c, however performance wise c should be higher (at the cost of complexity during the development phase).

Now on the design itself, the program needn't be complex with multi-threading, fancy status reports, xml or any other cluter. A command line tool taking a single or no command line arguments (but with a few options) and outputing results to the console should suffice at the beginning. You may consider this does not meet design goals (6) and (7) but theses could be catered by specifying a cusotm output format (to pipe the data out to a stream handler on the gui or to a files for loading the data into SQL).

The following command line help message should help define what the tool could do:

 

Log File Analyser help message. To run <<'prog_name>> use the following command line options:
	"/?" or "--help" or "/?" to show this help message

	Input control parameters:
		"/d:" or "--in-dir" or "-s" + "<<path_to_directory>>" to specify a directory that should be parsed
		"/f:" + file_name or "--in-file" or "-f" + "<<path_to_file>>" to specifcy a file that should be parse

	Output control parameters:
		"/c" or "--output-to-csv" or "-csv" to format the output using csv 
		"/s" or "--output-to-sql" or "-sql" to format the output using SQL insert statements

	In-memory analysis:
		"/a" or "--analyse-simple" or "-simple" to run a rough analysis in-memory
		"/b" or "--analyse-advanced" to run a thorough analysis in-memory

	Control parameters:
		"/v" or "--verbose" or "-v" to output extra control information to the console
		"/t" or "--top-

Now in term of what the program should do (and how it should flow) could or should remains rather simple:

flowchart.png
 

Back to top

Getting started with implementation

 This is another story to be told if I may say ;). I will get started with both a C and C# implementation more or less following the flow chart aboce (starting with a simple implementation of the command line argument handling, moving on to a single file process (with a minor payload such as counting byes per line and summing them) and then it'll be down to the real implementation in order to create a specific output line from the given input line.

Back to top

Your input to {CWoC}

 As always you are most welcome to participate. I am more than willing to add people to the commiters list on Google code so you can contribute. Additionally there is a code review function on the view source pages tha tmakes it very easy to comment on a specific line of code of processing and vote up or down changes.

So go try it! [Google code site is https://code.google.com/p/altiris-ns-tooling/]

Back to top

Conclusion

Cwoc is starting. If you want to follow I suggest you add this featured search on your favorites:

     Connect Search :: { CWoC }

Alternatively you can contribute articles, forum posts or blog entries and tag them with {CWoC}!

Back to top