Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

aila2: A c# program to analyze Altiris IIS log files

Created: 13 Jan 2014 • Updated: 06 Mar 2014
Ludovic Ferre's picture
+1 1 Vote
Login to vote

Table of Content:

Introduction:

I started writing the first installement of aila back here on Connect in the winter of 2009. I had both Linux and Windows build for a while but when I implemented complex data structures the windows build broke and I switch to support Windows via Cygwin.

This made it difficult to install and run on servers, so I built-up a web-service to receive and handle IIS log files and show the results in a web-page using google charts. This worked well and I had a growing number of users. However interesting biulding and maintaining the web-service was a recent customer need showed me that it was time to offer a similar service in premise.

And this is why I have writen aila2, a c#, build from scratch IIS Log analyzer.

Top

Data processing:

aila2 starts by resolving the provided input (log file or stdin) to a schema. The IIS schemas are defined using commented line (#) and starting with the "Fields: " (case insensitive) string. Once a schema is found the tool will pasrse of the non commented line and update statistics according to the line content. This will continue until we reach the end of file or until we found another schema

aila2 currently collected the following data during its execution:

File based information:

  • file name
  • file md5 hash
  • linecount

Log entry based statistics:

  • Hourly hits count (Columns: Hit #, PostEvent, GetClientPolicy, GetPackageInfo, Task Management, Inventory Rule Management
    • 0
    • 1
    • 2
    • ...
    • 23
  • Mime type hit count (Columns: Hit #, Sum(time-taken), Max(time-taken), Avg(time-taken))
    • htm
    • html
    • asp
    • aspx
    • asmx
    • ascx
    • axh
    • ashx
    • xml
    • css
    • js
    • gif
    • png
    • jpg
    • other
  • Web applications (Columns: Hit# count, Sum(time-taken), Max(time-taken), Avg(time-taken))
    • /Altiris/NS/Agent/
    • /Altiris/NS/NSCap/
    • /Altiris/NS/
    • /Altiris/Resource/
    • /Altiris/InventoryRuleManagement/
    • /Altiris/PackageShare/
    • /Altiris/SoftwarePortal/
    • /Altiris/ClientTaskAgent/
    • /Altiris/ClientTaskServer/
    • /Altiris/TaskManagement/
    • /Altiris/Console/
    • /Altiris/ActivityCenter/
    • /Altiris/Workflow/
    • /Altiris
    • Other
  • Agent interface hit counts (Columns: Hit# count, Sum(time-taken), Max(time-taken), Avg(time-taken))
    • PostEvent.asp
    • PostEvent.aspx
    • GetPackageInfo.aspx
    • GetClientPolicies.aspx
    • GetPackageSnapshot.aspx
    • CreateResource.aspx
    • GetLicense.asmxGetLicenseDetails.aspxOther
  • Http status (Columns: Hit #)
    • 1xx/2xx Success
    • 3xx Redirected
    • 4xx Client error
    • 5xx Server error
  • Task Interface (Columns: Hit# count, Sum(time-taken), Max(time-taken), Avg(time-taken))
    • ExecSqlCommand
    • ReportTaskData
    • GetTaskServer
    • PersistentSetting
    • GetTaskVersion
    • RefreshTaskServer
  • Inventory Rule parameters (Columns: Hit# count, Sum(time-taken), Max(time-taken), Avg(time-taken))
    • Data Class Hash
    • Data Class Rule Summary
    • Rules
    • Data Class Rules
  • Ip hit - top 20 (Columns: IP address, Hit #)

The output generated is a java script object with the following outline:

{
	"file" : "<file name>",
	"hash" : "<file md5 hash>",
	"linecount" : <count>,
	"stats" : {
		"hourly" : [
			"Hour", 
			"Total hit #",
			"Post Event",
			"Get Client Policy",
			"Get Package Info"
		]
		"mime_type" : [
			"Mime type",
			"Hit #"
		]
		"web_application" : [
			"Web-application",
			"Hit #",
			"Sum(time-taken)",
			"Max(time-taken)",
			"Avg(time-taken)"
		]
		"http_status" : [
			"Http-status", "Hit #"
		],
		"agent_interface" : [
			"Agent interface",
			"Hit #", "Sum(time-taken)",
			"Max(time-taken)",
			"Avg(time-taken)"
		]
		"task_interface" : [
		 	"Task interface"
			"Hit #", "Sum(time-taken)",
			"Max(time-taken)",
			"Avg(time-taken)"
		],
		"invrule_interface" : [
			"IRM AgentRuleData.ashx"
			"Hit #", "Sum(time-taken)",
			"Max(time-taken)",
			"Avg(time-taken)"
		],
		"ip_hit_top" : [
			"Hit count", "IP Address"
		],
	}
}

Top

Usage:

Usage : aila2 [Parameter] [Option(s)]

Parameters:
    -h, --help              Show this help message
    -v, --version           Output program version only

    -f, --file <file path>  The IIS log file to parse

    --stdin                 The log file data will come from the console input
                            instead of a file.

Options:
    -l, --log-level <lvl>   Output log data <= to  to stdout:
            --log-level  1 -> error
            --log-level  2 -> warning
            --log-level  4 -> information
            --log-level  8 -> verbose
            --log-level 16 -> debug
    -o, --out-path <path>   The location where the result file will be created.

Samples:
    aila2 -f iis.log
    aila2 --file iis.log -l 4
    aila2 -f iis.log -o c:\inetpub\wwwroot\aila2\

{CWoc} info: http://www.symantec.com/connect/search/apachesolr_...

Top

Usage notes:

If no command line arguments are provided the program will expect to receive data from the console (stdin). If no output path is provided the output data will be writen to the console.

When processing a file aila2 prints out a counter of currently processed line, and upon completion a processed line count and time taken (in millisecond).

Top

Output:

Here is the output from running the following command on a test server (so the data will not be very imp[rpessive indeed):

{
	"file" : "u_ex131126.log",
	"hash" : "1cadf0dbcb17988b5c88626e7b1cdf2f",
	"linecount" : 12095,
	"stats" : {
		"hourly" : [
			["Hour", "Total hit #", "Post Event", "Get Client Policy", "Get Package Info"],
			["0", 388, 0, 0, 1], 
			["1", 338, 0, 0, 1], 
			["2", 460, 2, 0, 1], 
			["3", 457, 1, 0, 1], 
			["4", 385, 0, 0, 1], 
			["5", 359, 0, 0, 1], 
			["6", 363, 0, 0, 1], 
			["7", 366, 1, 0, 1], 
			["8", 435, 0, 0, 1], 
			["9", 458, 0, 0, 1], 
			["10", 455, 0, 0, 0], 
			["11", 449, 1, 0, 1], 
			["12", 456, 0, 0, 1], 
			["13", 493, 0, 0, 1], 
			["14", 2202, 0, 0, 1], 
			["15", 656, 1, 0, 1], 
			["16", 630, 0, 0, 1], 
			["17", 643, 0, 0, 1], 
			["18", 509, 0, 0, 1], 
			["19", 564, 1, 0, 1], 
			["20", 461, 0, 0, 1], 
			["21", 199, 0, 0, 1], 
			["22", 184, 0, 0, 1], 
			["23", 185, 1, 0, 1]
		], 
		"mime_type" : [
			["Mime type", "Hit #"],
			["htm", 0], 
			["html", 22], 
			["asp", 0], 
			["aspx", 5425], 
			["asmx", 5065], 
			["ascx", 0], 
			["axd", 416], 
			["ashx", 0], 
			["xml", 0], 
			["css", 60], 
			["js", 154], 
			["gif", 691], 
			["png", 230], 
			["jpg", 8], 
			["Other", 24]
		], 
		"web_application" : [
			["Web-application", "Hit #", "Sum(time-taken)", "Max(time-taken)", "Avg(time-taken)"], 
			["/altiris/ns/agent/", 31, 363308, 44343, 11719.61], 
			["/altiris/taskmanagement/", 81, 11378, 4250, 140.4691], 
			["/altiris/inventoryrulemanagement/", 0, 0, 0, 0], 
			["/altiris/ns/nscap/", 0, 0, 0, 0], 
			["/altiris/ns/", 1112, 924678, 193530, 831.545], 
			["/altiris/packageshare/", 0, 0, 0, 0], 
			["/altiris/swportal/", 0, 0, 0, 0], 
			["/altiris/clienttaskagent/", 0, 0, 0, 0], 
			["/altiris/clienttaskserver/", 48, 167013, 21968, 3479.438], 
			["/altiris/console/", 5633, 674149, 32289, 119.6785], 
			["/altiris/activitycenter/", 0, 0, 0, 0], 
			["/altiris/resource/", 0, 0, 0, 0], 
			["/altiris/workflow/", 1446, 165549, 47281, 114.4875], 
			["/altiris", 833, 120750, 16443, 144.958], 
			["other", 2911, 93745, 28406, 32.20371] 
		],
		"agent_interface" : [
			["Agent interface", "Hit #", "Sum(time-taken)", "Max(time-taken)", "Avg(time-taken)"], 
			["postevent.asp", 0, 0, 0, 0],
			["postevent.aspx", 8, 11372, 2531, 1421.5],
			["getpackageinfo.aspx", 0, 0, 0, 0],
			["getclientpolicies.aspx", 23, 351936, 44343, 15301.57],
			["getpackagesnapshot.aspx", 0, 0, 0, 0],
			["createresource.aspx", 0, 0, 0, 0],
			["getlicense.asmx", 0, 0, 0, 0],
			["getlicensedetails.aspx", 0, 0, 0, 0],
			["other", 0, 0, 0, 0]
		]
	}
}

Top

Conclusion:

You can now parse individual log file in their row format or filtered using aila2-filter [1]. Next we will see how to view the JSON data in a graphical format (similar to what was available on aila-web), how to create output file automatically for all IIS logs found in a given directory, and how to generate a calendar view webinterface for easy consumption of the generated data.

So this is quite a program we have here for Connect :D.

Top

References:

[1] aila2-filter Connect download

Top

Document history:

Version 1

Releasing version 1 of the tool (based on Github tag Version-1.2).

File details:

  • File name: aila2-version1.zip
  • File sha256 sum: d53034db3d0ed59ee4c5126d31b88685ca748da848fe16017e1f91bd0b5a8717

File content:

File name Description Documentation
aila2.exe The main aila2 program with the usage explained here. This download page.
aila2.html Part of the aila2-web Calendar user interface. It displays individual result file data usgni the Google Chart WAPI "aila2-web Calendar" - upcoming
aila2.js A javascript file with functions used in both aila2-web interfaces. n/a
index.html Part of the aila2-web Calendar user interface. This is the landing page that shows all or part of the result files in a reverse calendar (from most recent to oldest). "aila2-web Calendar" - upcoming
quickview.html A standlone viewer to use for quick checks on filtered IIS log files or to get up and running quickly with aila2 capabilities. "aila2-web Quickview" - upcoming
style.css A cascading style sheet containing styles used in aila2-web interfaces. n/a

Version 2:

Releasing version 2 of the tool. Updating documentation accordingly.

File details:

  • File name: aila2-version2.zip
  • File sha256 sum: 674fcbcf9b4c95d8acb96606a19958ab0ff6ffae3936792edf9e438fe77e4a5d

File content is unchanged.

 Top