{CWoC 2010} aila 0.5.0 Release and Documentation
This year Connect Winter of Code {CWoC} extended well into spring (and possibly will go on to the summer, but that remain to be seen) but the wait was well worth it, so I hope.
This is the main stub for the release of the "Altiris IIS Log Analyzer" (aka aila), and here is the list of documents made available:
- aila 0.5.0 Release notes
- aila 0.5.0 User manual
- Using Cygwin and aila Together on a Windows System
- Building aila from source on Cygwin
- aila 0.5.0 Samples: single parse analysis
- aila 0.5.0 Samples: using the query shell, part I
- aila 0.5.0 Samples: using the query shell, part II
Now, we'll look here at extract of each documents so if you don't want to read the formal documentation and get a head start, you you stay on this page.
Most noticeable changes since aila-0.4.2:
[full release notes / change list]
- Added query-shell "top <n>", "from <n>" global filters to limit the exec command to show / account for n entries only or to start filtering from position <n>
- Added "not" keyword to reverse the filtered operations (display every thing but the matching entries)
- Added a number of filter implementation to the query shell (uri-agent, time-taken, uri-webapp, guid)
- Added support for httperr log files
- Added cygwin support to build aila on Windows
- Implemented hierarchical shell, to support the default shell, print, filter and query sub-shells
- Switched project to 'make' for compiling aila
- Made numerous infrastructure changes paving the way to the query-shell
- Discontinued Windows native support (builds)
- Added interacitve shell functions to print cache data
- Refactored the string store to use a binary tree to store guids and ip addresses in order
- Added support to the ClientTaskServer (CTS) uri-webapp and guid search in the uri-query
- Improved the dynamic log file schema handling
Installing cygwin, compiling and running aila:
- Download and installCygwin (setup.exe). Run the installer and make sure you add the following packages:
- Gnu C compiler (gcc)
- Make
- Subversion (optional)
- Get the sources
- tar.gz delivery [Symantec Connect or Google code]
- or Synchronize from subversion
- Compile, install and run (command lines here, from the directory were you uncompress the tar.gz or your svn repository:
- make
- make install
- aila
This is it. The last command line will show you the standard help message, which includes a build time and platform as well as the command line options needed or optional (bold formatting is added here on Connect - also note that the platform should match cygwin on a cygwin environment):
ludovic@smallisbeautiful:~/PROD_003/SW-SRC/altiris-ns-tooling/aila$ aila
aila (Altiris IIS Log Analyser) is at version 0.5.0
Built for linux-amd64 on Apr 19 2011 at 15:16:00, brought to you by {CWoC}.
Usage : aila [Parameter] [Option(s)]
Parameters:
-h, --help to show this help message
-f, --file <path_to_file>
Options:
-c, --csv-format Format output using tab seperated values
-l, --log-level <lvl> Output log data <= to <lvl> to stderr:
--log-level 1 -> error
--log-level 2 -> warning
--log-level 4 -> information
--log-level 8 -> verbose
--log-level 16 -> debug
-n0, --no-zero Show results including 0 counts
-ndc, --no-dump-cache Do no writes the string cache content to file
-nt, --no-topper Do not output the top 20 entries from caches
-qs, --query-shell Open an interactive shell after parsing the
provided log file. Type 'help' on the shell
for details.
-S, --summary Parse file for summary review only
-V, --version Output program version only
Samples:
aila --file iis.log --no-zero --log-level 8
aila -f iis.log -l 4 -n0
aila -f iis.log --query-shell -n0
Sample 1: Altiris 6.0 IIS log file
Let's start with a basic analysis on a somehow long file (~1.3 million lines), broken down with an explanation of each sections and on what could be remarkable about the log file (i.e. is there anything worthy from a troubleshooting or operation standapoint):
ludovic@smallisbeautiful:~/PROD_003/SW-SRC/altiris-ns-tooling/aila/payload$ aila -f iis_6_logs/ex101202.log Current line #: 1290000 Program read 206035226 bytes from 1292010 lines
This section shows the invocation of aila from a linux shell (but it would be all the same from a cygwin shell), with only the "-f <file path>" command line switch. The file is 206,035,226 bytes long, aka 197MiB for 1,292,010 lines.
The current line section is a progress counters that is updated every 10,000 entries so you can see progress on the parsing...
Sample 2: Altiris 7.1 IIS log file
This second sample builds on experience acquired from the above execution, applied on a SMP 7.1 log files, which also is broken down to review the difference and additional outcome from the analysis:
ludovic@smallisbeautiful:~/PROD_003/SW-SRC/altiris-ns-tooling/aila/payload$ aila -f iis_7_logs/u_ex110309.log -n0 Current line #: 10000 Program read 3528377 bytes from 11955 lines
In this case you can see form the file name that it is not an IIS 6.0 log file. Also from the generic stats you can gather that this is not from a production system but rather from a test system, just after the 7.1 release (or was it just before?)...
Sample 3: IIS httperr log file
Whenever a http request fails outside of the w3wp.exe process the information is logged in the httperr directory (under '%windir%\system32\logfiles\httperr'). These entries are not handled properly by the worker process so it's worth looking at them in a number of cases (for example when you are seeing a lot of back-off message on agent or package servers).
ludovic@smallisbeautiful:~/PROD_003/SW-SRC/altiris-ns-tooling/aila/payload$ aila -f httperr/httperr10697.log -n0 -nt Program read 1048420 bytes from 5636 lines
This error log file is parsed with the -nt option, so it will no display the top 20 ip and guid entries (although they could be useful, pointing out to a single ip or computer guid)...
Using the query shell:
To use the query-shell you need to specify the '--query-shell' command or its shorthand '-qs' when invoking aila from the shell.
The query-shell is build around 4 run-time environment: the default shell (or root) and sub-shells that have specialised control. You can navigate from the default to the specilaised shell invoking the shell name to enter the shell, and quit to return to root, as illustrated here:
aila-qs/:help Currently supported commands are divided in 3 sub-sections: print: Enters the print sub-shell filter: Enters the filter sub-shell exec: Enters the execution sub-shell <sub-shell> <sub-shell command>: Execute the command within the sub- shell context. Samples: aila/:print top 10 ip aila/:filter c-ip 169.254.29.5 aila/:exec stat store aila/:filter uri-webapp 2 aila/:exec stat store log level <n>: set the current logging level to <n> exit: Leave the query shell and terminate aila. help, ?: Print this message or contextual shell help to stdout.
Each of the shell contains contextual help information, but you can also invoke sub-shell commands from the root, by adding the sub-shell command to the sub-shell names. This is easier done than explain, as you can see here:
aila-qs/: aila-qs/:filter c-ip 127.0.0.1 aila-qs/:filter top 20 aila-qs/:exec print store 162753, 127.0.0.1, 257, /Altiris/, (null), Other, 20101202, 60050, Client error (4xx), -2146893042, 1359, 162754, 127.0.0.1, 247, /Altiris/, (null), Other, 20101202, 60050, Client error (4xx), 0, 171,
Here are the exec commands you want to use and remember:
- 'stat store' which will generate statistics for the in-memory store with the defined filters applied and print the output to the console
- 'print store' which, as shown below, will print the information from the store to the console.
Here are the exec commands you want to use and remember:
- 'print top <n> ip' to print the first n ip address on the ip cache, sorted by hit count descending
- 'print top <n> guid' to print the first n guid on the guid cache, sorted by hit count descending
And here are the filter command you want to know:
- 'filter help' to get the generic filter help (it contains all the supported fields)
- 'filter help <supported fileds>' will show the help message for the supported fields, listed on the shell help.
- 'filter print' will write the current filter to the console. Here are a few samples:
- 'filter clear' will reset the filter to null (no filter)
Here are some samples on the filter shell:
aila-qs/:filter print top 20 uri-webapp 8 aila-qs/:filter from 25000 aila-qs/:filter print top 20 from 25000 uri-webapp 8 aila-qs/:filter not aila-qs/:filter print top 20 from 25000 uri-webapp not 8 aila-qs/:filter top 0 aila-qs/:filter print from 25000 uri-webapp not 8 aila-qs/:filter c-ip 127.0.0.1 aila-qs/:filter print from 25000 c-ip not 127.0.0.1 aila-qs/:filter from 0 aila-qs/:filter print c-ip not 127.0.0.1 aila-qs/:filter not aila-qs/:filter print c-ip 127.0.0.1 aila-qs/:filter not win32-status 0 aila-qs/:filter print win32-status not 0 aila-qs/:
Finally, you can use an input file to automate the processing and write the result to a file, for example if you want to output a large count of ip address from the store.
Here's an extract from the automated test input file I have created, which is on the subversion repository in Google code:
echo #################_Automated_test_file_by_ludovic_ferre_################# echo ### Test the aila shell (root) echo ### help help echo ### ? ? echo ### log level 32 log level 32 echo echo ### log level 2 log level 2 echo echo ###############_print_shell_test_suit_->_starting_from_root_############## echo ### print help print help echo ### print top 10 guid print top 10 guid echo ### print top 10 ip print top 10 ip echo ### print get ip 127.0.0.1 print get ip 127.0.0.1 echo ### print get guid 7B3D9240-6943-4776-A010-E86B192C5C9A print get guid 7B3D9240-6943-4776-A010-E86B192C5C9A echo ##############_print_shell_test_suit_->_starting_on_subshell_############# echo ### print print echo echo ###### help help echo ###### top 10 guid top 10 guid echo ###### top 10 ip top 10 ip
Using this input file with aila and output redirection allowed me to ensure all query-shell commands work as they should. Here's a sample command using the 3GiB payload:
ludovic@smallisbeautiful:~/PROD_003/SW-SRC/altiris-ns-tooling/aila$ aila -f huge_payload -n0 -ndc -qs < test_automation_answer_file > test_outfile
And here's some of the output (which in effect is 2,530 lines long):
Program read 3145645241 bytes from 17541689 lines LOG FILE ANALYSIS:: SUMMARY Mime type results: File type= htm , page hits= 64869 File type= html , page hits= 92 File type= js , page hits= 61454 File type= css , page hits= 60432 File type= asp , page hits= 10430112 File type= aspx , page hits= 5398716 File type= asmx , page hits= 228407 File type= ascx , page hits= 2 File type= xml , page hits= 2280 File type= Other, page hits= 1294990 Altiris Agent requests results: Agent request= Reg Client , page hits= 766 Agent request= Get Policies, page hits= 559689 Agent request= Get Pkg Info, page hits= 1687891 Agent request= Get Snapshot, page hits= 8524 Agent request= Post Event , page hits= 10599110 IIS Web-applications results: Webapp= /Altiris/NS/Agent/, dir hits = 12856658 Webapp= /Altiris/NS/NSCap/, dir hits = 11859 Webapp= /Altiris/NS/ , dir hits = 349479 Webapp= /Altiris/Resource/, dir hits = 654 Webapp= /Altiris/IRA[1]/ , dir hits = 351593 Webapp= /Altiris/Packages/, dir hits = 548965 Webapp= /Altiris/SWPortal/, dir hits = 181787 Webapp= /Altiris/CTA[3]/ , dir hits = 5687 Webapp= /Altiris/CTS[4]/ , dir hits = 2522835 Webapp= /Altiris/TaskMgmt/, dir hits = 78910 Webapp= /Altiris/ , dir hits = 393622 Webapp= Other , dir hits = 239305
The reader will probably have noticed that I am using an undocumented feature: the echo command, which like echo in dos or on gnu/linux. This is useful with an input file as the console normally echoes information that is not output by the program, which made reading the output really difficult.
Finally, I'll restate here that this is an open project, hosted on Google Code and that your input and feedback is valued and any kind of contribution is welcome as well ^_^.