I went a little silent this year on the work I am doing in this season Winter of Code, but I am not giving up far from it.
Today I bumped the current build version of aila from 0.3.6 to 0.3.9 which I am working on.
0.3.7 introduced data storage of the log file requests. This means the parsed data is not lost once we read the next line but kept for later retrieval in memory. The data structure was crafted (last year) to have minimal memory impact last year, so we can load clause to a million lines on just about 50MiB.
0.3.8 introduced the mechanism needed to safely retrieve the data from the in-memory store and a dump function to output the data in a text format to the console. This way I could test that data was stored and retrieve without any issue on files ranging from 10 to 960,000 lines (real payload from my customer).
0.3.9 is fresh and still untouched, as this is where I'll implement post parsing analysis. This could include counting entries matching a specific ip or guid, or entries for a specific hour. I'm wondering weather I should include data sorting or other interesting algorythms.
This is where you, my dear reader and user is welcome to comment and post ideas. :D
Nice evening everyone!