Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.
Cyber Security Services

Enhancing Apache Logging For Improved Forensic Capability Part II: Implementing Enhanced Apache Logging

Created: 04 Aug 2014 • Updated: 26 Nov 2014
Vince Kornacki's picture
+5 5 Votes
Login to vote

apache2.png

In the previous installment we examined default Apache logging. Now let's pump up the default Apache combined log format in order to supercharge forensic capability! We'll utilize the mod_log_config "LogFormat" directive in order to define the "enhanced" log format within the /etc/apache2/apache2.conf configuration file:

LogFormat "%{%a %m/%d/%Y @ %I:%M:%S.}t%{msec_frac}t %{%p %Z}t %h (%{X-Forwarded-For}i) > %v:%p \"%r\" %I %D %>s %b %k %L \"%{Referer}i\" \"%{User-Agent}i\" %u %{User}C %{SessionTracker}C" enhanced

Note that mod_log_config is statically compiled into Apache. A sample Apache enhanced log format entry looks a little something like this:

Wed 07/30/2014 @ 10:45:59.420 PM CDT 10.1.1.101 (-) > 192.168.1.1:80  "GET /example.html HTTP/1.1" 222 444 200 666 0 - "http://192.168.1.1/from.html" "Mozilla/5.0 (X11; Linux x86_64; rv:33.0) Gecko/20100101 Firefox/33.0" - jdoe 0bcdd30af79b6aca8c8f7808c02ab530ac4c4a75

Jeepers Creepers! That's even more gibberish than last time, so let's break down the enhanced log format step by step:

  • The "%{%a %m/%d/%Y @ %I:%M:%S.}t%{msec_frac}t %{%p %Z}t" component logs the time in a more intuitive format using strftime conversion specifications and embedded Apache tokens. The day of the week is now included, the time is now specified in 12-hour format with millisecond precision, and the time zone is now specified by abbreviation. Note that all production servers should be synchronized with a Network Time Protocol (NTP) server in order to ensure consistent time settings across the enterprise. In the example log entry this value is "Wed 07/30/2014 @ 10:45:59.420 PM CDT".
  • The "%h (%{X-Forwarded-For}i) > %v:%p" component logs the source and destination of the request. The %h directive specifies the client IP address, and the %{X-Forwarded-For}i directive specifies the underlying client IP address for proxied requests. Note that the value of the "X-Forwarded-For" header could be spoofed by the client. In the example log entry these values are "10.1.1.1" and "-", respectively. The %v directive specifies the server name or IP address, and the %p directive specifies the server port. The server port is useful to determine whether a request was transmitted over cleartext HTTP or encrypted SSL network connections. In the example log entry these values are "192.1681.1.1" and "80", respectively.
  • The "\"%r\" %I %D %>s %b %k %L" component logs details regarding the request and response. Note that the \" syntax specifies a literal double quote. The %r directive specifies the first line of the request, namely the request method, path, query string, and protocol. Uncommon methods such as "PUT" could indicate automated scanning tools or targeted attacks. Uncommon paths such as "/admin.html" could indicate automated scanning tools or targeted attacks. If present, the query string could contain attack strings indicative of common attacks such as Cross-Site Scripting (XSS) and SQL Injection. Uncommon protocols such as "HTTP/1.0" could indicate automated scanning tools or targeted attacks. In the example log entry this value is "GET /example.html HTTP/1.1". The %I directive logs the total number of bytes received from the client, including headers and the request itself. An unusually high number of bytes received could indicate certain types of attacks such as buffer overflows. Note that the %I directive requires mod_logio, which is statically compiled into Apache. In the example log entry this value is "222". The %D directive logs the number of microseconds taken to serve the request. Unusually long times could indicate certain types of attacks such as successful time-based SQL injection. In the example log entry this value is "444". The %>s directive logs the status code of the request. The > character specifies the final status code after all intermediate redirects have been processed. Uncommon status codes such as "405" ("Method Not Allowed") could indicate automated scanning tools or targeted attacks. In the example log entry this value is "200". The %O directive logs the total number of bytes sent by the server, including headers and the response itself. An unusually high number of bytes could indicate certain types of attack such as successful SQL injection. Note that the %O directive requires mod_logio, which is statically compiled into Apache. In the example log entry this value is "666". The %k directive logs the number of keepalive requests processed on the connection. Automated scanners typically do not utilize keepalive requests. In the example log entry this value is "0". Finally the %L directive logs the request log identifier from the error log. This identifier can be used to correlate the request with errors log entries from the /var/log/apache2/error.log logfile. In the example log entry this value is "-".
  • The "%{Referer}i" directive logs the "Referer" header sent by the client. Note that the value of the "Referer" header could be spoofed by the client. In the example log entry this value is "http://192.168.1.1/from.html".
  • The "%{User-Agent}i" directive logs the "User-Agent" header sent by the client. Note that the value of the "User-Agent" header could be spoofed by the client. In the example log entry this value is "Mozilla/5.0 (X11; Linux x86_64; rv:33.0) Gecko/20100101 Firefox/33.0".
  • The "%u %{User}C" component logs details about user authentication. The %u directive logs the username if the request was authenticated with HTTP Basic or Digest authentication. The identity of the authenticated user can be extremely useful during a forensic investigation. In the example log entry this value is "-", meaning that the user was not authenticated with HTTP Basic or Digest authentication. The %{User}C directive logs the "User" cookie. The application server would set this cookie to the username of the authenticated user upon successful form-based authentication. Note that the value of the "User" cookie could be spoofed by the client. However, the application server could verify the "User" cookie value sent with each request. If the cookie value does not match the authenticated user correlated with the submitted session identifier then a deterministic status code such as "403 Forbidden" could be returned to the client. Consequently a "200 OK" status code would confirm the "User" cookie value. In the example log entry this value is "jdoe".
  • The %{SessionTracker}C directive logs the "SessionTracker" cookie. The application server would set this cookie to a unique identifier, for example the SHA-1 hash of the session identifier, in order to track requests throughout the duration of a session. Note that the value of the "SessionTracker" cookie could be spoofed by the client. However, the application server could verify the "SessionTracker" cookie value sent with each request. For example, if the cookie value does not match the SHA-1 hash of the submitted session identifier then a deterministic status code such as "403 Forbidden" could be returned to the client. Consequently a "200 OK" status code would confirm the "SessionTracker" cookie value. Tracking requests throughout the duration of a session can be extremely useful during a forensic investigation. However, it is important to note that the "SessionTracker" is a one way hash of the session identifier, not the session identifier itself ("JSESSIONID") and therefore cannot be utilized to resume a session. If the session identifier itself was logged then attackers could leverage a compromised logfile in order to hijack authenticated sessions. Consequently, session identifiers and other sensitive security tokens should never be logged. In the example log entry this value is "0bcdd30af79b6aca8c8f7808c02ab530ac4c4a75".

As you can see, the enhanced log format captures a wealth of information that would be extremely useful during a forensic investigation. Additional information means larger logfiles, but disk space is cheap and the benefits of extended logging certainly outweigh the nominal increase in resource consumption. If you would like to log additional pieces of relevant information, you can refer to the complete list of supported Apache mod_log_config directives. If you would like to modify the date and time format, you can refer to the complete list of supported strftime conversion specifications. All that's left now is to enable the enhanced log format within the /etc/apache2/sites-available/000-default.conf and /etc/apache2/sites-available/default-ssl.conf configuration files:

CustomLog ${APACHE_LOG_DIR}/access.log enhanced

Finally we'll restart the Apache daemon in order to load the configuration changes:

root@debian $ service apache2 restart

Done and Done! Apache will now begin logging each request in the enhanced log format, providing additional information that would be extremely useful during a forensic investigation. There's no doubt about it, our enhanced log format is a lean mean forensic machine!

Blog Entry Filed Under: