Video Screencast Help
Give us your opinion and win with Symantec! Please help us by taking this survey to tell us about your experience with Symantec Connect, so that we can continue to grow and improve.  Take the survey.

Help with Apache Collector

Created: 16 Oct 2012 • Updated: 07 Nov 2012 | 15 comments
This issue has been solved. See solution.
Hi,
 
    I have been use Apache_Web_Server_Event_Collector_4.3.5_AllLinux_EN.zip for install Apache Collector, but I don't know what could be  wrong because I see only events of kind:
 
Event Type = Application Start  or Stop
 
Product = Apache Event Collector
 
    I make uninstall and install and nothing changes.
 
   Could me help?
 
   I am using Red Hat Linux  5.8  x86_64
 
Thanks!

Comments 15 CommentsJump to latest comment

Laurent_c's picture

one of the common problem with apache collector is the format of the apache log.

Could you go to your sensor properties, and disable the default filters ? See if you start to get events.

suporte.symantec's picture
Hello Laurent_c
 
       All filters are unchecked, but not worked.
 
       How could I testing this comunication if is really ok ?
 
 
Thanks
Avkash K's picture

can you please confirm, if you are getting any error in the log file.

Regards,

Avkash K

suporte.symantec's picture

Following my apache.log

INFO    2012-10-17 09:33:39,663 Collectors.3194.wGroup.[workinggroup0]  Thread-407      All sensors are stopped. Terminating workinggroup0 working group...
INFO    2012-10-17 09:33:39,733 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   Working group is off
INFO    2012-10-17 09:33:39,733 Collectors.3194 Thread-407      collector instance: 31940101 and its WorkingGroups are stopped
INFO    2012-10-17 09:33:39,736 Collectors.3194.aggregator.AggregatorCacheImpl  Thread-407      Aggregator BUFFER has been flushed and cleared. Capacity: 0
INFO    2012-10-17 09:33:39,737 Collectors.3194 Thread-407      ---------------- collector stopped ----------------
INFO    2012-10-17 09:34:01,650 Collectors.3194 com.symantec.management.util.TimerThread        Start initialization...
INFO    2012-10-17 09:34:01,851 Collectors.3194.wGroup.[workinggroup0]  com.symantec.management.util.TimerThread        Working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor" initiated (enable: true).
INFO    2012-10-17 09:34:01,851 Collectors.3194.aggregator      com.symantec.management.util.TimerThread        AGGREGATOR has been initialized with: 0 specs. Enabled: 0, disabled: 0.
INFO    2012-10-17 09:34:01,857 Collectors.3194 com.symantec.management.util.TimerThread        Collector (product ID: 3194) initialized.
INFO    2012-10-17 09:34:01,859 Collectors.3194 com.symantec.management.util.TimerThread        --------- The start is requested... ------
INFO    2012-10-17 09:34:01,860 Collectors.3194 com.symantec.management.util.TimerThread        Sender started
INFO    2012-10-17 09:34:01,861 Collectors.3194 com.symantec.management.util.TimerThread        Starting WorkingGroup (instance:31940101, group name: "workinggroup0")
INFO    2012-10-17 09:34:01,862 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   Starting working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor"...
INFO    2012-10-17 09:34:01,862 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   0 events were deserialized
INFO    2012-10-17 09:34:01,864 Collectors.3194 com.symantec.management.util.TimerThread        ---------------- collector started. ----------------
INFO    2012-10-17 09:34:01,869 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   All POLL sensor threads (1) have been created.
INFO    2012-10-17 09:34:01,879 Collectors.3194.wGroup.[workinggroup0].Sensor.[Sensor_0]        Thread-17       Start position successfully loaded. Starting reading /var/log/httpd/access_log from 3386.
 
 
Thanks!
Laurent_c's picture

Can you post a sample of the access_log ?

Just the first few lines ? I am sure it is a translation error, and the log format is not w3c or it is missing fields

suporte.symantec's picture

Hello

Following the access_log:

172.X.Y.Z - - [11/Oct/2012:15:29:18 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:29:18 -0300] "GET /icons/apache_pb2.gif HTTP/1.1" 200 2414 "http://172.A.B.C/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:29:21 -0300] "GET /asd HTTP/1.1" 404 279 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:29:23 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:30:31 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:31:44 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:40:22 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:40:22 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:40:32 -0300] "GET /index.htm HTTP/1.1" 404 269
172.X.Y.Z - - [11/Oct/2012:16:40:32 -0300] "GET /index.htm HTTP/1.1" 404 269 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:40:34 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:40:34 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:44:28 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:44:28 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:44:46 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:44:46 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:55:01 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:55:01 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
 
 
httpd.conf
LogFormat "%h %l %u %t \"$r\" %>s %b" collector
CustomLog logs/access_log collector
 
Thanks!
Laurent_c's picture

Correct format should be :

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" collector

example of good log :

81.23.112.138 - - [20/Oct/2005:19:42:20 +0400] "GET /scripts/conf.jsp HTTP/1.0" 200 59 "http://www.itplus.ru/itplus/forum_home/forum_reg.htm" "Mozilla/4.0 (compatible;MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; InfoPath.1)"

suporte.symantec's picture
Hello Laurent_c
 
It works for this local apache.
 
 
Now, I have a question:
 
What I need to do for obtain the logs of  apache in differents servers without install Apache Collector  there?
 
Thanks
Laurent_c's picture

Great this works now.

About collecting from several servers. Generally what customers do is to centrenlise the log on a single file server.

then you can map or point sensor to this folder.

This require a couple of commands/scripts to regularly move the log to this location to be read by collector.

suporte.symantec's picture
Laurent_c,
 
 
       So, I made it. I have a server  to centralize all servers logs, I copied a access_log of server2 to this server, I created a sensor 1 to read this new log and distribute, but the logs aren't recognized  by SSIM. 
 
       What could be wrong ?
 
       Both servers have the same LogFormat an permission .
 
Following my apache.log:
 
INFO    2012-10-19 13:51:51,436 Collectors.3194 com.symantec.management.util.TimerThread        ---------------- collector started. ----------------
INFO    2012-10-19 13:51:51,436 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   Starting working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor"...
INFO    2012-10-19 13:51:51,436 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   0 events were deserialized
INFO    2012-10-19 13:51:51,437 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   All sensor threads (2) have been created.
INFO    2012-10-19 13:51:51,439 Collectors.3194.wGroup.[workinggroup0].Sensor.[Sensor_0]        Thread-39       Start position successfully loaded. Starting reading /var/log/httpd/access_log from 5409.
INFO    2012-10-19 13:51:51,440 Collectors.3194.wGroup.[workinggroup0].Sensor.[Sensor_1_OTRS]   Thread-40       Reading will start from the end of /var/log/ssim/OTRS/access_log.1 log file. Log file is dynamic.
 
 
        Thanks
Laurent_c's picture

Hi,

Could you disable the working sensor, and leaving only the not working one enabled.

1- How do you tranfer the file ? -> Make sure the format is correct, this is a text file, so if you FTP make sure the type is not changed etc...

2- Is the secodn not working file coming from the same OS ? When you tranfer file from Windows to Unix, the EoL is not correct.

If you could maybe post a sampel log of the second sensor enabled only, and if possible put it in debug. If you don;t knwo how to put in debug let me know.

Laurent

suporte.symantec's picture

Laurent

I disabled the first sensor "sensor 0", right now is active onlly sensor 1 .

Following my answers:

1 - Transfer by "scp" 

2 - Both are Linux, but the  first is Scientific Linux and Second is Red Hat

Sample Logs (access_log.1):

172.X.Y.Z - - [19/Oct/2012:15:04:27 -0300] "GET /OTRS HTTP/1.1" 404 485 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
 

Apache logs :

  INFO    2012-10-24 07:40:12,459 Collectors.3194 com.symantec.management.util.TimerThread        ---------------- collector stopped ----------------

INFO    2012-10-24 07:40:12,467 Collectors.3194 com.symantec.management.util.TimerThread        Initializing collector...
INFO    2012-10-24 07:40:12,520 Collectors.3194.wGroup.[workinggroup0].translator       com.symantec.management.util.TimerThread        3 translator specifications exist.
INFO    2012-10-24 07:40:12,529 Collectors.3194.wGroup.[workinggroup0]  com.symantec.management.util.TimerThread        Working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor" initiated (enable: true).
INFO    2012-10-24 07:40:12,530 Collectors.3194.aggregator      com.symantec.management.util.TimerThread        AGGREGATOR has been initialized with: 0 specs. Enabled: 0, disabled: 0.
INFO    2012-10-24 07:40:12,531 Collectors.3194.filter  com.symantec.management.util.TimerThread        FILTER has been initialized with: 5 specs. Enabled: 0, disabled: 5.
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Configuration: NAPDLP01
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Product ID: 3194
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Product version: 4.3
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Software feature ID: 31940101
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Framework version: 2.47.00
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Collector uses logfile sensor v. 2.43.00
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Collector initialization completed.
INFO    2012-10-24 07:40:12,534 Collectors.3194 com.symantec.management.util.TimerThread        --------- The start is requested... ------
INFO    2012-10-24 07:40:12,535 Collectors.3194 com.symantec.management.util.TimerThread        Sender started
INFO    2012-10-24 07:40:12,535 Collectors.3194 com.symantec.management.util.TimerThread        Starting WorkingGroup (instance:31940101, group name: "workinggroup0")
INFO    2012-10-24 07:40:12,535 Collectors.3194 com.symantec.management.util.TimerThread        ---------------- collector started. ----------------
INFO    2012-10-24 07:40:12,535 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   Starting working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor"...
INFO    2012-10-24 07:40:12,535 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   0 events were deserialized
INFO    2012-10-24 07:40:12,535 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   All sensor threads (1) have been created.
INFO    2012-10-24 07:40:12,537 Collectors.3194.wGroup.[workinggroup0].Sensor.[Sensor_1_OTRS]   Thread-2714     Start position successfully loaded. Starting reading /var/log/ssim/OTRS/access_log.1 from 204.
 
How can I put in debug mode?
 
Thanks
suporte.symantec's picture
Laurent,
 
I have more informations.
 
Right now I see on Events logs recognized as Apache Event Collector, but they are atributed to AgentHost, but this logs are coming the other server.
 
 
Following the access_log to compare.
 
#### access_log by Agent Host
 
# tail -n 1 /var/log/httpd/access_log
172.X.Y.Z - - [24/Oct/2012:10:23:01 -0200] "GET /icons/apache_pb2.gif HTTP/1.1" 304 - "http://172.A.B.C/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
 
#### access_log by Other Computer (copy by scp )
 
# tail -n 1 /var/log/ssim/OTRS/access_log.1
172.X.Y.Z - - [24/Oct/2012:12:53:23 -0200] "GET / HTTP/1.1" 200 4133 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
 
 
What can I configure so this log can be correctly identified as comming from the source host ?
 
Thanks
Laurent_c's picture

Hi,

I have seen you have a case opened with Support, requested to get in touch and we could maybe run a webex and have a look at the configuration of your setup.

Laurent

suporte.symantec's picture

Hi

This case was solved by webex.

Thanks!

SOLUTION