Video Screencast Help
Search Video Help Close Back
to help

Help with Apache Collector

Created: 16 Oct 2012 | Updated: 07 Nov 2012 | 15 comments
suporte.symantec's picture
suporte.symantec
0 0 Votes
Login to vote
This issue has been solved. See solution.

 

Hi,
 
    I have been use Apache_Web_Server_Event_Collector_4.3.5_AllLinux_EN.zip for install Apache Collector, but I don't know what could be  wrong because I see only events of kind:
 
Event Type = Application Start  or Stop
 
Product = Apache Event Collector
 
    I make uninstall and install and nothing changes.
 
   Could me help?
 
   I am using Red Hat Linux  5.8  x86_64
 
Thanks!
Discussion Filed Under:
, , , ,

Comments 15 Comments • Jump to latest comment

Laurent_c's picture
Laurent_c Symantec Employee Accredited
Help with Apache Collector - Comment:16 Oct 2012 : Link

one of the common problem with apache collector is the format of the apache log.

 

Could you go to your sensor properties, and disable the default filters ? See if you start to get events.

0
Login to vote
  • Actions
suporte.symantec's picture
suporte.symantec
Help with Apache Collector - Comment:16 Oct 2012 : Link

 

Hello Laurent_c
 
       All filters are unchecked, but not worked.
 
       How could I testing this comunication if is really ok ?
 
 
Thanks

 

 

0
Login to vote
  • Actions
Avkash K's picture
Avkash K Partner
Help with Apache Collector - Comment:16 Oct 2012 : Link

can you please confirm, if you are getting any error in the log file.

Regards,

Avkash K

0
Login to vote
  • Actions
suporte.symantec's picture
suporte.symantec
Help with Apache Collector - Comment:17 Oct 2012 : Link

Following my apache.log

 

 

INFO    2012-10-17 09:33:39,663 Collectors.3194.wGroup.[workinggroup0]  Thread-407      All sensors are stopped. Terminating workinggroup0 working group...
INFO    2012-10-17 09:33:39,733 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   Working group is off
INFO    2012-10-17 09:33:39,733 Collectors.3194 Thread-407      collector instance: 31940101 and its WorkingGroups are stopped
INFO    2012-10-17 09:33:39,736 Collectors.3194.aggregator.AggregatorCacheImpl  Thread-407      Aggregator BUFFER has been flushed and cleared. Capacity: 0
INFO    2012-10-17 09:33:39,737 Collectors.3194 Thread-407      ---------------- collector stopped ----------------
INFO    2012-10-17 09:34:01,650 Collectors.3194 com.symantec.management.util.TimerThread        Start initialization...
INFO    2012-10-17 09:34:01,851 Collectors.3194.wGroup.[workinggroup0]  com.symantec.management.util.TimerThread        Working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor" initiated (enable: true).
INFO    2012-10-17 09:34:01,851 Collectors.3194.aggregator      com.symantec.management.util.TimerThread        AGGREGATOR has been initialized with: 0 specs. Enabled: 0, disabled: 0.
INFO    2012-10-17 09:34:01,857 Collectors.3194 com.symantec.management.util.TimerThread        Collector (product ID: 3194) initialized.
INFO    2012-10-17 09:34:01,859 Collectors.3194 com.symantec.management.util.TimerThread        --------- The start is requested... ------
INFO    2012-10-17 09:34:01,860 Collectors.3194 com.symantec.management.util.TimerThread        Sender started
INFO    2012-10-17 09:34:01,861 Collectors.3194 com.symantec.management.util.TimerThread        Starting WorkingGroup (instance:31940101, group name: "workinggroup0")
INFO    2012-10-17 09:34:01,862 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   Starting working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor"...
INFO    2012-10-17 09:34:01,862 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   0 events were deserialized
INFO    2012-10-17 09:34:01,864 Collectors.3194 com.symantec.management.util.TimerThread        ---------------- collector started. ----------------
INFO    2012-10-17 09:34:01,869 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   All POLL sensor threads (1) have been created.
INFO    2012-10-17 09:34:01,879 Collectors.3194.wGroup.[workinggroup0].Sensor.[Sensor_0]        Thread-17       Start position successfully loaded. Starting reading /var/log/httpd/access_log from 3386.
 
 
Thanks!
0
Login to vote
  • Actions
Laurent_c's picture
Laurent_c Symantec Employee Accredited
Help with Apache Collector - Comment:17 Oct 2012 : Link

Can you post a sample of the access_log ?

Just the first few lines ? I am sure it is a translation error, and the log format is not w3c or it is missing fields

0
Login to vote
  • Actions
suporte.symantec's picture
suporte.symantec
Help with Apache Collector - Comment:18 Oct 2012 : Link

Hello

 

Following the access_log:

 

172.X.Y.Z - - [11/Oct/2012:15:29:18 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:29:18 -0300] "GET /icons/apache_pb2.gif HTTP/1.1" 200 2414 "http://172.A.B.C/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:29:21 -0300] "GET /asd HTTP/1.1" 404 279 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:29:23 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:30:31 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:15:31:44 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:40:22 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:40:22 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:40:32 -0300] "GET /index.htm HTTP/1.1" 404 269
172.X.Y.Z - - [11/Oct/2012:16:40:32 -0300] "GET /index.htm HTTP/1.1" 404 269 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:40:34 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:40:34 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:44:28 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:44:28 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:44:46 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:44:46 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
172.X.Y.Z - - [11/Oct/2012:16:55:01 -0300] "GET / HTTP/1.1" 403 3985
172.X.Y.Z - - [11/Oct/2012:16:55:01 -0300] "GET / HTTP/1.1" 403 3985 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
 
 
httpd.conf
LogFormat "%h %l %u %t \"$r\" %>s %b" collector
CustomLog logs/access_log collector
 
Thanks!
0
Login to vote
  • Actions
Laurent_c's picture
Laurent_c Symantec Employee Accredited
Help with Apache Collector - Comment:18 Oct 2012 : Link

Correct format should be :

 

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" collector

 

example of good log :

81.23.112.138 - - [20/Oct/2005:19:42:20 +0400] "GET /scripts/conf.jsp HTTP/1.0" 200 59 "http://www.itplus.ru/itplus/forum_home/forum_reg.htm" "Mozilla/4.0 (compatible;MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; InfoPath.1)"

 

+1
Login to vote
  • Actions
suporte.symantec's picture
suporte.symantec
Help with Apache Collector - Comment:18 Oct 2012 : Link

 

Hello Laurent_c
 
It works for this local apache.
 
 
Now, I have a question:
 
What I need to do for obtain the logs of  apache in differents servers without install Apache Collector  there?
 
Thanks

 

 

0
Login to vote
  • Actions
Laurent_c's picture
Laurent_c Symantec Employee Accredited
Help with Apache Collector - Comment:19 Oct 2012 : Link

Great this works now.

 

About collecting from several servers. Generally what customers do is to centrenlise the log on a single file server.

then you can map or point sensor to this folder.

 

This require a couple of commands/scripts to regularly move the log to this location to be read by collector.

0
Login to vote
  • Actions
suporte.symantec's picture
suporte.symantec
Help with Apache Collector - Comment:19 Oct 2012 : Link

 

Laurent_c,
 
 
       So, I made it. I have a server  to centralize all servers logs, I copied a access_log of server2 to this server, I created a sensor 1 to read this new log and distribute, but the logs aren't recognized  by SSIM. 
 
       What could be wrong ?
 
       Both servers have the same LogFormat an permission .
 
Following my apache.log:
 
INFO    2012-10-19 13:51:51,436 Collectors.3194 com.symantec.management.util.TimerThread        ---------------- collector started. ----------------
INFO    2012-10-19 13:51:51,436 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   Starting working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor"...
INFO    2012-10-19 13:51:51,436 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   0 events were deserialized
INFO    2012-10-19 13:51:51,437 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   All sensor threads (2) have been created.
INFO    2012-10-19 13:51:51,439 Collectors.3194.wGroup.[workinggroup0].Sensor.[Sensor_0]        Thread-39       Start position successfully loaded. Starting reading /var/log/httpd/access_log from 5409.
INFO    2012-10-19 13:51:51,440 Collectors.3194.wGroup.[workinggroup0].Sensor.[Sensor_1_OTRS]   Thread-40       Reading will start from the end of /var/log/ssim/OTRS/access_log.1 log file. Log file is dynamic.
 
 
        Thanks
0
Login to vote
  • Actions
Laurent_c's picture
Laurent_c Symantec Employee Accredited
Help with Apache Collector - Comment:24 Oct 2012 : Link

Hi,

 

Could you disable the working sensor, and leaving only the not working one enabled.

1- How do you tranfer the file ? -> Make sure the format is correct, this is a text file, so if you FTP make sure the type is not changed etc...

2- Is the secodn not working file coming from the same OS ? When you tranfer file from Windows to Unix, the EoL is not correct.

 

If you could maybe post a sampel log of the second sensor enabled only, and if possible put it in debug. If you don;t knwo how to put in debug let me know.

 

Laurent

+1
Login to vote
  • Actions
suporte.symantec's picture
suporte.symantec
Help with Apache Collector - Comment:24 Oct 2012 : Link

Laurent

I disabled the first sensor "sensor 0", right now is active onlly sensor 1 .

Following my answers:

1 - Transfer by "scp" 

2 - Both are Linux, but the  first is Scientific Linux and Second is Red Hat

Sample Logs (access_log.1):

 

172.X.Y.Z - - [19/Oct/2012:15:04:27 -0300] "GET /OTRS HTTP/1.1" 404 485 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
 

Apache logs :

  INFO    2012-10-24 07:40:12,459 Collectors.3194 com.symantec.management.util.TimerThread        ---------------- collector stopped ----------------

INFO    2012-10-24 07:40:12,467 Collectors.3194 com.symantec.management.util.TimerThread        Initializing collector...
INFO    2012-10-24 07:40:12,520 Collectors.3194.wGroup.[workinggroup0].translator       com.symantec.management.util.TimerThread        3 translator specifications exist.
INFO    2012-10-24 07:40:12,529 Collectors.3194.wGroup.[workinggroup0]  com.symantec.management.util.TimerThread        Working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor" initiated (enable: true).
INFO    2012-10-24 07:40:12,530 Collectors.3194.aggregator      com.symantec.management.util.TimerThread        AGGREGATOR has been initialized with: 0 specs. Enabled: 0, disabled: 0.
INFO    2012-10-24 07:40:12,531 Collectors.3194.filter  com.symantec.management.util.TimerThread        FILTER has been initialized with: 5 specs. Enabled: 0, disabled: 5.
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Configuration: NAPDLP01
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Product ID: 3194
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Product version: 4.3
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Software feature ID: 31940101
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Framework version: 2.47.00
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Collector uses logfile sensor v. 2.43.00
INFO    2012-10-24 07:40:12,531 Collectors.3194 com.symantec.management.util.TimerThread        Collector initialization completed.
INFO    2012-10-24 07:40:12,534 Collectors.3194 com.symantec.management.util.TimerThread        --------- The start is requested... ------
INFO    2012-10-24 07:40:12,535 Collectors.3194 com.symantec.management.util.TimerThread        Sender started
INFO    2012-10-24 07:40:12,535 Collectors.3194 com.symantec.management.util.TimerThread        Starting WorkingGroup (instance:31940101, group name: "workinggroup0")
INFO    2012-10-24 07:40:12,535 Collectors.3194 com.symantec.management.util.TimerThread        ---------------- collector started. ----------------
INFO    2012-10-24 07:40:12,535 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   Starting working group for sensor "com.symantec.cas.ucf.sensors.logfile.LogFileSensor"...
INFO    2012-10-24 07:40:12,535 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   0 events were deserialized
INFO    2012-10-24 07:40:12,535 Collectors.3194.wGroup.[workinggroup0]  workinggroup0   All sensor threads (1) have been created.
INFO    2012-10-24 07:40:12,537 Collectors.3194.wGroup.[workinggroup0].Sensor.[Sensor_1_OTRS]   Thread-2714     Start position successfully loaded. Starting reading /var/log/ssim/OTRS/access_log.1 from 204.
 
How can I put in debug mode?
 
Thanks
0
Login to vote
  • Actions
suporte.symantec's picture
suporte.symantec
Help with Apache Collector - Comment:24 Oct 2012 : Link

 

Laurent,
 
I have more informations.
 
Right now I see on Events logs recognized as Apache Event Collector, but they are atributed to AgentHost, but this logs are coming the other server.
 
 
Following the access_log to compare.
 
#### access_log by Agent Host
 
# tail -n 1 /var/log/httpd/access_log
172.X.Y.Z - - [24/Oct/2012:10:23:01 -0200] "GET /icons/apache_pb2.gif HTTP/1.1" 304 - "http://172.A.B.C/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
 
#### access_log by Other Computer (copy by scp )
 
# tail -n 1 /var/log/ssim/OTRS/access_log.1
172.X.Y.Z - - [24/Oct/2012:12:53:23 -0200] "GET / HTTP/1.1" 200 4133 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET4.0C; .NET4.0E)"
 
 
What can I configure so this log can be correctly identified as comming from the source host ?
 
Thanks
0
Login to vote
  • Actions
Laurent_c's picture
Laurent_c Symantec Employee Accredited
Help with Apache Collector - Comment:30 Oct 2012 : Link

Hi,

 

I have seen you have a case opened with Support, requested to get in touch and we could maybe run a webex and have a look at the configuration of your setup.

 

Laurent

0
Login to vote
  • Actions
suporte.symantec's picture
suporte.symantec Partner
Help with Apache Collector - Comment:07 Nov 2012 : Link

Hi

 

This case was solved by webex.

 

Thanks!

SOLUTION
0
Login to vote
  • Actions