Contents of the Readme file for Deepsight Extractor for Symantec Network Security 7100 Series

Article:TECH112647  |  Created: 2005-01-07  |  Updated: 2005-01-14  |  Article URL http://www.symantec.com/docs/TECH112647
Article Type
Technical Solution


Issue



This document is a copy of the Readme file for the Deepsight Extractor utility. This utility is included with Symantec Network Security 7100 Series (SNS 7100).


Solution



The DeepSight Extractor tool is used to parse and send SNS 7100 log files to the DeepSight Analyzer system. The DeepSight Analyzer service is free for those who contribute data to it by means of the DeepSight Extractor tool.

The tool is already installed to the appliance. To use the tool, configure SNS 7100 to copy the log files to a text file, by editing the /etc/manhunt.conf file, and configure the tool by editing the ConfigFile.ini file. Detailed instructions for DeepSight Extractor configuration and usage are included in the readme file, README-FIRST, which is located at: /root/.extractor/README-FIRST

The following text is a copy of the README-FIRST file.

Guide to DeepSight Extractor 4

DeepSight Extractor is a tool to parse and upload IDS and firewall logs to the DeepSight Analyzer system. DeepSight Analyzer provides data contributors a facility for firewall log and IDS data management. With it, data contributors can build and maintain cumulative, date-sensitive incident portfolios for each firewall and IDS system based on the data supplied. DeepSight Analyzer allows correlation of event information from a variety of firewall and intrusion detection systems; it allows tracking statistics, printing reports; and identifying and notifying the parties responsible for the attacking host(s).

Some of the specific services available to DeepSight data contributors are:

- Tracking Incidents by IP address
- Tracking Incidents by attack type
- Generating notification letters
- Generating reports
- DeepSight Users news and discussion forum

DeepSight Analyzer is a free service for DeepSight Extractor data contributors.

This guide is intended as an introduction to Extractor 4.0, it includes coverage of .x updates. It is broken into sections to provide appropriate information from your initial planning through installation and configuration:

1. Extractor Overview
2. Extractor Installation
3. Configuring ManHunt to Log to Text File
4. Extractor Configuration
5. Command Line Usage
6. Running Extractor in Daemon Mode
7. Help with DeepSight Extractor


1. Extractor Overview
Two components are required in order to use the DeepSight Analyzer service. First, the user must register for an account on the DeepSight Analzyer website (https://analyzer.symantec.com/). Second, an organization must upload firewall or intrusion detection system log files using Extractor 4 to their Analzyer account.

Extractor has been deployed on the Symantec Network Security appliance ready for use and requires only minimal configuration. The user is required to have a DeepSight Analyzer account. The account details will be used to configure Extractor as described below. Once configured and running, Extractor will upload security events from your ManHunt node to your Analyzer account.

By default, Extractor is installed in the following directory on the Symantec Network Security appliance: /usr/local/bin

The support files for proper operation of Extractor are also installed on the Symantec Network Security appliance:

/root/.extractor/ConfigFile.ini.dist
A sample configuration file. Please copy this file to ConfigFile.ini and modify the copy

/root/.extractor/manhunt.rules
The rules file for parsing ManHunt events from log file.

/root/.extractor/README-FIRST
This file


NOTE: The first upload of a new Extractor profile may be CPU intensive because Extractor must parse the entire log file. The duration of this CPU load is dependent on the size of the log file to parse. Subsequent uploads are incremental, this minimizes the CPU load when Extractor is parsing.


2. Installing Extractor
Extractor is automatically installed on the Symantec Network Security appliance. The user is only required to configure Extractor via the ConfigFile.ini file and start the process.

The required support files for Extractor are installed in the hidden sub-directory .extractor in the user's home directory. By default this is: /root/.extractor.

You must first configure the Extractor configuration file ConfigFile.ini. Please copy the supplied sample ConfigFile.ini.dist to ConfigFile.ini and modify the copy. This file contains two sections both of which are outlined in section 4 below.


3. Configuring ManHunt to log to text file
Extractor will parse and upload incidents detected by the Symantec Network Security node. ManHunt must be configured to log events to a text log file. To do this you must add the ManHunt configuration file /etc/manhunt.conf.

Login with root or equivalent privileges. Using any text editor open /etc/manhunt.conf and add the following line to the file.

eventwriter_outfl=/usr/SNS/logs/eventwriterd.log

You may create the log file in any location on the Symantec Network Security appliance file system, the above is the suggested location.

Once you have made the change save the file and restart ManHunt using the following command:

/usr/SNS/restart

Now ManHunt will log all events to the text file you specified in the configuration file.

If you decide to rotate or archive the generated log file then please note that Extractor must be restarted once the file has been rotated or archived. More details are available in section 6.


4. Configuring Extractor
Login with root or equivalent privileges.

Change to the hidden Extractor sub-directory .extractor

cd /home/user-directory/.extractor

Use Vi or your favorite editor to open and modify the ConfigFile.ini file

The ConfigFile.ini configuration file is divided into 2 sections: globals and profiles. Each component is described in more detail below, but the [globals] section configures Extractor itself and the [profile] section details individual sensor configurations. To add and configure additional sensors, duplicate a properly configured [profile] section then modify the copied [profile] section.


4.1 [globals] Configuration Parameters

EnableStatusLogging="0"
Set to "1" to turn on Extractor status logging. If you activate status logging you must specify a location for the Extractor status log using the StatusLogFile parameter.

Connecting through a proxy
The next group of parameters defines how Extractor connects to the upload server through a proxy.

EnableProxy="0"
Set to "1" to connect through a proxy.

ProxyHost=""
Enter the DNS name or IP address of the proxy, i.e., "fwall.yourdomain.com" or 192.168.0.1

ProxyPort=""
Specify the connection port to the proxy. Port 3128 is a common proxy connection port value.

ProxyEnableAuth="0"
Set to "1" when the proxy requires authentication.

ProxyUser=""
Enter the authorized proxy users' name "jchosely"

ProxyPassword=""
Enter the authorized proxy users' password "password"

Account information
The next parameters specify the Analyzer upload account information.

AnalyzerUsername="analyzer-user"
The Analyzer user account for the upload.

AnalyzerPassword="analyzer-password"
The user account password.

StatusLogFile="/home/user-directory/.extractor/logfile"
Specify the location for the extractor status log when EnableStatusLogging is set to "1"

UploadEnableEventLimit="0"
Set to "1" to limit the number of events in one upload.

UploadEventLimit="10000"
Limit the number of events Extractor will upload at one time by entering a "#value." 10000 is a reasonable starting point. UploadEnableEventLimit must be set to "1" to use this parameter.

UploadReportErrors="1"
When set to "1," Extractor will log a connection failure to another Symantec server. The log entry specifies the time of the failure and your Analyzer username to assist technical support in troubleshooting the connection issues. No entry is logged for connection issues on your local network.

UploadEnableRetry="1"
Set to "0" to disable retries.

UploadRetries="10"
Set the number of retries after a connection failure.

UploadEnableCompression="1"
Set to "0" to turn off log file compression.

UploadSSL="1"
Set to "0" to upload log files without SSL. Login authentication still requires SSL, but the log is sent in the clear.

AlternateHost parameters
The "AlternateHost" parameters should be used only at the direction of Symantec technical support.

UploadEnableAlternateHost="0"
Set to "1" to upload to a special location.

UploadAlternateHostName=""
Enter the "hostname" provided by Symantec.

ScheduleFreq="5"
Specify the number of minutes between Extractor uploads. The maximum value for ScheduleFreq is 1440 minutes which limits uploads to once a day.

SOPath="/usr/local/extractor/bin"
This provides the path to the shared object files for parsers. By default, this is the path where Extractor is installed.


4.2 [profile=] Configuration Parameters
The profile section defines differences in firewalls and Intrusion Detection Systems acting as Analyzer sensors. To add a new sensor, copy and modify an existing sensor profile using Vi or your favorite editor.

[profile="ids-sensor1"]
Enter a profile name for the sensor. Unique profile names are required.

IDSRules="/root/.extractor/manhunt.rules"
Specify the complete path and the .rules file that defines the sensor device. These device definitions are in the /rules directory.

DataSource1="/usr/SNS/logs/eventwriterd.log"
Enter the complete path to the sensor log files.

DataSource2=""
The secondary DataSource is provided for NetProwler and Dragon systems as well as for future use.

DataSourceUsername=""
Enter the username required to access the sensors' log files; this is usually not required.

DataSourcePasswd=""
Enter the password required to access the sensor log files. This is necessary only if a DataSourceUsername value was required.

StripList=""
Anonymize a network or device by entering an address in CIDR notation or just an IP address. Multiple addresses may be specified using a "," or ";" as a delimiter between the addresses.

EnableSchedule="1"
Set to "0" to require manual uploads of sensor data. The frequency of the Extractor uploads is specified in the [globals] section.

Time="gmt"
Valid values are: "local", "gmt", or "tz" These values must be entered in lower case. If "TZ" is specified, an offset must be specified for the GMTOffset parameter. For details on using the "TZ" parameter, see the "extractor-tz-offset" file.

GMTOffset=""
Enter your GMT offset.

PulseRate="0"
Not used at this time.


5. Extractor Command Line Usage

extractor
Show the command line options available (help text).

extractor <-u [Analyzer-username] -p [password]>
This is the basic command line format. The username and password may be supplied by the ConfigFile.ini.

extractor -U [profile-name] <-u [Analyzer-username] -p [password]>
Upload log data from a specific sensor.

extractor -L [xml file] <-u [Analyzer-username] -p [password]>
Upload an xml file.

extractor -X [profile] -o [output_file.xml]
Save the log data of a specific sensor to an xml file.

extractor -D
Run Extractor in Daemon mode using the configured schedule.

extractor -c
Write a blank configuration file to: /root/.extractor/ConfigFile.ini


-q Quiet operation prints only error messages to the screen

-f Ignore saved sensor timestamps for this profile and start reading from the beginning of the log file. (used with -X or -U)

-n Do not update sensor timestamps for this profile (used with -U)

-d Write all status log messages to the syslog; verbose logging. The default behavior is to write only important messages and errors. (used with -D)

-o Allows specification of an output file name. (used with -X)



6. Running Extractor in Daemon Mode
When started, Extractor uploads all profiles you have selected for scheduling with the profile configuration option EnableSchedule="1". Extractor continues these uploads based on the profile configuration option ScheduleFreq.

During operation, Extractor logs to your syslog user log. The log entries include Extractor's pid [Process ID]. While in daemon mode, Extractor handles the following signals:

Hangup Signal
- HUP [Hangup Signal]

Send a HUP to the Extractor process when:
  • Any of the logfiles your profiles point to are rolled or renamed.
  • You change your configuration file.

When sent a HUP, Extractor will finish the current round of profile uploads; close all log files; reload your configuration; and then restart the upload cycle.

Terminate and Quit signals
- TERM [Terminate signal] or QUIT [Console quit signal]

Send a TERM to the Extractor process when you want it to exit cleanly. To send this signal to the Extractor process:

For BSD/Linux use:

killall -HUP extractor (to send a HUP)
killall extractor (to send a TERM)

For Solaris use:

pkill -HUP extractor (to send a HUP)
pkill extractor (to send a TERM)


7. Help with DeepSight Extractor
If you have any comments or questions regarding Symantec DeepSight Extractor, please email extractor@symantec.com.





Legacy ID



2005010710534853


Article URL http://www.symantec.com/docs/TECH112647


Terms of use for this information are found in Legal Notices