Protection Engine for Cloud Services

We had received a requirement to scan files with 1.2 GBytes on size, and we are looking on stream scan process.

We have been testing our Symantec Scanning Server Engine with the following command line which call a wrapper class that wait for any key and then call the  sample program JavaAPICheck.java with the parameters showed, we are using JVisualVM to monitor the Memory,etc.

These are example files you are using. They are not designed for production usage.

You should be using ssecls, which is our compiled command-line scanning tool.

The expansion in memory is to be expected. You sent us a PDF file which is a container. We have to expand that container to scan all of the pieces inside of it.

Hi Oswald,

For test #1 it sounds like we could not write to the output file for some reason.  Do you consistently get this Error On Server Write error, or was this just a one off? 

For test #3 how large is the .zip file in question, is it fairly large?  Is Scan Engine under a lot of load, in terms of a concurrent requests?  If the .zip file is fairly large you might be running into the Socket communication error because our Java API is hitting our default readWriteTime timeout parameter.  By default it is only 3 seconds.  You can try increasing this parameter to a larger value, for example 60 seconds.  Then confirm if the issue is still occurring.  You can set the readWriteTime parameter when you call the createScanEngine function.  The parameter is set when you call createScanEngine. 

Regarding the memory usage of the client, I believe the client typically uses a buffer in a loop to pass/stream the file to the scanner.  Which is what our example program does, but our example program creates a new buffer array every time it goes through the, in this case a do while, loop.  If you are doing something similar, this might be where the memory consumption is coming from.  I do not believe the memory allocated each time the array of bytes is created is cleared until the function is finished.  So this could be accounting for the memory usage you are seeing on the client.  If this is case, this is not on our API, it would be on the connector you wrote.  Just do not create a "new" array of bytes every time you go through the while loop.

Hello,

We are working on .Net website application. Using the application, users can upload documents and those attachments to be scanned during upload to the server (could be from some temp folder on file server).

For achieving this we are planning to use Symantec Scan Engine 5.2 for scanning the file. We observed different classes/methods available in .NET SDK and we are confused with the Utility (StreamScanFile) and StreamScanRequest options.

 As per the help files provided,

Utility - Abstracts the local stream file scanning mehcanism. It can be used when the file to be scanned is accessible only to the connector and not to the Scan engine.

StreamScanRequest  - Abstracts the stream file scanning mehcanism. This is used to Scan file which is not accessible to Scan engine. 

The two options are the difference between streaming the file to Scan Engine, or telling Scan Engine the location of the file on a shared drive that Scan Engine will be able to access (example: \\webserver\scanenginetemp\testfile.txt).

Thanks Dave for your immediate response. But as per the help file provided in .NET SDK,  Utility class method can be used when the file to be scanned is accessible only to the connector and not to the Scan engine. What does the connector means? Is it calling application or Scan Engine Server? Our understanding was application. Please confirm.

Thanks, Anil.

Hi Anil,

Your application, which is using our .NET API, is the "connector". 

Thanks,
Ben

TSE-JDavis,

Thanks for your answer but we are interested in the java API not in the commnad line version, we are using the command line for other use cases.

You will want to build on the example files we have created and flesh out the code providing for more memory usage and adjusting timeouts for the sizes of files you will be scanning.

Benc Smith,

Thanks for your replies. Our Test 1 and 3 are using the same resources and same zip file and the error are interchangeable.

I do appreciate the tip about the array buffer, I will test it.

Could you suggest me what will be the best way, using the java api to be able to process large files above 1.2 GBytes.

1.- Streaming chunks pieces one at the time.

2.- Perhaps communicate using specific symantec protocol from java api to pass the server:dynamic-file-path in a share directory.

thanks.

TSE-JDavis,

thanks for your replies.

I wounder which streaming scanning will be more appropriate for large size file ( over 1.2 GBytes).

1.- I understand using the compiled command line option as you suggested without the java api.

2.- Using java api streaming chunks of the file to the server.

3.- Using java api to pass the server:dynamic file path location of the file in a shared directory.

Considering efficiency in resources.

thanks in advance

Hi,

We are working on a .Net application which uses Symantec Scan engine .net API for file scan. I tried using the Utility option. When we called the API method, we did not get any error. But the scan result is showing the socket error and below is the full information.

 
File Status - CLEAN
Total Infections - 0
Definition Date: 1/1/0001 12:00:00 AM
 
and scanResult.connTriesInfo.problemEncountered as ERR_SOCKET_ERROR. i am not sure whether the scan is successful or not as it shows connection error.
 
Please suggest what could be the issue. 

Regards,

Anil.

Hi,

I identified the above issue (ERR_SOCKET_ERROR) issue and is because of passing incorrect port number. Now I am getting ERR_INITIALIZING_STREAM_REQUEST error while calling the below method. Can you please suggest what could be the issue.

objStreamScanRequest.Start(objFileScanEntity.fileForScan, objFileScanEntity.scanFileOriginalName);

 
Thanks,
Anil.