New York Data Loss Prevention User Group

Hi All-

We are looking deploying Vontu Web Prevent, and are looking to use some basic loadbalacing to create a "farm" of sensors.

The load balacing would be handled by the proxies themselves (Bluecoat), using their round-robin algorithm to send to the "farm".

So for example, if I had 3 proxies- and a "farm" of 6 Vontu Web Prevent Sensors- could I set each proxy up to roundrobin ICAP sessiosn to the 6 Vontu Sensors, in a mesh style format?

The real question is-  will the Vontu sensors have issues taking ICAP from multiple servers? As long as each "sessions" isn't split up- I don't think there should be a problem.

We are currently experimenting in the lab.

Is anyone doing this currently- or perhaps following a different model?

Thanks in advance!

Regards,

Will

One more important thing to add- we are *not* engineer this in "blocking" mode- we are only monitoring. In other words, when the ICAP comes into the Web Prevent Sensor- it will scan the traffic, but will always send back an "OK" to the proxy- it won't block the traffic.

I only add this bit of information, as it may make a difference as to whether or not the Sensors can support this ICAP load balancing solution.

Thanks!

One more thing I've noticed in the Web Prevent config under Server Detail-Advanced Server Settings:

Icap.LoadBalanceFactor=1

Anyone know what this is or how it can (or should) be manipulated?

Well personally, I've never done all that load-balancing stuff (no even between 2 servers ) but as you said, once the sessions remain in tact you shouldn't have a problem.

As for the Icap.LoadBalanceFactor=1 variable...there's no documentation on those advanced settings available from Symantec so I assume that you can't get anything official unless you've probably attended the Instructor-based training and asked them what it is.

I'm assuming that you have a pretty big organisation/deployment to be test running something like that (or just insatiable curiosity ). Why'd you opt for this model - 6 servers load balancing at the egress point of the whole network - as opposed to say deploying them singly or in pairs (for load balancing and redundancy I guess) at different sections of the network?

I'd encourage you to take care in manipulating it and share your results if you can because every bit of information can go a long way y'know?

Thanks for the response....

Having spoken with our sales rep, it looks like this is indeed possible (though not documented) as such.

By using the ICAP load balancing round-robin feature on the Bluecoats, we should be able to create a "farm" of Web Prevent  sensors for resiliency.

As for the test lab in our environment? VM's are your friend!  In fact, the actual Web Prevent Sensors will be VM's as well- which will make it easier to add more sensors as we add more proxies in the future.

In a way, we're almost treating the Web Prevent sensors like one would treat ProxyAV servers. Same ICAP concepts- just monitoring for data leakage rather than viruses.

Actually, those advanced settings are documented in the user documentation (available through the Help button).  Yes, you need to change that setting to indicate the number of web proxies that each Web Prevent server is going to communication with. 

Per the documentation for Icap.LoadBalanceFactor:

• The number of web proxy servers that a Network Prevent (Web) server is able to communicate with. For example, if the server is configured to communicate with 3 proxies, set the Icap.LoadBalanceFactor value to 3. 

I had a client who had configured multiple BlueCoat proxies to send traffic to multiple WebPrevent servers in this manner.  Their only real problem was that they then wanted to start inspecting more traffic based on the capacity that this provided (PUTS, GETS, etc), which did ultimately lead to latency issues as they approached capacity of the Web Prevent infrastructure, which forced them to have to do some creative filtering on the ICAP Service Layer in Bluecoat. 

So this definitely can be done, but be careful of what capacity you're ultimately providing and proclaiming you have, as the business side of the project will utimately ask you to inspect everything...slippery slope.

~Keith

GREAT stuff, Keith- thanks. 

And I feel quite meek for not checking the help file for descriptions of those advanced server settings...  :)

We are testing this out in the lab today, and I'll be sure to set the  Icap.LoadBalanceFactor to match the number of proxies reporting to it.

I understand the potential issues with load- and will be sure to share that with the users of the system in terms of wanting to  inspect more traffic. A slippery slope indeed.

Thinking about this....you said that the issue with that additional inspection (GETS/PUTS/etc) was realized " as they approached capacity of the Web Prevent infrastructure".  In this type of "farm" model though, wouldn't throwing more Web Prevent Sensors at the problem solve it?

Good things to think about of course. Thanks for the response.

Oh...I guess I must have missed it >_<

Thanks for the info about the documentation!

By the way, what were the specs you dished out to your VMs in the lab environment? Personally as a test I wouldn't want to allocate the "required" 500GB or whatever it is Symantec has in it's guide.

I'm planning on doing a test deployment similarish to yours but I wanted to know about how much I should expect to allocate. I had a virtualised endpoint server that was pretty much the same as a basic computer and it performed fine (in a test environment).

So yeah...what'd you do?

...but then they're going to want you to inspect more stuff, which means you'll need even more capacity, and so forth and so on.  My point is that you want to focus on the data that's most important from a data loss perspective (i.e. POSTs).  And if you're doing that, really you should not be inspecting all that much traffic as an overall percentage of all web traffic, so your infrastructure for Web Prevent should be able to stay rather lean and mean.

Yes, you could theoretically inspect everything, but what good is it doing you with regards to protecting your organization from data loss.  Use the tool for what it's made for, and let the other applications (i.e. your proxy) do what it's made to do, like web filtering, etc.

~Keith

Good points again, and agreed on restricting to POSTs. There is the edge case of posting via GET requests, and I completely understand/agree with it- but sometimes we need to adhere to the 80/20 rule- else go crazy trying to make everyone happy :)

xlloyd-  My VM's are waaaaay under powered that I'm using for testing. The VM sensors in the lab though have like 50GB hard drives, and 8 gig of memory- and not sure on the processor. To be honest, I'm going to take Symantec at their word for the load that will melt the servers- and be sure to add enough load balanced sensors in the farm to never get near that number....

If I had a perfect lab (in a perfect world), my testing would be different. Once I got the "farm" built out with properly spec'd VM's- I would  look to point a traffic generator at a lab load balancer which in turn fed to lab proxies, which connected to the farm. At that point, I would have a way to "crank up" the traffic to see at what point the Vontu Sensors melted- similar to how  they may work in the event of a datacenter failover.  Cool stuff!

Anyway- we ran through testing today and were successful in loadbalancing traffic through two test proxies through a "farm" of 3 Vontu Web Prevent Sensors. Worked well.

Heey good stuff. Congrats on the successful tests. Did you run into any issues? How'd you resolve them?

I've never even thought of doing that kinda testing though (the melting idea). I'll definitely try that out once I get mine up and running (been kinda busy lately). I asked about the specs 'cause I remember that Symantec doesn't support Network Prevent and Monitor on virtualised machines so if something goes wrong I guess you're on your own =P.