Video Screencast Help

clients sending large amount of data to SEP server

Created: 09 Jan 2013 | 21 comments

It seems our clients are sending a large amount of data to the SEP server. Last night for instance we observed one client that sent 1.2 GB of data to the server. I think this was after a new/fresh install of the 12.1671 client version.

We do have a GUP on that site, which I think helps reduce traffic from the SEP server to a local server and then to the client, but not the other way around.

Is that much communication normal? That seems way out of line with what I would expect.

Comments 21 CommentsJump to latest comment

.Brian's picture

Are you sure it wasn't the other way around, perhaps updates being sent to clients?

I can't see a client sending that much info to a SEPM...

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

rpatty's picture

Pretty sure, yeah. I asked the network guys the same thing, and they insisted it was from the client to the SEP server, and are of course demanding answers for why these clients are generating that much traffic.

Hypothetically, even if they've got it flipped, isn't 1.2 GB a pretty big transfer? Definitions shouldn't be that large, are they? And they should be coming from the GUP, not the main server, right?

.Brian's picture

Updates to multiple clients could be that big...

What port was this over?

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

sandra.g's picture

Seconding Brian's comment. Seems very unusual, unless that client has massive amounts of logs to send to the SEPM.

Not that I necessarily think that a past known issue is the cause, but I did want to point out that 12.1.671 is the very first release of 12.1, from July of 2011. May be worthwhile overall to move to RU2 (12.1.2015).

Best practices for upgrading to Symantec Endpoint Protection 12.1.2
http://www.symantec.com/business/support/index?pag...

sandra

Symantec, Information Developer
Installation, Migration, Deployment and Patching
User Protection & Productivity, Endpoint Protection

Don't forget to mark your thread as 'solved' with the answer that best help

rpatty's picture

I'll schedule an upgrade. I admit I'm struggling with the Symantec versioning notation, where my client says I've got 12.1.600-something, and the downloads page says 12.1.2 is available. I can't figure out what math makes 12.1.2 be newer than 12.1.600.

Cameron_W's picture

If it is indeed the client sending that much data to the SEPM the only thing I could think of that may send that much data would be having application learning enabled. Below is a link to application learning best practices.

 

http://www.symantec.com/docs/TECH134367

 

If I was able to help resolve your issue please mark my post as solution.

Ajit Jha's picture

Thumbs Up to Pete, its really unusual, the definition size is never such large, as you said its sending, means SEPM is recieving, so defifitions doen't comes under picture. All we can expect is the Logs.

As pete mentioned communication mode, i would recommend you to change to PULL Mode and Increase the Heartbeat Interval.

Configuring push mode or pull mode to update client policies and content

http://www.symantec.com/business/support/index?page=content&id=HOWTO80912&actp=search&viewlocale=en_US&searchid=1357792567172

 

Regard's

Ajit Jha

Technical Consultant

ASC & STS

rpatty's picture

Clients are in pull mode. Heartbeat is set for 1/hour, which I think is already decreased from the default.

Application learning WAS turned on at the site level. I don't think it was turned on for any of the group policies, though. I have turned it off, because I don't think we've got any interest in that feature. Is that on by default? I don't know how or why it would have been turned on otherwise. Not sure if that would make a difference, if the site had it on but the client groups had it turned off.

.Brian's picture

It is not on by default.

Was the traffic over TCP port 8014?

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

pete_4u2002's picture

if it is off on the group end, this should not matter?

is this the only machine that shows high bandwidth utilizing machine? is this GUP by any chance?

 

rpatty's picture

No, this isn't the GUP, and it's not the only machine. We had three boxes do this on the same night, all following new installs of the software. One at 1.2 GB, one at 1.1 GB, and one at 900 MB.

Trying to track down the network ports, gotta get ahold of the network guy for more analysis.

.Brian's picture

I just can't imagine that with logging and every feature turned on, the client would send this much data to the SEPM.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

SMLatCST's picture

 

Regarding the issue of a single client uploading 1.2GB to the SEPM however, have you taken a look at the client itself?  Is it all healthy (fully updated) and is it uploading Packet and Traffic logs to the SEPM?  If it is uploading these sorts of logs, then can you check if you have the FW Policy assigned and configured to log alot?  Do the affected clients see alot of traffic in general?

Also, it might be worth noting the that the communcations session between client and SEPM are initiated by the client (dunno if that helps in your network guys' investigations).

On another note, Symantec version numbers are fun.  The order is a tiny little bit clearer when you take a look at the full version number however (latest being 12.1.2015, and the older being 12.1.671), but you gotta be looking for it wink

Finally, I can confirm that the Application Learning checkbox at the site level is enabled by default, and that this doesn't do anything unless the Upload of Leanred Applciations is enabled in a group's Communications Settings.

GeoGeo's picture

Do you have logging on in all your firewall policy rules for those servers? Could increase traffic of logs being sent to SEPM. 

Please review ideas and vote there could be something useful :)

https://www-secure.symantec.com/connect/security/ideas

 

A. Wesker's picture

Hi,

Agreed.

Pull mode communication. Heartbeat interval to increase depending the quantity of managed clients you have as well.

Check the logs file set on the clients. It sounds really weird that you received that much data by simple threads sent by your SEP clients.

Indeed, do not set a firewall policy to log all traffic as it has been already mentioned. The product is not designed for that at all.

 

Kind Regards,

A. Wesker

rpatty's picture

Shouldn't be any firewall logging. The only rule we've got is "allow all traffic". We've got enough corporate firewall stuff set up that we don't use that feature.

Still trying to get the port for the traffic from the network guy - he's tied up with a big project this week, probably won't have it until Monday.

.Brian's picture

Are the clients consistently sending data or has it since stopped? One-time occurrence?

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

cconoc's picture

We are seeing this exact same issue in our environment. Individual clients will upload several GB of data to the server everyday, sometimes totalling in excess of 1TB per day for all clients. Traffic is definetly inbound to the SEPM servers and is on port 8014. We previously had the clients uploading logs to the SEPM servers, but have since disabled this, with no effect. When we fail over the management servers to our backup location, traffic to our main site drops down to around 10Mbps, and when we fail them back to the main site, traffic spikes back up to around 40 Mbps, sometimes spiking up to 70 Mbps.

 

Any ideas?

rpatty's picture

In our case, further investigation revealed the network analysis was backwards. The data was being sent from the SEPM to the client, rather than the other way around. The confusing monitoring system was talking about "ingress" and "egress" between two points in a way that wasn't making much sense. Traffic was over port 8014, which is definitions traffic.

So, that's part of the mystery solved. I'm still a little surprised that:

  1. client definitions are close to 1 GB - that's huge!
     
  2. data downloaded varied so much. With one at 1.2 GB, one at 1.1 GB, and one at 900 MB - that's a 33% inconsistencency between low and high values. This from two machines in the same office with SEP installed on the same day. Seems like they ought to be more similar.

Anyway, we also figured that another part of the problem was the GUP not delivering these updates, but that's because I think the clients were misfiled in the SEPM. Or rather, they weren't filed at all, and the default group made the definitions come from the SEPM.

I've been keeping an eye on the process for a few weeks, and everything else seems to either be a misfiled client or one case where someone retired the GUP and didn't let me know to set up a new one.

So I think our issue is solved. Cconoc, I don't really have any suggestions for you other than to check the other things they had me look at in the thread, like checking pull mode, that application learning is off, that the firewall isn't logging data, etc.

 

.Brian's picture

A full definition set varies but can be around 150Mb. 1Gb is stil way to big, maybe the defs are corrupt. I would ensure the GUPs are working correctly as they would take this load off the SEPM.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.