Server Network Adapter Teams Broken - Network Traffic Failing - NIC / Team / Adaptive Load Balancing
Updated: 23 Jun 2010 | 33 comments
Hello All;
This is an FYI that we have discovered on our SEP 11.0 installations. If you are running a Network Adapter Team on the Server, we have found that anywhere from immediately to up to two weeks after installation, that all network traffic begins failing. We have traced it back to the Network Team on the Server. Typically two Intel Nics in either an Intel Express Team or Adaptive Load Balancing configuration. If we break the team, and configure only one adapter on the server, then network traffic returns to normal. Our Specs, Windows Small Business Server 2003. We have witnessed it specifically on the S5000VSA server board. Typical Symptoms are suddenly no-one can log into server. Remote Desktop to server connects, disconnects, connects, disconnects. Hope this helps someone scratching their heads. On the first one, we thought that the Switch was failing.
discussion Filed Under:
Comments
Yeah, we've experienced some of the same problems with our load-balanced terminal servers. Some part of SEP client seems to cause incomplete connections on complex network configurations.
In our situation, we have 2 terminal servers load-balanced to a single ip address. The terminal servers use a virtual loop-back adapter for direct-server return with our load balancer. After a period of 6-14 hours of use, the server becomes unstable, unable to login even at the console, and preventing all new terminal services connections and logging a 1006 event in the system log:
"The terminal server received a large number of incomplete connections. The server may be under an attack."
As soon as I removed SEP and re-installed SAV 10.1 the server ran stable again.
I only install the Anti-vrius and Anti-spyware client on the servers. I've tried disabling and enabling the firewall and any threat protection options. None of the changes improve the stability of the load balanced server with SEP installed.
Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint
Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint
Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint
same problem - NIC Team causes unpredictable network behavior.
I have two HP NC7771 Gigabit NIC cards teamed up using HP Network Configuration Utility.. when I deleted the team, disabled the 2nd card, and reconfigured the TCP/IP setting to only use one NIC, the server seems to be functioning normally so far and not killing my internet connection as it did before. (it would totally kill internet access for everyone in the company within minutes after booting the server)
is Symantec aware of the problem with servers using teamed network adaptors? It is mission critical that I have load balancing on my server, and to sacrifice this in order to get SEP not to kill my Internet access is, frankly speaking, ridiculous. Please forward this to you developers.
does anyone on this thread have the MR2 beta or tried it in this situation?
Paul Murgatroyd
Principal Product Manager, Symantec Endpoint Protection
Endpoint twitter feed: http://twitter.com/symc_endpoint
Not sure if this will help. Try editing the policy settings for Intrusion Prevention and checking the "enable excluded hosts" and adding the source IP(s) to the list of excluded hosts.
DoS attacks appear to be detected by Intrusion Prevention and the firewall exceptions don't affect the default (undocumented policies) in Intrusion Prevention.
@Lyle
https://symbeta.symantec.com/
It's been 9 months since this thread was active - I was wondering if there are any updates on this issue? I have Intel NICs that use load balancing on all our servers. I'm having intermittent failures with LAN communications on servers with SEP Client MR3. I'm not using anything other than AV/AS. No firewall, etc.
nope. We are still waiting for Symantec to get this fixed.... it's really a bummer.
MR4 was released today to general (non-Platinum) support people like me. The Symantec tech I was working with today (on a different issue) told me that MR4 will fix the issue with load balancing/teaming. However, a quick search of the MR4 release notes made no mention of teaming or load balancing.
What is the work around you all have been using for now? No NIC teaming? I can't afford to have servers dropping out, so unless we know MR4 is fixing this issue, I'm going to have to implement a workaround.
I spoke with a Symantec tech this morning who told me specifically that MR4 fixes the issue with teaming. I asked him why the release notes don't mention teaming or load balancing, and he said the issue wasn't specific to these, but had to do with the Teefer2 driver. I don't feel completely reassured. But if anyone wants to test and report back, have at it!
Teefer2 (SEP 11 MR3) and NIC teaming on HP e.g. DL 385 G2 do not get on - the network does not work at all! If NTP is removed from the client installation all is OK. Call open with Symantec and HP. HP have reproduced the issue. Ongoing investigation with HP and Symantec.
Anyone have the link to MR4 since it is not out yet? I will test it with Intel Teaming and let everyone here know if it is fixed, but I need a link to the download to do this.
It's been out since 12/17, but you'll need to enter your serial number on the Symantec site to get it.
David, I am trying to get a statement from Symantec on what teaming issues MR4 has fixed but the response is that there are no fixes for teaming in the MR4 release notes . . . Not sure why the tech you spoke to said there are fixes . . . especially without anything to back that statement up . .
Any further info appreciated
I think the tech I spoke with was mistaken. I asked why there was no mention of teaming or load balancing in the release notes, and he said that plus other issues were all resoved with improvements in the Teefer driver. But when I went back and looked at the release notes, in the "Components in this release" section, there's no mention of Teefer2 in MR4 either.
Question for those with this issue - do you have any additional components installed on your servers besides antivirus/antispyware? I am only using AV/AS, and didn't install NW Threat Protection (firewall, instrusion prevention) on my servers.
I tried MR4 with Intel NIC teaming and I am still having issues. With teaming off, issues go away.... so I am wondering how it is fixed??
Lyle,
I'd like to compare notes with you. I have Intel PRO 1000 MT Dual Port Server adapters in a few servers. I have the latest Intel v13.4 installed on one server, and an earlier v12.1 on a few others. Thus far, I've had only one server fail to load with network connectivity, right after I installed MR3. I immediately uninstalled SEP and haven't retried that one yet. The other servers are running MR4, and have been running OK with teaming (adaptive load balancing). All my servers are Dell PowerEdge.
-Dave
I'm running ver. 11.xxx driver. I'll update my driver and then try it again.... good to hear you are working with teaming with v13.4 ..... Hopefully it is FIXED!
Tonight I upgraded a client with SAV 10 to SEP MR4. They were waiting till I could confirm that teaming was fixed in SEP. The install went perfect, and yes, TEAMING is fixed now in MR4! The NIC's in the server I installed it on were Intel Quad GIG NIC's, all teamed together. So I guess we can put this issue to rest now. Thanks Symantec for fixing it!
we still have a teaming issue with the MR3 and MR4 firewall component.
The affected adapters are HP NC371i Multifunction Gigabit Server Adapters. Driver v. 4.4.15.0
When the firewall is installed, the NC371i adapters are no longer available in HP teaming.
So no change for our issue.
This has been reported to HP and Symantec, who are figuring out how they will work on this together.
Teefer2 and HP NIC Teaming failure
HP have identified the issue as an incompatibility betwwen the NCU and Teefer2. They have pointed the finger at SEP, as their findings show that Teefer2 changes the HP NICs identity in device Manager and the NCU can then no longer recognise them. Awaiting theoutcome of the meeting between HP and Symantec engineering.
Issue seen on HP ProLiant DL385 G2 with Server 2008 SP1
the conclusion form HP and
the conclusion form HP and Symantec is that the problem is with Microsoft due to the way in which adapters and protocols are bound together. There is hence no solution, leaving us with no guarantee that NTP will not break network connectivity in future.
Possible solution
http://support.microsoft.com/default.aspx?scid=kb;en-us;912222
Type: REG_DWORD
Values: 1 (enabled) 0 (disabled
Due to a change freeze we have not implemented yet but may be worth trying.
Would you like to reply?
Login or Register to post your comment.