Video Screencast Help

discussion but no real answer re: protocol ethernet type=0x806

Created: 15 Jan 2013 | 45 comments

We are at 12.1 RU2 on the servers - most of them anyway so I replaced the existing 11.xx and original early 12.1.xx firewall rules with the new, fresh default firewall supplied with the latest SEP/SEPM install for RU2. It had totally different rules. Allow local, then block, next rule was allow local, then block all the way down to "block all other IP traffic and log", then the last rule, "block all other traffic and don't log". This last rule is the question.

We are troubleshooting a lot of DNS and other issues - our servers are stating things don't exist that do exist, a lot of workstation logs are filled with "DNS server did not respond" type messages. I find the DNS servers never received the question, the client never received a reply and so on.
First suspect - FIREWALLS. Since there are BUILT-IN rules to allow server-type traffic, I wasn't really thinking it was an issue, but the folks helping us troubleshoot our on-going and vast issues with DNS and AD, I said what the heck, let's tell that last rule to LOG - I made no changes to the last rules, other than check logging on the very last "block all other traffic and don't log" rule.
The logs on the servers started filling up fast. All are showing constant hits on the last "block all other traffic" rule. Last I looked on just one server there were 12 hits on that last rule inside of 9 seconds, and ALL to and from the DCs, which are also our DNS servers. The folks working this issue don't like the fact SEP is blocking all this. So I must find out WHAT it is blocking and why, and what is the impact. So far, no replies to this question exist here.

The question - I have searched several times over the last year and see a lot of threads on this topic, but not a single one was actually SOLVED. OK, solved in "make the log entries go away" - sure, but that's like putting a blindfold over your eyes while someone takes your dinner - you just can't see it but your dinner is still missing. Maybe this rule blocks good traffic, but by not logging it, you simply can't see SEP doing it. That's what I need to know..... 
Please - I'm not wanting another "uncheck the logging and it will go away" replies, please. Well-meaning, well-intentioned, but it does nothing to address the real problem or question.

In a nutshell - WHY is SEP blocking this traffic  ETHERNET [type=0x806], and what is the result - what are the possible side-effects of SEP blocking this traffic?
Is there any harm or risk in allowing this traffic  it calls ETHERNET [type=0x806]?
What is ETHERNET [type=0x806] and why is there so much of it?
What is the impact on domain controllers and DNS servers if all ETHERNET [type=0x806] is blocked by SEP - like it is due to that last "block all other traffic and don't log" rule in SEP 12.1 RU2 

I am hoping that in this thread we can help dozens, even hundreds of others who have asked, but made it go away by simply not logging the blocking that still goes on even though they can't see it.

Thanks!

Comments 45 CommentsJump to latest comment

Rafeeq's picture

 

Hope this helps

Allow IP Traffic - This means that if any traffic doesn't match the firewall rule(either allow or deny) and if this option is checked then that particular traffic will be allowed. Sometimes we configure the firewall rule in such a way that we add only rules that allows a particular traffic and we do not add a rule at the bottom to deny traffic which doesn't match the above rules. If "Allow IP Traffic" is not checked then it will act as the bottom rule(deny all traffic)to drop all the packets except for the traffic which is generated from some applications(prompts user to allow or block traffic from that particular application). By default this option is checked so that it does not block all traffics other than specified in default firewall rule immediately after installation.

check the last suggestion

https://www-secure.symantec.com/connect/forums/eth...

ShadowsPapa's picture

Please note the rule is BLOCK, there is no allow, BLOCK ALL OTHER IP traffic, then below that, the very last or bottom rule is BLOCK ALL OTHER TRAFFIC.

I am using the default rules supplied with SEP 12.1 RU2

Thanks - but like I said, I've searched this for some time now, and I've seen that link - unfortunately,  there is no answer or solution in that or the other threads - it simply says to 'update the rules' and 'turn off logging' .

All that does is make it so you don't see the indication it's being blocked. However, it will still be blocked. Not logging doesn't mean it isn't being blocked.

I need to know why SEP is setup to block all  of this traffic, which GOOGLE says is ARP traffic ...............   interesting I have to use Google and go to other sites to find that out and not see it here in a network security forum  ;-)  ......... and what is the impact.

_Brian's picture

I "believe" ETHERNET [type=0x806] is an ARP request. Since ARP works below the TCP level, it is tied to the NIC.

Where is the traffic originating? What is the remote/local IP showing in the logs? Is is destined for the Boradcast address, 255.255.255.255? MAC would be ff:ff:ff:ff:ff:ff in the NTP traffic log...

It is possible to be ARP poisoned:

https://en.wikipedia.org/wiki/ARP_spoofing

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

ShadowsPapa's picture

No, no poisening. It's all local traffic. Like in my message - to and from OUR servers, including DCs, DNS, etc. It's done this for over a year, thus my comment that I've been searching for an answer here for months.
The IP and MAC addresses are all valid, and are all our own servers, the DCs and DNS and file servers - even the SEPM servers are being blocked from sending or receiving this traffic.

I've not seen a single entry related to anything other than our very own servers and traffic. That makes sense since we sit behind other firewalls.

_Brian's picture

Did you do a Wireshark capture?

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

ShadowsPapa's picture

We have not.......... since their time is quite expensive they didn't want to spin wheels until we knew - what is SEP blocking, why, and what is the possible impact. It's our traffic. I've had SEP in "sniffing mode" a while back (doable, you can even export the SEP logs and read them in wireshark - pretty cool)

Since this is a rule that is in SEP as it comes from the factory - meaning that I didn't put these rules there, Symantec did - they must have someone somewhere that knows firewalls maybe? - we must assume someone somewhere knows why the rule is blocking this traffic. What is the impact. If they put it in there, they must know the effects - I hope.

 I am tasked with finding out the answers to my original questions before the folks helping us determine our DNS issues, and AD resolution issues will return.

To clarify - 
I'm not attempting to see if we have malware on board - I just need to verify what SEP is or is not doing, and why. If SEP is blocking ARP - and if this traffic is needed for DNS - then SEP is what is breaking things.

Why does Symantec program SEP to block all ARP traffic on a network - nicluding server to server?

What does it break - if anything - what is the impact?

On the other hand - on this server I'm looking at now, SEP is blocking OUTGOING and INCOMING. If it's doing the same on the other servers - how could there be any incoming Ethernet 0x806traffic at all?
 

 

_Brian's picture

I turned on packet logging and I see two rules, both of which appear to be built-in:

one called Block_all

another called Build-in Allow All IP Traffic

What's interesting:

Block_all does what it says, blocks everything

Build-in Allow All IP Traffic BLOCKS all traffic as well, at least per the action in my log.

Either way, this needs to be clarified as you mentioned.

I just can't imagine ARP being blocked? How the heck would devices communicate than? We don't any issues with communication (that I know of) and SEP has been in our environment since it's inception.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

_Brian's picture

I created a rule to allow Ethernet type=0x806

I see traffic coming from clients on our local subnet 10.x.x.x going to our gateway 10.x.x.254\

If I do a Wireshark capture and compare time stamps with my NTP traffic logs, I can match them quite easily to verify it is ARP traffic.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

_Brian's picture

What stumps me is ARP is needed in a network otherwise PCs can't communicate so what is actually being blocked?

Lol, every time you post on here, you get my brain working over time...

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

ShadowsPapa's picture

Last sentence almost funny........... except in my case, it's literally genetic and mostly uncontrollable. It's a curse - and a blessing. when these things arise, I can "hyperfocus" to the extent that I forget to eat, sleep and other things. The phone can ring and I'll not even hear it. It's good for problem-solving, but bad in daily life (some day I will walk in front of a big bus as the poor driver blasts the air horn)

I have packet logging enabled, and it's arp alright. At least according to SEP. And funny thing - when I compare the logs of 2 servers, I see entries only 2 or 3 seconds apart (time difference can be several seconds at times even on a Microsoft network) - if SEP is blocking sending and receiving on server A, and SEP is blocking sending and receiving on server B, how can A see the packets from B to block them coming in - SEP is supposedly blocking B from sending to A, but A says it blocked ARP from B, B says it blocked ARP from A, but A says it blocked ARP going to B.

 

_Brian's picture

Yep, basically seeing the same thing here. However, if ARP was truly blocked, we'd have some major issues with device communication. Now I'm curious as to what's going on...

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

ShadowsPapa's picture

It's so totally strange - it says it is blocking. The same firewall is assigned to all servers. Our DCs are a bit more relaxed in other areas, but as far as the actual firewall itself, all servers have the same rules applied. 

How can another machine "see" a packet and block it according to the last rule if that same last rule prevents the first machine from even sending the packet.

I'm going to see about exporting the packet log - I turned on logging for the last/bottom rule "Block all other traffic and don't log" so I could try to see the packets - the actual traffic and see what the dickens is up.

Thank goodness Symantec was forward looking and left that feature in!

_Brian's picture

There are some built in rules, which I cannot find anything about. I'm not sure if they're undocumented or maybe just kept internally. I plan to place a call, probably Friday when it's slow.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

Shawn T.'s picture

Typically the ARP requests necessary for proper DHCP operation on a client are handled through built-in smart filters. If the client machine initiates an ARP request, the SmartDHCP rule allows that packet and the associated response through. When the firewall detects an unsolicited ARP packet it is left for the firewall rules to handle. What you see when you log the block all other traffic rule is a result of all of the ARP requests that do not originate from your client machine.

More information on the built-in rules may be found here: http://www.symantec.com/docs/TECH148648. Though the document refers to version 11.0, this still applies in 12.1.

In cases where the firewall is installed on a DHCP server, without a rule to allow Ethernet 0x806, you may see DHCP failures across the network. This may also affect certain VPN clients, so adding an allow rule may help there as well.

If your goal is to see what other traffic (non-ARP) is getting blocked, then I would set up a block and don't log rule for Ethernet 0x806 until you are done troubleshooting. This should allow you to find useful data in the logs generated by the default block rules at the bottom of the policy.

_Brian's picture

Shawn,

Out of curiosity, does SEP have undocumented built-in rules?

I enabled packet logging and two rules were showing up that I did not create nor were they in the default FW policy in the SEPM

One is called Block_all which is blocking

The other is called Build-in Allow All IP Traffic which according to the Action column is actually blocking traffic

I couldn't find anything in the KB about these and searches turned up nothing.

I opened a case with support but they don't seem to know much as it has been a few days with little response.

Any help you provide would be great.

Thanks

Brian

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

ShadowsPapa's picture

I agree and disagree with this:
>>What you see when you log the block all other traffic rule is a result of all of the ARP requests that do not originate from your client machine.<<

the way ARP works is that the initial "send" is a broadcast. The machine it is "aimed at" is the only one to reply. If I have IP address 1.2.3.4 I may send a broadcast looking for IP 1.2.3.4 - to see that no one else has my address. If there IS another on my LAN with the same IP, and it doesn't accept the broadcast ARP because it was not solicited (it did not ask for it!), then my computer will never know there is already a machine with the same IP - there would be duplicate IPs popping up here and there. Well, actually we do seem to have that issue now and then as well.

This would also mean that modern DNS servers which are also DCs on a Windows 2008R2 and later network would never be allowed to receive the ARP broadcasts and there would be a whole lot of troubles.
Our subnet that is servers would never allow ARP to work. No server could ever accept an ARP packet according to your description.

ARP broadcasts need to get through so that if my machine needs to contact another on my LAN or find information, it would mean seriously that ALL ARP would always fail.

You say ARP will be allowed since the firewall or even smart rules allow such things if it was asked for - it's contradictory since ARP is NEVER asked for. ARP caches would never exist and that actually explains why when I attempt to find the content of ARP caches on our local computers - there is little to nothing in them.

However, I have set up SEP on our servers to allow me to export to CAP format, and I've done a ton of lookng around and I see replies to pretty much all ARP requests - so according to the firewall it is blocking all ARP - in and out  literally. I check every firewall log on every server, all ARP traffic is always blocked, BUT, when I check sniffs, ARP is getting replies.

Something isn't adding up - somewhere the REAL explanation is missing.

 

ShadowsPapa's picture

I have come to believe that this is an area that:
A. No Symantec online support staffers know the answer to and are just as stumped and perplexed as all SEP firewall users are
B. This is an area that Symantec is holding very close to the vest and won't release the information as it's proprietary or corporate methods they don't want public
C. No one cares that folks are seeing this - it's viewed internally as a minor issue seen only by a few users and isn't causing any real harm or problems anyway.

If it is B. please at least tell us that much as we've got managers looking over our shoulders asking for progress reports and IT Enterprise waiting for us to find answers/resolution before they'll move on with further testing.

If it's A. then please, by all means, tell us that so we can move on and either find a solution in another product, or find someone somewhere who does know and can help.

I sincerely hope it's not C. Just because the masses are not in here hollering about systems that are totally down doesn't mean that it's not important or not major.  We have AD and DNS issues. SEP's firewalls are blocking things - and yet are also telling us things are getting through as there ARE replies to the packets. If it's blocked, then no device could see the question in order to know to send an answer - let alone WHO to send it to. If ARP isn't blocked, then why is SEP telling us it's blocked. DNS isn't working properly here. We get time-outs, failures, "no such domain" messages and so on. We have slow-downs, lags in service, etc. - something is amiss with DNS, and IT Enterprise is saying that we need answers to what the firewall is blocking. The firewall SAYS it's blocking ARP, traces tell me it's not blocking ARP as there are actual ARP conversations going on.
Further - I see and can refer you to numerous other posts/threads asking the same question, but the response is always a Microsoft sort of response sort of like the old joke where a helicoptor tour gets lost in Washington State and they come up to a building and ask the folks via a hand-written sign "where are we" and the response from the building is "in a helicopter". The building was Microsoft HQ.
I refer to that because this:
>>If your goal is to see what other traffic (non-ARP) is getting blocked, then I would set up a block and don't log rule for Ethernet 0x806 until you are done troubleshooting.<< was never the question. I can tell by the packet log what ELSE is being blocked, I want to know what the traffic is that IS being blocked, and why although it states "ARP" then are ARP replies getting through?
 

Here is a real life example. I'll show the SEP logs, and I'll show a bit of packet trace.

Here we see part of a packet trace showing ARP is getting through. You see an ARP request, src, then an ARP response - DEST, for 2 or 3 here.

Let's look at it logically - If the requests, which are a broadcast to ff-ff-ff-ff-ff-ff (to ALL addresses), were getting BLOCKED by SEP, and SEP says it IS blocking these, then how would the computer SEE or RECEIVE the request in order to respond with "that's me, that's my IP address, here is my MAC address. The sender knows the IP address, but needs the MAC. The recipient who has that IP address responds back to the sender with the answer -  the MAC for the IP. See here that there is a request -  xx.xx.xx.17 asks for xx.xx.xx.3 and xx.xx.xx.3 responds back to xx.xx.xx.17 "that's me, here xx.xx.xx.3 at my MAC address of 00-etc etc etc So we see at least SOME ARP is working fine - in fact, it's woking on this trace from one of our DCs

OK, here we have FIREWALL logs showing SEP is stating it is BLOCKING ARP->

Gee, what's up here. SEP says BLOCKING - it says it's ARP, it's destined for ff-ff-ff-ff-ff-ff, which is a broadcast to all.......... so if it's blocked, then how did the OS get the request in order to send THIS response back, which was successful on the other end ->

Sure looks like a response - but it was blocked, according to SEP.
So SEP says "I'm blocking all this via the "Block all other........" rule.
It says it blocked an incoming ARP request (makes sense since it never asked for the packet), it says it blocked the corrosponding ARP response (how can you respond to a question you supposedly never received?), and the trace itself above shows a successful ARP conversation - question and then a proper answer to the correct asker.

Which is it? Is SEP blocking ARP?
It SAYS it is.  But if it IS blocking ARP, then how can anything possibly respond? If I can't hear someone ask a question because the question was blocked from my ears because I never asked for a question to be asked of me, how could I respond to the question I never heard.
If SEP firewalls with the SAME rules are running on ALL computers, and the rule "block all other..........." is running and blocking anything coming in that wasn't asked for, then no computer on the network could ever possibly receive an ARP request, so it would never reply.
Yet I see queries and replies. All blocked but not blocked according to packet traces.
(If you eliminate the impossible, whatever remains, however improbable, must be the truth).

MAYBE the answer is so simple that it's difficult, or maybe it truly is so difficult that no one knows the answer.
.....but I need the answer one way or another as it may - or may not be - the basis or part of the basis for us having crazy DNS issues and problems. It must be ruled in - or out.

Since a week has passed, I assume and conclude there is no one who knows, or who finds the question interesting or curious enough to wish to reply or find the answer and respond.
 

_Brian's picture

I sent you a PM and that should give you the answer. I had my case escalated so we'll see.

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

pandher's picture

Hey guys, it is an interesting discussion and i too see a lot of ETHERNET [type=0x806] blocks on my logs. I really cant understand what is happening here. And in our environment we see a lot of network disconnects for few of our users. Last year we saw this issue and we werent able to find anything. Finally we have to disable the NTP completely on all users because it was really frustrating.

 

The network would disconect on the machine and everything will go offline and then it will come back up after a few seconds or a minute.

 

Now the management wants that to be enabled for security reasons and i am not sure what to do. Dont know if someone faced such issues...

_Brian's picture

Yes

Please click the "Mark as solution" link at bottom left on the post that best answers your question. This will benefit admins looking for a solution to the same problem.

Shawn T.'s picture

 

Hello ShadowsPapa,
 
Based on that log information I would have to agree with you. It doesn't fully make sense.
 
Here's what I do know: 1. WireShark inserts their packet filter driver "beside" the SEP firewall driver. This causes it to see things that don't always make it to the application level. (Beyond that I don't have an answer for the mysterious reply). 2. If you need to troubleshoot the DNS issues sans interference from SEP, create a rule to allow Etherenet type 0x806 and log or don't log it as desired. If there is no change, try disabling the firewall on the affected client. 3. Running Firewall or Intrusion Prevention components on High Availability or Critical servers such as Domain Controlers, Terminal Servers, Backup Servers and Mail Servers should be done with caution, ensuring proper rules are in place to permit the necessary traffic. (Also note that Intrusion Prevention can limit available bandwidth to a server. http://www.symantec.com/docs/TECH162135)

ShadowsPapa's picture

I am running the firewalls on the servers still configured as Symantec sent them so I've not added weird stuff in this case. The new practices according to Symantec are that the firewall - network thread detection in general, can be run on and are fine on servers now, unlike the earlier versions, 11.xx and so on which it was said "best not to do this". So it's default rules, nothing funky. Hope that clarifies that part a bit.

It's SEP that is stating the packets are coming in, and being replied to. The trace is just verifying.
To clarify a bit - the trace I show you is NOT wireshark but is in fact SEP logs exported to cAP format.
 the trace I showed above inside of the Microsoft Network Monitor product isn't from Microsoft Network Monitor, nor is it a WireShark trace but is actually an export from SEP itself.
So good or bad, we can't say that "wireshark inserts its driver....." as this is 100% SEP in this case. I leverage SEP a lot, maybe some folks don't realize SEP will capture packets and can export nicely to CAP and other formats for analysis. VERY nice feature, by the way!

I opened SEP, chose logs, opened the packet log on the machine, used the export function, exported from SEP firewall logs to a CAP format, then used Microsoft Network Monitor to open the trace taken by SYMANTEC ENDPOINT PROTECTION's own firewall.  Just want to make sure there's no other drivers involved, it's pure SEP this time.  Thus, what we see is SEP saying "hey guys, I'm blocking this incoming packet!"
Then we see sep telling us "Hey guys, I'm blocking the response to the packet I blocked to begin with"
................all the while SEP it telling us that it blocked an incoming ARP request, and telling us it's blocking the outgoing ARP response to the request SEP blocked, the originating computer does seem to be getting the response back to it's query - the query that SEP didn't allow  past the SEP firewall on the server to receive, and SEP didn't allow the server to respond to because it blocked all this ARP traffic. Why it would block an outgoing packet, that's another question.

Or did it?

SEP is lying, right?  ;-)  or is it just being clever?   If SEP is lying about this, what else isn't it doing, or is it doing but not really doing and telling us or not telling us about what it's not doing, or doing without saying?  LOL Now I'm even confused!
Gotta laugh about it or go crazy............ We must decide or determine what it REALLY is doing to move on.

Computer A sends an ARP request broadcast to Server 1. SEP on Server 1 blocked the ARP request packet(s). Server 1 responds to the ARP request packet anyway, even though it didn't get it because SEP blocked it. SEP blocks the response Server 1 sends back to Comptuer A, but SEP blocks the response out. Computer A receives the ARP response that SEP running on Server 1 blocked. Make sense? 

The lingering question is What is the bottom rule - "Block all other traffic and don't log" REALLY doing - anything? Why does it state it's blocking traffic when it clearly is not blocking said traffic? Or is it simply me not getting it (always a possibility to be honest)

I DID later run real live WireShark on our DCs which are also our DNS servers. All ARP traffic is moving successfully. HOWEVER, the pictures above, the ones I posted and said "packet trace" are in fact SEP firewall logs. The one that looks like a trace is Microsoft product reading a file I exported using SEP's firewall logs. No packet tracing product of any sort was installed on the machines at that time. Only SEP. Thus, since SEP was the ONLY product installed, there were NO network drivers other than Symantec's own, and the normal Microsoft NIC drivers. No other drivers. That means no other product was interfering or interacting, or causing confusion.

Ok, to address the >> If there is no change, try disabling the firewall on the affected client.<<
Normally I'd say you are spot-on. That is indeed proper troubleshooting, and I'd be all over it in most cases. However, we're talking obscure random events on over 300 computers. Some days it's 1 or 2 having several issues, other days they work ok and other computers have weird issues. Often the issue is "behind the scenes", the logs are loaded with DNS errors, and to the user it just seems "slow" or "can't get there", then it clears up. It's random, poltergeist-like in a way. So although I really do appreciate the tips and attempts - I only wish it was that simple.
the link you supplied doc 162135 or whatever, I'm going to get that and read - it might explain some other issues as well.

Overall, the SEP system is great, it's a grand package and works very well and I'm happy, it's the quirks, such as the ARP that it says is blocked, but obviously isn't or the server could not even know to respond.
Too often I get into things so deep even engineers need to bring in bigger shovels to sort through it for an answer. (just ask Jim Waggoner about the time back 10 or 11 years ago when I was with Principal and found a weird protocol issue with SAV CE - "no one else is seeing or reporting this".)

As always, I do appreciate responses, tips, attempts to help, etc. - even if it's not the answer that solves things, I appreciate anyone trying. Sometimes even a wrong answer gives me a lead to work on.

IPfreely's picture

I am unfortuantly seeing the same i have Ethernet 806 being blocked in the logs but everything is working as expected ...... well i think !

I have made numerous rules to allow this traffic but still see it in the logs as being blocked

with me i see my subnets gatewaty as the remote host and another machine on my network as the local host so since neither are my machine i understand it is being blocked but i have created rules to allow this to happen but i still get a block entry in the logs

it leaves me with the idea are all my rules working ?

if ARP is being logged i wonder what is not being logged .

now with in the Firewall Rules " Protection and Stealth "

i turned on

Now im thinking this will remove the ARP blocking log entries from the firewall log cause the ARP did not originate from my Machine or Host but my security log is as dry as a dead dingos donga !

and the blocks are still being logged to the friewall log

so since these requests are comming from my Subnet Switches i have created a host Group and used the MAC address of the switches and placed that into an allow remote host MAC address for Thernet 0x806 Incomming this appears to now allow the traffic or at least give the illusion that it is

i know this wont help you guys with 300 machine like hell you gonna put in 300 worth of MAC addresses

but since i only have a few switches per subnet i shose the smaller option ..

Still does not leave me with much confidance with regard to what is going on

 

 

 

 

 

Terriblted's picture

I have the very same issues with our new install of unmanaged SEP12 RU2.  I have an active Support case that is in the middle of being worked. 

I tried several workarounds even after reading the commentary above.  The log files for our server are full of our local network blocking reports. That makes it difficult to narrow on anything I am trying to follow.

I am thankful for finding the googled comments that I am reading. That gives me some relief, but no solution.

 

I may revert to my seriously outdated SEP 11.0 ---------- ARRGGHH.

 

 

Elisha's picture

SEP is not actually blocking Ethernet type 0x806 (ARP).  This is just a faulty log event from a rule migrated from SEP 11.  This will be resolved in a future release of SEP.

Here is a work around:

  1. Create a new firewall policy on SEPM under the "Policies" tab.
  2. Find the group where the clients are that have this issue and find out the name of the 'faulty' firewall policy.
  3. On the "Policies" tab replace the old firewall policy with the new one created in step 1.
  4. Delete the old firewall policy.

Alternatively you can use this work around:

  1. Find the group where the clients are that have this issue and edit the firewall policy.
  2. Find the rule that logs all blocked Ethernet traffic and turn off the logging for this rule.  This is generally the last rule in the rule set.
IPfreely's picture

I have a open case as well as they are unable at the moment resolve the issue

unfortunately for me Elisha resolution does not work in my instance as I have created a new firewall policy after upgrade and I still get the log entries but I do agree it does not seem that ARP is being blocked but how can I feel confident that the logs are showing me things that are being blocked or things that are not being blocked unfortunately leave me with no confidence at all

with the alternative work around it is not really a real case work around as to turn off logging would mean that you would not see anything else that might be blocked

I have had my call open for a few months now sort of lost track I think before Christmas I just hope it does not turn into another call logged that I get stringed along for 6+ months while support wait for a new version to be released in the hope that this resolved all issues.

Elisha's picture

I talked to the developer and he reviewed the code.  This issue only happens with ARP traffic.  All other log events can be trusted.  This is related to a defect in the "Smart ARP" feature of SEP.  This is a hidden feature, but it only affects ARP traffic.

ShadowsPapa's picture

So Elisha - where have you been hiding for all these months?

I suspect the "Smart ARP is a piece that allows only ARP requests destined for that computer to come through - all the rest is blocked, and yet it logs all ARP being blocked??

I thought I found a reference to something that did this very thing somewhere......but can't recall where I saw it now.

Elisha's picture

Hello ShadowsPapa, I have been around for a while, but I don't always have time to respond to the forum requests.  This one came to me by a developer.

"Smart ARP" is an internal feature that allows all ARP traffic.  Blocking ARP on a IPv4 network would not be good.  However we don't document this feature.  You may be able to see it if you open the xml profile on a client (but I did not check this).

The reason we log ARP traffic is for the Unmanaged Detector feature.  This features uses the ARP traffic to determine what systems are currently on the network.  You can enable/disable this feature from the "Clients" tab on SEPM.

If my suggestion above does not work you may want to try disabling/enabling the Unmanaged Detector on a client exhibiting this issue to see if that helps.

CornboyOde's picture

Well i have met the same thing here -  i have Ethernet 806 being blocked in the logs but everything is working as expected, at least as far as ia can see.

ShadowsPapa's picture

That's the first thing I thought of -  problem with firewall rules, so I took the default new firewall policy that gets installed when you install SEP 12 RU2. It's totally different, better structure, more logic, however, I found the SAME thing with it - it's the SERVERS and CLIENTS that this hits for.

I applied the new fresh unmodified firewall policy to our server groups. I had to enable logging as we had problems with Outlook/Exchange, DNS and other areas and they needed the logging to see what was blocked.

As soon as they saw the ARP stuff being logged as blocked, they stopped working and walked away saying that the blocking of these mystery things had to be resolved first.

Funny thing - I've seen this, countless others have seen this, and if this thread is any indication, several people have support cases open - and have had for quite some time now, but no solid resolution.

My thing is that - how can SEP log it as blocked, and yet ARP is being replied to? (I guess I already covered that above more than once - how can I answer a question that SEP says I never heard) Thus - our other problems won't be resolved, they won't come back until this issue is resolved or explained. If it was due to a firewall import from 11 - then why didn't removing the policy from 11 and putting in the new fresh policy from 12 help?

I am going to try again with a different group and see if it makes any difference. If it doesn't, back to the top post again, but if it does, then I'm going to be asking - why was this not caught with beta or regression testing with the lastest SEP? It's pretty obvious - and I suspect the odds are pretty decent that it is there for almost anyone who looks.

Terriblted's picture

protocol ethernet type=0x806

Been sendig log files to Symantec on our shared issue. We came to a conclusion that any unmanaged Enpoint client SEP 12 is problematic.

Solution:

Backup up OS, etc. Save Registry file externally.

I used CleanWipe and then ran CC Clean a couple of times, actually ran CleanWipe twice.

Re-install SEP 12, update, etc.

Result: no errors, so has to be the rule overlay as I had 7 total in my SEP 11 and autoload of 21 in SEP 12 clean install

Terriblted's picture

@ShadowsPapa:

I just commented upon my setup for small network.  I would bet that SEPM is not going to do the trick as Eliasa posted.  If I found twice as many default rules in a clean install and my original overlaid install on SEP 11 only listed seven, betcha everyone who reports the errors has same issue. Plus, who knows what type of conflicts are ongoing from leftover bits and pieces.

I went back to regedit, after using CleanWipe (from Symantec), and CClean, still finding traces of Symantec Endpoint.

Ted

Elisha's picture

This is not related to "leftover bits and pieces".  The issue is related to a firewall rule that is set to block ARP (Ethernet type 0x806) traffic and log the traffic.  Of course it does not block ARP traffic because of Smart ARP (mentioned above).

Somewhere in your ruleset you have a rule that is set to block Ethernet traffic and to log it.  Changing this rule to stop logging will resolve the issue.

One other workaround for this will work on both managed and unmanaged is to create a rule with the highest priority to allow ARP (Ethernet type 0x806) traffic and not to log it.  Make sure this rule is set to not log and it is the highest priority (the top of the ruleset).

If you are willing to send me your firewall policy I can tell you which rule is causing the issue and how you can fix it.

Terriblted's picture

@Elisha,

Start with my first posting, here, to follow the thread. Thanks for chipping it, extra eyes and opinions always help.

I did a long WebEx with Sam over the past couple of days, capturing troubleshooting log files for two parts of the issues I had with installing SEP 12 over SEP 11. The old rules were kept, and the new, default rules, imbedded in SEP 12 package were invilsible.

The ARP 0x806 issues could not be resolved in an unmanaged client situation. What you suggested, and others, simply did not work.

Sam sent me an email in Manage Case sector with the link to CleanWipe as well as the tech doc to manually remove traces of both versions. With that tool, I cleaned the table to install a fresh copy of SEP 12 for all my machines. They work wonderfully, now, no errors, and what appears to be less overhead. I had also used SymHelp in this session as well as Fixit by Microsoft while trying to solve the issue.

Thanks for being there, on a busy schedule.

Elisha's picture

Hello Terriblted, about your comments below.

The old rules were kept, and the new, default rules, imbedded in SEP 12 package were invilsible.

We don't add the default rules during an upgrade.  That is why they looked invisible because they did not exist.  We did not add them.  For an upgrade we simply keep the existing rules.  This is not related to the Ethernet 0x806 issue.

The ARP 0x806 issues could not be resolved in an unmanaged client situation. What you suggested, and others, simply did not work.

Which exact steps of mine did you try?  I think the key issue here is that an unmanaged client does not have the ability to disable the traffic log so you cannot create a rule to allow ARP (Ethernet 0x806) and not log.

One way to fix this, rather than removing the client and reinstalling the client would be to export the policy from an unmanaged client that is working correctly and then import it into the unmanaged client that has the issue.  You can do this using the "smc -exportconfig" and "smc -importconfig" options.  See this KB: http://www.symantec.com/docs/HOWTO27460

tocohel's picture

well I want to contribute to this because I got huge problems with a new system last week. While backuping with Backup Exec 2010 R3, it took 10h30 to backup 280gb over networking with Endpoint Firewall installed on all servers with default rules, exceptions to Backup Exec processes in Firewall, disabling scan backuped files and scan on accessing files.

yesterday the backup was again going slower as starting to backup over the network, i then decided to disabling Endpoint Firewall, here is the result in attachment, 2-3x times FASTER WITHOUT ENDPOINT........

i decided to no more use Endpoint Firewall but only Windows one because other clients had other issues with slow softwares over network, now I understand if even two apps from the same company don't work well together, what would happen with other companies...... thanks Symantec for such beautiful applications that you destroy years after years (yes i still use Backup Exec 2010 R3 because just don't even ask me about Backup Exec 2012 without monitoring or the possibility to backup all servers in one job... big joke

 

endpoint.PNG
Elisha's picture

Hello tocohel,

Thank you for your comments.  However, the issue you are experiencing is not related to this thread.  Your issue is related to the performance of the firewall.  The SEP firewall needs to check each packet and thus will cause slowdown in the network speed.  Normally this slowdown is not noticeable but for some heavy network applications (like backup software) the slowdown can be very noticeable.  The performance slowdown would happen on all applications and it not related to an incompatibility.  For backup we generally recommend not installing the firewall.  If you would like to discuss this more please open a new thread.

Thanks,

IPfreely's picture

Does anyone here on there client machines run LANDesk agents or more importantly LANDesk Agent with XDD installed ( Extended Device Discovery )

one of my main problems with symantec has been that my Local IP address keeps showing up in the logs

as the remote IP address so for exampole i need to place a rule in my logs that allows my server subnet to communicate with my client subnet cleint IP being local and server being remote but the logs show my ip being remote and server being local which obviously is not right .

I found that having LANDesk XDD installed caused this weird issue  and removing the XDD componant fixed this it also did this for wireshark too so.

for the ones who have LANDesk installed , if you dont and still have this issue then F'in Symantec !

 

intial testing this seems to be the case .

 

 

eteck's picture

Like everyone else on this post, I am having issues...Seeing Ethernet type 0x806 blocked between two DCs.  DNS is missing records.  I did migrate my firewall policy from 11 to 12 RU2 a long while back.  Likely this was going on the whole time, but I only noticed it when I was preparing to replace one of the DCs and found that the GC was missing from a server 2008 R2's DNS Server.  I added ports required for DC to DC communication and started looking at the firewall logs. 

What actually fixed this?  A lot of reading left me to think this isn't't even fixed yet.  I did already have the Built-In Rules enabled for Smart DHCP and Smart DNS.  I am getting an error when I check the DNS logs that says RPC Server cannot be contacted on one DNS server that's missing entries.  The ports required for RPC communication where within the DC to DC comm ports added a few days ago.  No change.

I need to keep all the rules that I've added in the firewall policy, thus making me less excited about creating a new policy.  Any suggestions?

Elisha's picture

Hello eteck,

Any logs you are getting about Ethernet type 0x806 can be safely ignored.  That is just a logging issue.  We are not actually blocking that traffic even though the logs seem to indicate that we are.

The DNS and DHCP issues are different.  From your comments it sounds like SEP was blocking some traffic and you created rules to allow that traffic.  Can you give more details on what you actually did?

mjmartino's picture

"Any logs you are getting about Ethernet type 0x806 can be safely ignored" These logs cannot be safely ignore until this issue has been resolved. I have over 300 clients that are having Arp related issues and the only thing blocking them is the symantec firewall. Even with it disabled it blocks and logs all arp traffic. We connect to IP-less machines via arp and it will continue to block us until I uninstall Symantec. Seems to work perfect then. I install AVG, works great. I install Mcaffee and again works flawless. Reinstall Symantec and once again no traffic passes. So you can claim it doesn't block traffic all you want but unless I have an app on my systems that blocks it only when symantec is installed, you are just either lying or do not understand how your product works. I am betting on the latter.

Elisha's picture

Hello mjmartino,

First of all let me say that I am very sorry to hear that you are having these issues.  Let me help to explain the issue here and what is happening.

The SEP client has an internal feature called "Smart ARP".  This feature is not exposed to our customers but it is an internal engine.  This engine runs before the firewall rule set is evaluated and its job is to allow all ARP traffic, because blocking ARP traffic is not good.

However we had an issue because our Unmanaged Detector (a.k.a. LAN Sensor) also needs to see the ARP traffic.  The Unmanaged Detector works by creating hidden firewall rules to log ARP traffic.  Since the Smart ARP was evaluated first Unmanaged Detector stopped working.

Therefore in a recent update (SEP 12.1 RU2) we changed the Smart ARP engine so that it was evaluated after the firewall rule set was evaluated.  This way the Unmanaged Detector rules would get triggered and the ARP traffic would get logged.  Once the firewall rule was triggered we would then log the traffic and engage the Smart ARP.  The result of this would be a logged ARP traffic which made the Unmanaged Detector work, while also allowing the ARP traffic.

But the issue is that sometimes customers would create a firewall policy to block ARP traffic.  In this case the flow is the same.  We run the firewall rules first logging the ARP traffic as blocked and then we engage Smart ARP so that the ARP traffic is not actually allowed.  Thus way will log that ARP traffic is blocked while in reality it is allowed.

From your comments it sounds like no IP traffic is working at all.  Of course if we were to block ARP traffic that would be the result of blocking ARP traffic.  But I am wondering if something else is happening in your case.  Would you be willing to help troubleshoot this with me?

The SEP firewall driver is installed as a "mandatory" driver.  This means that if there is something wrong with the SEP driver the OS will not allow any traffic to go in or out of the NIC.  You can read more about mandatory drivers from this link here.  I am wondering if this is what is happening for you.

Here are a few things I recommend trying:

  1. Can you tell me what version and build of SEP you are running?
  2. Can you check if the teefer or teefer2 driver is running?  You can do this by running the "sc query teefer" and the "sc query teefer2" commands from the Command Prompt.
  3. Can you let me know what type of network card you have installed on the system and if you have TCP checksum offloading enabled on the network card?
  4. Can you create a rule on SEPM with the highest priority?  Keep all fields blank and set the firewall rule to allow.  This will override any rule that is blocking ARP.

Thanks,