Video Screencast Help

LLT WARNING V-14-1-10498 recvarpreq cross links? with VCS on VMware

Created: 29 Sep 2013 • Updated: 30 Oct 2013 | 3 comments
thstettler's picture
This issue has been solved. See solution.

We have started to receive the following warnings and messages about every 30 minutes or so.

 
Sep 29 10:45:27 node1 kernel: LLT INFO V-14-1-10205 link 0 (eth1) node 1 in trouble
Sep 29 10:45:27 node1 kernel: LLT INFO V-14-1-10205 link 1 (eth2) node 1 in trouble
Sep 29 10:45:29 node1 kernel: LLT INFO V-14-1-10205 link 2 (eth0) node 1 in trouble
Sep 29 10:45:30 node1 kernel: LLT INFO V-14-1-10499 recvarpreq link 1 for node 1 addr change from 00:50:56:A9:30:0E to 00:50:56:A9:30:0F
Sep 29 10:45:30 node1 kernel: LLT WARNING V-14-1-10498 recvarpreq cross links? links 0 and 1 saw the same peer link number 1 for node 1
Sep 29 10:45:30 node1 kernel: LLT INFO V-14-1-10499 recvarpreq link 0 for node 1 addr change from 00:50:56:A9:30:0F to 00:50:56:A9:30:0E
Sep 29 10:45:30 node1 kernel: LLT INFO V-14-1-10024 link 1 (eth2) node 1 active
Sep 29 10:45:30 node1 kernel: LLT INFO V-14-1-10024 link 0 (eth1) node 1 active
Sep 29 10:45:31 node1 kernel: LLT INFO V-14-1-10024 link 2 (eth0) node 1 active
 
 
Description of environment:

Four nodes running on an ESX each

Two heartbeats on eth1 and eth2

Low-Pri Heartbeat on eth0

Each heartbeat is connected to a distributed vSwitch and is then routed through the management net of the VMware environment.

 

[root@node1 ~]# cat /etc/llttab 
set-node node1
set-cluster 1
link eth1  eth-00:50:56:A9:30:0E - ether - -
link eth2  eth-00:50:56:A9:30:0F - ether - -
link-lowpri eth0 eth-00:50:56:A9:30:0D - ether - -
 
[root@node1 ~]# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:50:56:A9:30:0D  
          inet addr:xx.xx.xx.xx  Bcast:xx.xx.xx.xx  Mask:255.255.255.128
          inet6 addr: fe80::250:56ff:fea9:300d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:11414328 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14069031 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3814234814 (3.5 GiB)  TX bytes:11927457378 (11.1 GiB)
 
eth0:0    Link encap:Ethernet  HWaddr 00:50:56:A9:30:0D  
          inet addr:xx.xx.xx.xx  Bcast:xx.xx.xx.xx  Mask:255.255.255.128
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
 
eth1      Link encap:Ethernet  HWaddr 00:50:56:A9:30:0E  
          inet6 addr: fe80::250:56ff:fea9:300e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:52761246 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22113328 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4128460055 (3.8 GiB)  TX bytes:1701202284 (1.5 GiB)
 
eth2      Link encap:Ethernet  HWaddr 00:50:56:A9:30:0F  
          inet6 addr: fe80::250:56ff:fea9:300f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:62670760 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22112578 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4936739054 (4.5 GiB)  TX bytes:1411498489 (1.3 GiB)
 
The environment has been productive for four days. These messages have started today. During the test phase we did not encounter these warnings. The setup was not changed in the last few days, so it is a little puzzling why this should turn up all of a sudden.
 
I suppose because the heartbeats are routed through the level 3 network, the interfaces an now see each other and receive all of the heartbeats, thus causing these messages. Is there a configuration in VCS to handle this or do I have to get the VMware guys to change their setup?
Operating Systems:

Comments 3 CommentsJump to latest comment

mikebounds's picture

Have a look at http://www.symantec.com/docs/TECH78057 and https://www-secure.symantec.com/connect/forums/how-does-vcs-check-if-different-heartbeat-nics-are-same-network

Mike

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has answered your question then please click on "Mark as solution" link below

thstettler's picture

Hi Mike,

Thank you for your reply.I have read the two posts before and think I understand the problem. The question is, if I can do anything in VCS or if I have to hope the VMware guys can fix this.

Actually, I was wrong in my first post, the heartbeats are routed via the iSCSI network which is configured to be round-robin. So, I suppose, if there is a high load on the iSCSI the heartbeats may be routed this way or that way, thus causing the intermittent warnings.

I will need to talk to the VMware guys tomorrow.

Thomas

mikebounds's picture

Messages says that link0 and link1 are sharing the same network so you need to use separate networks for your heartbeats.

Mike

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has answered your question then please click on "Mark as solution" link below

SOLUTION