Video Screencast Help

Three node cluster interconnect options?

Created: 10 May 2010 • Updated: 12 Jun 2010 | 4 comments
This issue has been solved. See solution.

Hello,
I'm going to be implementing two different three node clusters to provide HA for a number of java application servers and their hosted applications. The current plan is to use two nodes of each cluster as the master for a subset of applications and have the third node be failover for the first two nodes. Each node will be running RHEL 5.3 64-bit.

I've implemented simple two node clusters in the past and used cross over cables to enable the LLT interconnect. I'm curious as to how the interconnect can be configured in a three node setup. Is it a requirement that it be switched? If it must be switched and I use two interfaces per node, should each of the interfaces be cabled into a different switch? This would seem to cause the ISL in my switches to have to carry the LLT traffic, but using only one switch would be a single point of failure. Is using cross over cables in a meshed configuration a viable solution to interconnect in a three or more node cluster?

Any pointers are appreciated

Thanks

Comments 4 CommentsJump to latest comment

Gaurav Sangamnerkar's picture

Hi There,

Yes, switch would be recommended rather than using meshed cross over cables.

Each nodes should have 2 NIC for heartbeats (minimum)...  So here is your hardware requirements:

2 Private NIC on each host (you can use public NIC as low pri link for additional redundancy)
2 different dedicated switch for heartbeat, If not then 2 different VLAN's in 2 different switch.

For e.g  lets say your 3 nodes are A, B & C   & lets say 2 switches or VLANs are  X & Y

One private NIC from A, B & C will connect to X
Another private NIC from A,B & C will connect to Y

So you achieved redundancy for NIC, redundancy for switch.  Also connect the switches to separate power source

Hope this helps....

Gaurav

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Marianne's picture

This picture from the Install Guide says it all:
On the left is your traditional 2-node cluster and on the right how to connect more than 2 nodes.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

SOLUTION
Leigh Brown's picture

Hello,

I have tried to answer your specific questions :-

-- Is it a requirement that it be switched?
-- Is using cross over cables in a meshed configuration a viable solution to interconnect in a three or more node cluster?

This is the same question stated differently.  The answer is, you cannot use crossover cables for a three or more node cluster, and hence you must use one switch (or hub :-) ) per private LLT link.  The reason you cannot use crossover cables is because LLT must be able to broadcast its updates to every other node, on a particular private link.  You can't cable up crossover links to support this.

-- If it must be switched and I use two interfaces per node, should each of the interfaces be cabled into a different switch?

To be precise, each set of NICs that form a private link network should be connected to a different switch.

-- This would seem to cause the ISL in my switches to have to carry the LLT traffic, [...]

This is not actually true.  The private link networks are totally independent of each other, so if all the NICs on the first private network are connected to a single switch, there will be no interswitch traffic required (in fact two isolated switches will do the job nicely).  If you are using VLANs, then the VLAN used on switch A for private network A, does not need to be defined on any of the ports on switch B (and vice versa).

Regards,

Leigh.

shaggy62's picture

Thanks for all the answers and information. All the replies were very helpful and provided a clear path for the installation of my new clusters.