Veritas Cluster Server (VCS) I/O Fencing parameters for racing (Solaris)
| Article:TECH51519 | | | Created: 2007-01-28 | | | Updated: 2008-01-12 | | | Article URL http://www.symantec.com/docs/TECH51519 |
Problem
Veritas Cluster Server (VCS) I/O Fencing parameters for racing (Solaris)
Solution
When communication between cluster nodes fails, causing the
cluster to be divided into sub-clusters, these sub-clusters start a race to grab
coordinator disks for data protection (VCS I/O Fencing). vxfen has a mechanism
that enables cluster administrators to give larger sub-clusters better odds to
win this race. This document describes the differences in implementation between
VCS versions and their tunable parameters.
Note: While this mechanism can be used to give larger sub-clusters much better odds to win the race condition, it cannot be used to guarantee that the larger sub-cluster will always win.
1. How to give the odds
Note: While this mechanism can be used to give larger sub-clusters much better odds to win the race condition, it cannot be used to guarantee that the larger sub-cluster will always win.
1. How to give the odds
Prior to 4.1 MP2
If the number of nodes in a
sub-cluster is less than the number of nodes leaving the original cluster, the
sub-cluster repeats reading the coordinator disks to delay the start of the
race. By default, the number of reads is calculated as cube of (the number of
leaving nodes). For example, if a 5-node cluster is divided into a 3-node and a
2-node cluster, the 2-node sub-cluster repeats reading coordinator disks
27 (= 3 cubed) times. A tunable parameter,
max_read_coord_disk, can be used to
change this value, as described later.
4.1 MP2 and 5.0 or
later
If the number of nodes in a
sub-cluster is less than the number of nodes leaving from the original cluster,
the sub-cluster waits for a number of seconds before joining the race. This wait
time is calculated as (the number of leaving nodes) x 5. For example, if a
5-node cluster is divided into a 3-node and a 2-node cluster, the 2-node
sub-cluster will wait for 15 (3 times 5) seconds. Tunable parameters
min_delay and
max_delay (4.1MP2), or
vxfen_min_delay and
vxfen_max_delay (5.0 or later), can be
employed to change this wait time, as described later.
2. Tunable parameters
The following parameters can be
specified in the file /kernel/drv/vxfen.conf. Use/tune these parameters
only in situations where you often see a larger sub-cluster losing the
vxfen race. Note that careful and ample testing is required to determine the
most optimal values for a specific environment.
Prior to 4.1 MP2
max_read_coord_disk:
The maximum number of times vxfen will loop reading coordinator disks. If
the calculated repeat count exceeds this limit, this value will be used
instead.
Default = 25
Min = 1
Max = 1000
4.1MP2
min_delay: The
lower limit of the wait time in seconds. If the calculated wait time is below
this limit, this value will be used instead.
Default = 1
Min = 1
Max = N/A
max_delay: The
upper limit of the wait time in seconds. If the calculated wait time exceeds
this limit, this value will be used instead.
Default: 600
Min = N/A
Max = 600
Limitations: The
implementation of 4.1 MP2 is a subset 5.0 (or later) implementation and has some
limitations. vxfen in 4.1 MP2 will "round down" the
min_delay and
max_delay values to a number that is a
multiple of 5. For example, if the calculated delay time is 20
and max_delay specified is 18, the wait
time value chosen will be 18. However, vxfen will only the wait for 15
seconds, ignoring the remaining 3 seconds. Therefore, to avoid confusion, it is
recommended that only numbers that are a multiple of 5 be specified.
Note: With 4.1 MP2, the
default and minimum values implemented for
min_delay are a bit confusing, as
they are both 1 and not a multiple of 5. Magic in the code prevents this value
from being rounded down to 0, so specifying 1 here - or using the default value
of 1 - is safe and will not be a problem.
VCS 5.0 or later allows more
granularity, so the delay can be specified in any number of seconds within
the minimum and maximum boundaries.
5.0 or later
vxfen_min_delay:
The lower limit of the wait time in seconds. If the calculated wait time is
below this, this value will be used instead.
Default = 1
Min = 1
Max = 600
vxfen_max_delay:
The upper limit of the wait time in seconds. If the calculated wait time exceeds
this, this value will be used instead.
Default: 60
Min = the value of
vxfen_min_delay
Max = 600
|
|
| Source | ETrack |
| Value | 563541 |
| Description | vxfen panics the larger cluster in a cascading split brain |
| Source | ETrack |
| Value | 524705 |
| Description | fencing shuts down the larger cluster after a splitbrain |
| Source | ETrack |
| Value | 524044 |
| Description | Two node cluster loosing race with one node cluster. |
| Source | ETrack |
| Value | 593089 |
| Description | vxfen policies for time based delay |
Related Articles
Legacy ID
288510
Article URL http://www.symantec.com/docs/TECH51519
Terms of use for this information are found in Legal Notices
Thank you.