Video Screencast Help

Connection issue in basic replication settings

Created: 12 May 2011 • Updated: 12 May 2011 | 10 comments
This issue has been solved. See solution.

Hello guys, I am a newbi in vvr. I am trying to setup replication between three volumes but not able to do it after creating an RDS. I have done following things:-

1. Created two RHEL 5 nodes in vmware, installed volume manager on it.

Created following volumes on both node1 and node2:-

2. Attached three disks sda, sdb and sdc. OS installed on sda.

3. Created disk group ov-dg for sdba and sdc.

4. Created four mirrored volumes ov-volume-etc 1g, ov-volume-var 1g, ov-volume-lcore 1g and ov-volume-srl 3g

5. Created ext3 filesystem on ov-volume-etc, ov-volume-var and ov-volume-lcore

6. Added a + in single line in vi /etc/vx/vras/.rdg file on both nodes.

7. Created primary RVG on node1 : vradmin -g ov-dg createpri ov-rvg ov-volume-etc,ov-volume-var,ov-volume-lcore ov-volume-srl

8. Added a secondary using : vradmin -g ov-dg addsec ov-rvg rhela rhelb

Upto this step everything is good. I can see an RVG ov-rvg created on node2 also. But when I start replication I get following error:-

vradmin -g ov-dg -a startrep ov-rvg rhelb

VxVM VVR vxrlink INFO V-5-1-5298 Unable to establish connection with
remote host rhelb, retrying.

 

I have tried doing this many times deleted and recreated RDS but same error all time. I dont have any firewall enabled. I can telnet both nodes on heartbeat port. Passwordless ssh is enabled but still can't start replication.

 

Can anyone please guide me what is wrong here? It would be really appretiatable.

Note that I cannot make filesystem using vxfs format as it throws error when creating so I have used ext3 format.

 

Regards,

Aditya

Comments 10 CommentsJump to latest comment

Gaurav Sangamnerkar's picture

can u paste following outputs from both sites:

# rpm -aq | grep -i vxvm

# vxprint -qthg <diskgroup> 

# vxprint -Pl

# vradmin -g <diskgroup> -l printrvg <rvg>

 

G

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Gaurav Sangamnerkar's picture

one more thing, are your hosts rhelb/rhela resolvable ?  If yes, is it through hosts file or DNS ?

 

G

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

mikebounds's picture

This does look like a firewall issue - are you sure UDP ports are not blocked.  Note being able to telnet to port only means TCP is not blocked.  I had a problem getting VVR working on RHEL5 in VMWare and I had the following entries in iptables:

 

-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8199 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8989 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 4145 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 4145 -j ACCEPT
 
But it still didn't work until I deleted "RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited" so that everything goes through.  As its only VMWare I didn't investigate any further
 
It could be host resolution, but the fact that vradmin addsec worked, means rhela can resolve rhelb, but you should check that rhelb can resolve rhela.
 
Mike

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has helped you, please vote or mark as solution

adityazone's picture

Hi Gaurav,

Both hosts are resolvable via hosts file.

Below is the output from both nodes:-

rhela:------------------------------------------------

[root@rhela ~]# rpm -aq | grep -i vxvm
VRTSvxvm-5.1.100.000-SP1_RHEL5
[root@rhela ~]# vxprint -qthg ov-dg
dg ov-dg        default      default  4000     1305224572.10.rhela

dm ov-dg01      sdb          auto     65536    16703232 -
dm ov-dg02      sdc          auto     65536    16703232 -

rv ov-rvg       1            ENABLED  ACTIVE   primary  3         ov-volume-srl
rl rlk_rhelb_ov-rvg ov-rvg   DETACHED STALE    rhelb    ov-dg     rlk_rhela_ov-rvg
v  ov-volume-etc ov-rvg      ENABLED  ACTIVE   2097152  SELECT    -        fsgen
pl ov-volume-etc-01 ov-volume-etc ENABLED ACTIVE 2097152 CONCAT   -        RW
sd ov-dg01-01   ov-volume-etc-01 ov-dg01 64    2097152  0         sdb      ENA
pl ov-volume-etc-02 ov-volume-etc ENABLED ACTIVE 2097152 CONCAT   -        RW
sd ov-dg02-01   ov-volume-etc-02 ov-dg02 64    2097152  0         sdc      ENA
pl ov-volume-etc-03 ov-volume-etc ENABLED ACTIVE LOGONLY CONCAT   -        RW
sd ov-dg01-02   ov-volume-etc-03 ov-dg01 0     64       LOG       sdb      ENA
pl ov-volume-etc-04 ov-volume-etc ENABLED ACTIVE LOGONLY CONCAT   -        RW
sd ov-dg02-02   ov-volume-etc-04 ov-dg02 0     64       LOG       sdc      ENA
v  ov-volume-var ov-rvg      ENABLED  ACTIVE   2097152  SELECT    -        fsgen
pl ov-volume-var-01 ov-volume-var ENABLED ACTIVE 2097152 CONCAT   -        RW
sd ov-dg01-03   ov-volume-var-01 ov-dg01 2097280 2097152 0        sdb      ENA
pl ov-volume-var-02 ov-volume-var ENABLED ACTIVE 2097152 CONCAT   -        RW
sd ov-dg02-03   ov-volume-var-02 ov-dg02 2097280 2097152 0        sdc      ENA
pl ov-volume-var-03 ov-volume-var ENABLED ACTIVE LOGONLY CONCAT   -        RW
sd ov-dg01-04   ov-volume-var-03 ov-dg01 2097216 64     LOG       sdb      ENA
pl ov-volume-var-04 ov-volume-var ENABLED ACTIVE LOGONLY CONCAT   -        RW
sd ov-dg02-04   ov-volume-var-04 ov-dg02 2097216 64     LOG       sdc      ENA
v  ov-volume-lcore ov-rvg    ENABLED  ACTIVE   2097152  SELECT    -        fsgen
pl ov-volume-lcore-01 ov-volume-lcore ENABLED ACTIVE 2097152 CONCAT -      RW
sd ov-dg01-05   ov-volume-lcore-01 ov-dg01 4194496 2097152 0      sdb      ENA
pl ov-volume-lcore-02 ov-volume-lcore ENABLED ACTIVE 2097152 CONCAT -      RW
sd ov-dg02-05   ov-volume-lcore-02 ov-dg02 4194496 2097152 0      sdc      ENA
pl ov-volume-lcore-03 ov-volume-lcore ENABLED ACTIVE LOGONLY CONCAT -      RW
sd ov-dg01-06   ov-volume-lcore-03 ov-dg01 4194432 64   LOG       sdb      ENA
pl ov-volume-lcore-04 ov-volume-lcore ENABLED ACTIVE LOGONLY CONCAT -      RW
sd ov-dg02-06   ov-volume-lcore-04 ov-dg02 4194432 64   LOG       sdc      ENA
v  ov-volume-srl ov-rvg      ENABLED  ACTIVE   4194304  SELECT    -        SRL
pl ov-volume-srl-01 ov-volume-srl ENABLED ACTIVE 4194304 CONCAT   -        RW
sd ov-dg01-07   ov-volume-srl-01 ov-dg01 6291744 4194304 0        sdb      ENA
pl ov-volume-srl-02 ov-volume-srl ENABLED ACTIVE 4194304 CONCAT   -        RW
sd ov-dg02-07   ov-volume-srl-02 ov-dg02 6291744 4194304 0        sdc      ENA
[root@rhela ~]# vxprint -Pl
Disk group: ov-dg

Rlink:    rlk_rhelb_ov-rvg
info:     timeout=500 rid=0.1150
          latency_high_mark=10000 latency_low_mark=9950
          bandwidth_limit=none
state:    state=STALE
          synchronous=off latencyprot=off srlprot=autodcm
assoc:    rvg=ov-rvg
          remote_host=rhelb IP_addr=192.168.1.11 port=
          remote_dg=ov-dg
          remote_dg_dgid=
          remote_rvg_version=unknown
          remote_rlink=rlk_rhela_ov-rvg
          remote_rlink_rid=0.0
          local_host=rhela IP_addr=127.0.0.1 port=4145
protocol: TCP/IP
flags:    write enabled detached inconsistent cant_sync disconnected

[root@rhela ~]# vradmin -g ov-dg -l printrvg ov-rvg
Replicated Data Set: ov-rvg
Primary:
        HostName: rhela <localhost>
        RvgName: ov-rvg
        DgName: ov-dg
        datavol_cnt: 3
        vset_cnt: 0
        srl: ov-volume-srl
        RLinks:
            name=rlk_rhelb_ov-rvg, detached=on, synchronous=off
Config Errors:
        rhelb: Pri or Sec IP not available or vradmind not running

[root@rhela ~]# ping -a rhelb
PING rhelb (192.168.1.11) 56(84) bytes of data.
64 bytes from rhelb (192.168.1.11): icmp_seq=1 ttl=64 time=5.17 ms
64 bytes from rhelb (192.168.1.11): icmp_seq=2 ttl=64 time=0.395 ms

--- rhelb ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.395/2.783/5.171/2.388 ms

 

rhelb:----------------------------------------------------

[root@rhelb ~]# rpm -aq | grep -i vxvm
VRTSvxvm-5.1.100.000-SP1_RHEL5
[root@rhelb ~]# vxprint -qthg ov-dg
dg ov-dg        default      default  3000     1305224725.10.rhelb

dm ov-dg01      sdb          auto     65536    16703232 -
dm ov-dg02      sdc          auto     65536    16703232 -

rv ov-rvg       1            ENABLED  ACTIVE   secondary 3        ov-volume-srl
rl rlk_rhela_ov-rvg ov-rvg   ENABLED  ACTIVE   rhela    ov-dg     rlk_rhelb_ov-rvg
v  ov-volume-etc ov-rvg      ENABLED  ACTIVE   2097152  SELECT    -        fsgen
pl ov-volume-etc-01 ov-volume-etc ENABLED ACTIVE 2097152 CONCAT   -        RW
sd ov-dg01-01   ov-volume-etc-01 ov-dg01 64    2097152  0         sdb      ENA
pl ov-volume-etc-02 ov-volume-etc ENABLED ACTIVE 2097152 CONCAT   -        RW
sd ov-dg02-01   ov-volume-etc-02 ov-dg02 64    2097152  0         sdc      ENA
pl ov-volume-etc-03 ov-volume-etc ENABLED ACTIVE LOGONLY CONCAT   -        RW
sd ov-dg01-02   ov-volume-etc-03 ov-dg01 0     64       LOG       sdb      ENA
pl ov-volume-etc-04 ov-volume-etc ENABLED ACTIVE LOGONLY CONCAT   -        RW
sd ov-dg02-02   ov-volume-etc-04 ov-dg02 0     64       LOG       sdc      ENA
v  ov-volume-var ov-rvg      ENABLED  ACTIVE   2097152  SELECT    -        fsgen
pl ov-volume-var-01 ov-volume-var ENABLED ACTIVE 2097152 CONCAT   -        RW
sd ov-dg01-03   ov-volume-var-01 ov-dg01 2097280 2097152 0        sdb      ENA
pl ov-volume-var-02 ov-volume-var ENABLED ACTIVE 2097152 CONCAT   -        RW
sd ov-dg02-03   ov-volume-var-02 ov-dg02 2097280 2097152 0        sdc      ENA
pl ov-volume-var-03 ov-volume-var ENABLED ACTIVE LOGONLY CONCAT   -        RW
sd ov-dg01-04   ov-volume-var-03 ov-dg01 2097216 64     LOG       sdb      ENA
pl ov-volume-var-04 ov-volume-var ENABLED ACTIVE LOGONLY CONCAT   -        RW
sd ov-dg02-04   ov-volume-var-04 ov-dg02 2097216 64     LOG       sdc      ENA
v  ov-volume-lcore ov-rvg    ENABLED  ACTIVE   2097152  SELECT    -        fsgen
pl ov-volume-lcore-01 ov-volume-lcore ENABLED ACTIVE 2097152 CONCAT -      RW
sd ov-dg01-05   ov-volume-lcore-01 ov-dg01 4194496 2097152 0      sdb      ENA
pl ov-volume-lcore-02 ov-volume-lcore ENABLED ACTIVE 2097152 CONCAT -      RW
sd ov-dg02-05   ov-volume-lcore-02 ov-dg02 4194496 2097152 0      sdc      ENA
pl ov-volume-lcore-03 ov-volume-lcore ENABLED ACTIVE LOGONLY CONCAT -      RW
sd ov-dg01-06   ov-volume-lcore-03 ov-dg01 4194432 64   LOG       sdb      ENA
pl ov-volume-lcore-04 ov-volume-lcore ENABLED ACTIVE LOGONLY CONCAT -      RW
sd ov-dg02-06   ov-volume-lcore-04 ov-dg02 4194432 64   LOG       sdc      ENA
v  ov-volume-srl ov-rvg      ENABLED  ACTIVE   4194304  SELECT    -        SRL
pl ov-volume-srl-01 ov-volume-srl ENABLED ACTIVE 4194304 CONCAT   -        RW
sd ov-dg01-07   ov-volume-srl-01 ov-dg01 6291744 4194304 0        sdb      ENA
pl ov-volume-srl-02 ov-volume-srl ENABLED ACTIVE 4194304 CONCAT   -        RW
sd ov-dg02-07   ov-volume-srl-02 ov-dg02 6291744 4194304 0        sdc      ENA
[root@rhelb ~]# vxprint -Pl
Disk group: ov-dg

Rlink:    rlk_rhela_ov-rvg
info:     timeout=500 rid=0.1146
          latency_high_mark=10000 latency_low_mark=9950
          bandwidth_limit=none
state:    state=ACTIVE
          synchronous=off latencyprot=off srlprot=autodcm
assoc:    rvg=ov-rvg
          remote_host=rhela IP_addr=192.168.1.10 port=
          remote_dg=ov-dg
          remote_dg_dgid=
          remote_rvg_version=unknown
          remote_rlink=rlk_rhelb_ov-rvg
          remote_rlink_rid=0.0
          local_host=rhelb IP_addr=127.0.0.1 port=4145
protocol: TCP/IP
flags:    write enabled attached inconsistent cant_sync disconnected

[root@rhelb ~]# vradmin -g ov-dg -l printrvg ov-rvg
Replicated Data Set: ov-rvg
Secondary:
        HostName: rhelb <localhost>
        RvgName: ov-rvg
        DgName: ov-dg
        datavol_cnt: 3
        vset_cnt: 0
        srl: ov-volume-srl
        RLinks:
            name=rlk_rhela_ov-rvg, detached=off, synchronous=off
Config Errors:
        no contact from Primary

[root@rhelb ~]# ping -a rhela
PING rhela (192.168.1.10) 56(84) bytes of data.
64 bytes from rhela (192.168.1.10): icmp_seq=1 ttl=64 time=1.37 ms
64 bytes from rhela (192.168.1.10): icmp_seq=2 ttl=64 time=0.328 ms

--- rhela ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.328/0.851/1.374/0.523 ms

adityazone's picture

Hi Mike,

 

Here is the output of iptables list command. Let me know if something's wrong here?:-

 

[root@rhelb ~]# iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere            udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere            udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:bootps

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             192.168.122.0/24    state RELATED,ESTAB
LISHED
ACCEPT     all  --  192.168.122.0/24     anywhere
ACCEPT     all  --  anywhere             anywhere
REJECT     all  --  anywhere             anywhere            reject-with icmp-po
rt-unreachable
REJECT     all  --  anywhere             anywhere            reject-with icmp-po
rt-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
[root@rhelb ~]# iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere            udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere            udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:bootps

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             192.168.122.0/24    state RELATED,ESTABLISHED
ACCEPT     all  --  192.168.122.0/24     anywhere
ACCEPT     all  --  anywhere             anywhere
REJECT     all  --  anywhere             anywhere            reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere            reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Gaurav Sangamnerkar's picture

ok, so here is the concern:

 

Config Errors:
        rhelb: Pri or Sec IP not available or vradmind not running

 

I can see you can ping the IP address so shouldn't be an issue .... can u make sure vradmind is running on both the hosts ? you can use below commands to start vvr daemons

/usr/sbin/vxstart_vvr stop

/usr/sbin/vxstart_vvr start

once this is done, make sure vradmind is running

# ps -ef |grep -i vradmind

 

In case above doesn't help, I would suggest this:

- try editing the config & set the local IP_addr to relevant io address rather than to 127.0.0.1, I believe this IP address is getting picked from /etc/hosts only ... try removing the loopback address from /etc/hosts for VVR specific name & check.

If changing /etc/hosts doesn't help, you can also try to change IP address using vxedit command

# vxedit -g <diskgroup> set IP_addr=192.168.x.x <rlink_name>

 

Hope this helps

 

G

 

 

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

SOLUTION
adityazone's picture

Thank you!!! so much Gaurav and Mike!!!!!! Changing the IP in hosts file from 127.0.0.1 to 192.168.x.x and restarting vvr daemon worked it.

[root@rhela ~]# vradmin -g ov-dg -a startrep ov-rvg rhelb
Message from Primary:
VxVM VVR vxrlink WARNING V-5-1-3359 Attaching rlink to non-empty rvg. Autosync will be performed.
VxVM VVR vxrlink INFO V-5-1-3614 Secondary data volumes detected with rvg ov-rvg as parent:
VxVM VVR vxrlink INFO V-5-1-6183 ov-volume-etc: len=2097152              primary_datavol=ov-volume-etc
VxVM VVR vxrlink INFO V-5-1-6183 ov-volume-lcore: len=2097152              primary_datavol=ov-volume-lcore
VxVM VVR vxrlink INFO V-5-1-6183 ov-volume-var: len=2097152              primary_datavol=ov-volume-var
VxVM VVR vxrlink INFO V-5-1-3365 Autosync operation has started

 

I have few last questions regarding filesystem:-

 

1. Is it necessary to format the volumes in vxvm format? I have done it in ext3.

2. Do I need to format srl volume also? Or vvr would adjust is automatically?

Gaurav Sangamnerkar's picture

Glad to have helped..

1. Is it necessary to format the volumes in vxvm format? I have done it in ext3.

-- I m not pretty clear with question, but to add here, yes VVR can replicate vxvm volumes only. Once you create vxvm data volume under a DG, that should be enough

2. Do I need to format srl volume also? Or vvr would adjust is automatically?

-- SRL volume would be completely managed by VVR, again you should simply create a SRL volume & tell VVR config the name of SRL volume (this vradmin addprimary would have done for you)

If you have any further doubts on this, would suggest you to open a new discussion & mark this thread as solved..

 

G

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

mikebounds's picture

I think you mean  Is it necessary to format the volumes in vxfs format? - the answer is no - ext3 or any filesystem or even raw is fine.

You should not format SRL as VVR uses its own structures on here.

Mike

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has helped you, please vote or mark as solution

adityazone's picture

Thanks again Gaurav n Mike!! This was my first thread posted on symantec forum and its solved at day 1 itself!!