Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

NFS Export of Solaris, Non-Global Zone, Direct Mounted VxFS Filesystem

Created: 08 Jul 2014 • Updated: 09 Jul 2014 | 13 comments
This issue has been solved. See solution.

Solaris 10 Update 11

SF-HA 6.1 with VxVM update 100

I've got a direct mounted VxFS filesystem in a Solaris non-global zone that needs to be NFS shared.  This would be part of a failover group.

group dbprod1 (
        SystemList = { mia-sgz-p11 = 0, mia-sgz-p12 = 1 }
        AutoStartList = { mia-sgz-p11, mia-sgz-p12 }
        )

        Mount MNT_prod1_rpt1_arch01 (
                Critical = 0
                MountPoint @mia-sgz-p11 = "/export/zones/mia-sdb-p04/root/od/prod1_rpt1_arch01"
                MountPoint @mia-sgz-p12 = "/export/zones/mia-sdb-p05/root/od/prod1_rpt1_arch01"
                BlockDevice = "/dev/vx/dsk/prod1_dg/prod1_rpt1_arch01"
                FSType = vxfs
                FsckOpt = "-y"
                )

I've read over both 6.1 bundled agent guide and the virtualization gudie and cannot figure out how to get NFS Server to start in the non-global zones.  This service group will control one of two Oracle databses running in this local zone.  The databases have to be able to failover independently of each other.

Operating Systems:

Comments 13 CommentsJump to latest comment

mikebounds's picture

To start resources in a local zone, you set the ContainerInfo attribute on the service group, however I don't believe NFS and Share resources are Zone aware as virtualization gudie does not list these agents in "Zone-aware resources" section.

I believe there are limitations from a Zone point of view also - see:

http://docs.oracle.com/cd/E19683-01/817-1592/z.admin.ov-2/index.html which says

 Zones cannot be NFS servers

Mike

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has answered your question then please click on "Mark as solution" link below

DCress's picture

Sigh ...  I thought Oracle had gotten the NFS Server in a local zone working but after trying it by hand, no go.

All right, how about:

  1. Let VCS mount prod1_rpt1_arch01 somewhere in the global zone.
  2. Then have VCS 'share' the mountpoint out.
  3. Now VCS can do a directed lofs mount of prod1_rpt1_arch01 into my local zone.

Does that sound like it should work?  Not in the office so cannot try it myself right now.

mikebounds's picture

Anthing that works in Solaris, should work in VCS.  The post https://www-secure.symantec.com/connect/forums/mount-nfs-share-non-global-zone#comment-6871361 says you can't loop an NFS mount from the global zone into the local zone, but I would think a share would work.

I also found this link on Oracle:

http://docs.oracle.com/cd/E23824_01/html/821-1460/gljbf.html

Mike

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has answered your question then please click on "Mark as solution" link below

DCress's picture

The Oracle document 821-1460 refers to a Solaris 10 local zone running on a Solaris 11 global zone.  This is what confused me to start with. :)

My question becomes, how do I preform the following inside VCS so this file system can failover.?

Mounting VxFS as lofs into a non-global zone

# zonecfg -z newzone
zonecfg:newzone> add fs
zonecfg:newzone:fs> set dir=/mnt1
zonecfg:newzone:fs> set special=/mnt1
zonecfg:newzone:fs> set type=lofs
zonecfg:newzone:fs> end
zonecfg:newzone> verify
zonecfg:newzone> commit
zonecfg:newzone> exit

I cannot use the ContainerInfo attribute as it seems to require that both zones be the same name and it tries to shutdown the running zone when it does a failover.  I can't have that,  Just because we want to failvoer one databse in the zone does not mean we want to fail all of them over.

mikebounds's picture

If you have 2 databases in a zone and each database requires its own lofs mount then you can put Zone resource in a parallel service group and put each database with its lofs mount resources in a separate failover group dependent on the parrallel group.  See post https://www-secure.symantec.com/connect/forums/can-we-configure-vcs-51-ha-so-disk-groups-file-systems-related-one-application-solaris-10-non

Mike

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has answered your question then please click on "Mark as solution" link below

DCress's picture

And why oh why does the second NFS restart cause NFS to come up under SMF?  This causes everything to fail.

mia-sgz-l02# svcs -a | grep nfs
disabled       16:16:05 svc:/network/nfs/cbd:default
disabled       16:16:06 svc:/network/nfs/client:default
online         10:37:58 svc:/network/nfs/status:default
online         10:37:58 svc:/network/nfs/nlockmgr:default
online         10:37:58 svc:/network/nfs/rquota:default
online         10:37:59 svc:/network/nfs/server:default
maintenance    10:37:59 svc:/network/nfs/mapid:default

I disable all these resources, clean up and restart with the top NFSRestart disabled and everything works fine.

The NFS service:

 NFS NFS_dbtest (
                Critical = 0
                Protocol = tcp
                )

So this should not be using SMF, yet this grabage keeps restarting under SMF.

DCress's picture

I've gotten the my filesystem mounted in the global zone and lofs mounted into my local zone.

        Mount MNT_share (
                Critical = 0
                MountPoint = "/export/zones/data/db_share"
                BlockDevice = "/dev/vx/dsk/dbtest_dg/test_share"
                FSType = vxfs
                FsckOpt = "-y"
                )

        Mount MNT_share_zone (
                Critical = 0
                MountPoint @mia-sgz-l01 = "/export/zones/mia-slz-l01/root/db/share"
                MountPoint @mia-sgz-l02 = "/export/zones/mia-slz-l02/root/db/share"
                BlockDevice = "/export/zones/data/db_share"
                FSType = lofs
                FsckOpt = "-n"
                )

        MNT_share requires DG_dbtest_dg
        MNT_share_zone requires MNT_share

 

This is working great.  I just cannot get NFS working correctly in the Global zone.

DCress's picture

I've added my main.cf so you can see what all I've done. :(

AttachmentSize
main.cf_.bz2 1.34 KB
mikebounds's picture

Have you read the Notes for NFSRestart agent which say:

Service Management Facility

You need to enable the NFS attribute UseSMF to enable monitoring of NFS and
Lock daemons through Service Management Facility (SMF) in Solaris.
If the UseSMF NFS attribute is not enabled, you must disable SMF for NFS daemons
for the NFSRestart agent to work on Solaris. SMF is the service framework for
Solaris. SMF provides an infrastructure to automatically start and restart services.
Previously, Unix start-up scripts and configuration files performed these functions.
SMF maintains the Service Configuration Repository, which stores persistent
configuration information and runtime data for all the services. Thus, SMF now
controls all NFS locking daemons (lockd, statd, etc.) To keep these daemons under
VCS control, you need to modify the configuration repository to disable the SMF
framework for NFS daemons.

It then goes onto give you the commands you need to run

Mike

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has answered your question then please click on "Mark as solution" link below

DCress's picture

I've repeatedly "svcadm disable"  all the nfs asscoated services.  The NFS service has "UseSMF = false" , the default I've made sure the /etc/dfs/dfstab files on all systems is clear.  I've no idea why SMF keeps getting NFS when VCS tries to start my share.  My dependancy tree looks just like that in the NFS example sent with VCS and the Admin.pdf docuemntation.

        NFS NFS_dbtest (
                Critical = 0
                Protocol = tcp
                )

        NFSRestart NFS_restart (
                Enabled = 0
                Critical = 0
                NFSRes = NFS_dbtest
                )

        NFSRestart NFS_restartL (
                Critical = 0
                NFSRes = NFS_dbtest
                Lower = 1
                )

        Share Share_dbtest (
                Critical = 0
                PathName = "/export/zones/data/db_share"
                )

 

 

DCress's picture

My post keep vanishing.  I've disabled SMF NFS services, I've cleaned out /etc/dfs/dfstab, but everytime I restart NFS inside VCS I see SMF running NFS and I get the "?" over my NFS and NFSRestart services.

mia-sgz-l02# svcs -a | grep nfs
disabled       16:16:05 svc:/network/nfs/cbd:default
disabled       16:16:06 svc:/network/nfs/client:default
disabled       11:05:26 svc:/network/nfs/nlockmgr:default
disabled       11:05:26 svc:/network/nfs/server:default
disabled       11:05:26 svc:/network/nfs/status:default
disabled       11:05:26 svc:/network/nfs/mapid:default
disabled       11:05:26 svc:/network/nfs/rquota:default

        NFS NFS_dbtest (
                Critical = 0
                Protocol = tcp
                )

        NFSRestart NFS_restart (
                Enabled = 0
                Critical = 0
                NFSRes = NFS_dbtest
                )

        NFSRestart NFS_restartL (
                Critical = 0
                NFSRes = NFS_dbtest
                Lower = 1
                )

        Share Share_dbtest (
                Critical = 0
                PathName = "/export/zones/data/db_share"
                )

 

VCS-Tree.PNG
DCress's picture

OK, that document was half of what I needed, another doc shows disabling autostart on two more NFS services.  This has fxed all the problems I've been having in this thread.

https://sort.symantec.com/public/documents/sf/5.0M...

svccfg -s nfs/status setprop "application/auto_enable = false"
svcadm refresh nfs/status
svccfg -s nfs/nlockmgr setprop "application/auto_enable = false"
svcadm refresh nfs/nlockmgr
svccfg -s nfs/server setprop "application/auto_enable = false"
svcadm refresh nfs/server
svccfg -s nfs/mapid setprop "application/auto_enable = false"
svcadm refresh nfs/mapid

Mike,

Thank you for all your help on this, the wall next to my desk has a large head imprint. :)

SOLUTION
mikebounds's picture

Sorry, I only mentioned you need to follow notes for NFSRestart and as you say above you need to follow notes for the NFS agent too - but these are both in the same VCS bundled agent guide document.  You have referenced 5.0 bundled agents guide, but you should follow notes (for both NFSRestart agent and NFS agent) from 6.1 bundled agents guide, just in case there is a difference.

Mike

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has answered your question then please click on "Mark as solution" link below