Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Missing Drive Path

Created: 19 Dec 2012 • Updated: 20 Dec 2012 | 35 comments
This issue has been solved. See solution.

Already did

OS can see devices (/dev/rmt entries)

tpconfig -d

Shows

some drives up and

some drives with missing path.. 

-->cfgadm -al

-->cfgadm -c configure

-----------------------------------------------------

-->Reset HBA

-----------------------------------------------------

-->Hard reset Drives in Library

-----------------------------------------------------

-->mt -t /dev/rmt/## status

output:

no such file or directory

----------------------------------------------------------

Cluster Envir

NBU 7.1

Solaris

Drives SAN attached

Comments 35 CommentsJump to latest comment

revaroo's picture

Ok so the device paths do not reference any hardware anymore. Either the hardware is now not presented at SAN level or you do not have persistant bindings set at the HBA level and they are now showing on another device path.

Delete the drives that show the missing drive paths: volmgr/bin/tpconfig -delete -drive <drive index>

then use volmgr/bin/scan to see if you can see the drives. If you can, volmgr/bin/tpautoconf -a will discover all devices and add them to NetBackup

note: volmgr is located on unix in /usr/openv/     on windows usually c:\program files\veritas\

 

 

Marianne's picture

 

mt -t /dev/rmt/## status

output:

no such file or directory

The fact that device path exists does NOT mean that OS can see the device.

Your OS has lost connectivity to the drive or else device path has changed due to non-existing Persistent Binding. Check /var/adm/messages for device-related errors.

Do 'devfsadm -C -c tape' to remove non-existing device paths.

Lookup 'Persistent Binding' for your HBA make/model. This will prevent device name change when system is rebooted.

 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Ah45's picture

devfsadm -C -c tape' to remove non-existing device paths.

does it remove all Drives with missing drive path, 
after that how can i add them back to netbackup..and does it effect active jobs, tape and DD jobs are running..now..

 

Thanks

 

 

Ah45's picture

Revaroo,

after I delete the drive with
volmgr/bin/tpconfig -delete -drive <drive index>

now I am unable to see drive with scan command

 

Need to run cfgadm -c configure???

mph999's picture

 

after I delete the drive with
volmgr/bin/tpconfig -delete -drive <drive index>

now I am unable to see drive with scan command

 

No - scan command will not have shown drives before you deleted the drive config.

scan has absolutly 100% nothing to do with the drive config shown in any tpconfig or vmoprcmd command.

cfgadm probably won;t help, I suspect the command you need is devfsadm.

Persoanlly I would delete all the drives on this server from NBU and from the OS (rm the /dev/rmt files) and then reboot and see what comes back (hopefully all the drives).

If after the reboot some drives are missing (at the OS level) then you have an issue between the OS and the drives, and thiss needs to be fixed first.

Once all the drives are visiable at the OS level, check each responds with :

mt -f /dev/rmt/xcbn stat

Once this is confirmed, run scan - are all the drives visible.

If yes, then reconfigure the drives in NBU and check that persistant binding is set.

If scan does not show all the drives, you may need to rebuild the sg driver, but this is not that likely.

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Yasuhisa Ishikawa's picture

mt -t /dev/rmt/## status

output:

no such file or directory

This means link target of /dev/rmt/## does not exist - OS can not see drive or OS can see drive through other device file. You said "OS can see devices", but it is unclear.

For first, you have to run "devfsadm -C -c tape"(or simply "devfsadm -C") as Marianne suggested to get OS recognize tape drives.

Then you have to rebuild sg driver configuration following this technote.

http://www.symantec.com/docs/TECH71073

Authorized Symantec Consultant(ASC) Data Protection in Tokyo, Japan

Yasuhisa Ishikawa's picture

After thar, you can see drives in scan command. Then, run Device Configuration Wizard("Configure storage devices") in GUI to configure tape drives.

Authorized Symantec Consultant(ASC) Data Protection in Tokyo, Japan

Marianne's picture

devfsadm -C -c tape will scan for devices, leave existing, valid device names, remove non-existing devices, create entries for devices with no valid device names. 

You really need to forget about NBU and troubleshoot at OS level. 
Device-related errors will be logged in /var/adm/messages.
Also ensure Persistent Binding is in place.
 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Ah45's picture

I run devfsadm -C -c tape

but still, I see some drives with missing drive path

in tpconfig -d

Yasuhisa Ishikawa's picture

As above, you have to rebuild sg driver configuration.

Authorized Symantec Consultant(ASC) Data Protection in Tokyo, Japan

Ah45's picture

i have rebuild the sg drivers, but still same thing 

usr/openv/volmgr/bin/driver
sg.build all -ml 9
rm /kernel/drv/sg.conf
sg.install
scan
sgscan

 

 

tpconfig -d

drives with missing drive path

Marianne's picture

devfsadm has nothing to do with NBU. If drives have new OS device paths, you need to re-config NBU devices.

Once you are 100% sure that ALL drives can be seen in OS, then delete the drives with Missing Path, restart NBU daemons, then ensure scan or sgscan can see all drives, then re-run Device Config wizard.

Do the NBU re-config when no backups are running.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

revaroo's picture

Delete your drives that show missing device paths. tpconfig -delete -drive <index>

 

Ah45's picture

After sg.build
do we need to restart netbackup
or ltid restart is fine

revaroo's picture

No only need to restart ltid after changing/adding devices at NBU level.

Ah45's picture

after rebuilt and running all these commands
now I am not seeing Drive with missing path in 
tpconfig -d and /dev/rmt/#cbn

Yasuhisa Ishikawa's picture

Can you see all tape drives in scan command?

Ifo so, go to next step. Delete drives in NetBackup(from GUI or "tpconfig -delete -drive <index>" as revaroo wrote), then run Device Configuration Wizard,.

Authorized Symantec Consultant(ASC) Data Protection in Tokyo, Japan

Ah45's picture

Device Configuration wizard in not wroking,
Applying confirguration is getting fail

 

Ah45's picture

$more ipfc.conf

 

scsi-binding-set = "fcp";

Marianne's picture

There is more to Persistent Binding that just that line.

Find the User Guide for your HBA on Emulex web site. See this one for example:

http://www-dl.emulex.com/support/solaris/620j/manual.pdf 

The Configuration File (lpfc.conf)

Note: The fcp-bind-WWNN, fcp-bind-WWPN and fcp-bind-DID driver properties do not
apply to a specific HBA. They are the global properties. These properties specify a list
of persistent bindings. Each entry in this list applies to a specific instance of an HBA.
You can only use one type of binding per adapter.
 
See table 4 for explanation for each of these methods with examples.
You need one line per drive, e.g.
fcp-bind-WWPN="21000020370cf8263:lpfc1t0";
 
Maybe easiest to use HBAnyware utility (also available on Emulex web site).

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Ah45's picture

Thanks Marianne for the info

I will ask my Unix team, to implement this,..

but, what else you suggest for the current situation...

Ah45's picture

is HBA persistant binding is equal to zoning,
if yes, then how can I check the zoning in solaris

revaroo's picture

speak to your SAN specialists in your company.

 

mph999's picture

You didn't follow my advice (or if you did, apologies, but you have not explained that you did).

As Maianne explained, forget NBU - the cause is not NBU (although it is affected).

You need to prove at the OS level that all the drives are visible.

Do exactly these steps, do not miss any out.

1. Remove the drives from NBU config

2. Remove the /dev/rmt/ files

3. Run devfsadm or reboot server

4. Run cfgadm -al  - do you see all the drives, if so, goto 6, if not goto 5

5. Run cfgadm -al -o show_FCP_dev , do you see all the devices

(If any are missing you have an OS / device issue) - if they are all there continue

5b.  Confirm all the drives have a device file in /dev/rmt

5c.  Confirm all the drives respond to this command 

 

 
/usr/openv/volmgr/bin/scsi_command -d /dev/rmt/0cbn
/usr/openv/volmgr/bin/scsi_command -d /dev/rmt/1cbn
/usr/openv/volmgr/bin/scsi_command -d /dev/rmt/2cbn
 
(Substitute in each device file, 0cbnm, 1cbn, 2cbn etc ...)
 
For each drive, the command should give details like this:
 
Inquiry data: removable dev type 1h HP      DAT72X6         B604
 

6.  Run scan, do you see all the devices, if not rebuild the sgdriver like this

7. (Follow these commands exactly ...)

 

cd /usr/openv/volmgr/bin/driver
cp sg.conf sg.conf.safe
cp sg.links sg.links.safe
../sg.build all   (should not need mt /-mt options, drives will be using WWN)
modunload -i $(echo $(modinfo |grep "sg (SCSA" |awk '{print $1}'))
mv /kernel/drv/sg.conf /kernel/drv/sg.conf.old
/usr/openv/volmgr/bin/driver/sg.install
 

 

8.  Run scan - do you see all the devices, if so, configure the drive in NBU - if not you have device issues.

Until scan shows the devices, do not run any other NBU commands.

 

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
SOLUTION
Ah45's picture

step 2
2. Remove the /dev/rmt/ files

 

I cant remove all the drives, its a very big environment, and everytime backup wil go on, 

I was wondering, trying failover, and if that works, then i can restart the current node..

revaroo's picture

You can delete the drive paths of the drives that shows as Missing Device Path in Netbackup (tpconfig -d), then continue with the steps. Either way, once you've got the drives back and working you will have to stop ltid so ALL of your drives on your server will have to be unused at that time!!!!

mph999's picture

Fair enough - in that case just remove the ones that aren't working ...

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Ah45's picture

Martin as i said in my earlier post
I already rebuilt sg.conf

 

 

i have rebuild the sg drivers, but still same thing 

usr/openv/volmgr/bin/driver
sg.build all -ml 9
rm /kernel/drv/sg.conf
sg.install
scan
sgscan

 

 

tpconfig -d

drives with missing drive path

 

 

Ah45's picture

Good News, Guy
All drives up.

what I did

bp.kill_all--shutdown netbackup

 

1. Remove the drives from NBU config (tpconfig -delete -drive <drive index>

                                       (tpconfig -l   ---> for drive index)

2. Remove the /dev/rmt/ files 

                                        ($rm /dev/rmt/#cbn)

3. Run devfsadm -C -c tape

 

cd /usr/openv/volmgr/bin/driver
sg.build all -ml 9 (where 9 is the highest lun)
 
rm /kernel/drv/sg.conf 
/usr/openv/volmgr/bin/driver/sg.install
 
devfsadm -C -c tape
 
sgscan
 
bp.start_all
---------------------------------------------------------------------------------------------------------------------------------
Plz guys suggest me, In marking Solution
 
My suggestion
Solution for Martin
for Others which I got few Information like
tpconfig -delete -drive <drive index>, devfsadm -C -c tape, persistent binding etc..
I will give thumbs up

 

Ah45's picture

please suggest, what command should I run on Other media server with SSO

to get them these drives

or I just need to run cfgadm -c configure  or Reset HBA

mph999's picture

For the solution you should mark the post that you think helped you the most.

I am not sure why you ran the sg.build command with the -ml 9 option, with SAN configure drives it shouldn't be necessary.  However, as it is working, best to leave it alone.

For the other media servers you should start a new post as a solution has been found for this issue that you opened this post for.

However, to start off ...

You need to zone the drives to the media servers.

Follow my steps above to make sure they work, exactly the same steps.

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Ah45's picture

 

4) if Target is a WWN
a) run the command ../sg.build all (make sure to include the two leading periods)
i) if the maximum Lun value above is 1 or higher, add a "-ml L" to this command with L being the highest value found from step 1.

 

http://www.symantec.com/business/support/index?pag...

i got from

Ah45's picture

How can I build sg drives

 

in AIX Media server

mph999's picture

Fair enough, you weren't to know that the majority of the TNs, are 'generic'  - that is, cover most possibilities.  Creating the sg driver is actually quite easy, but then again, I've done it many times.  When you are perhaps less familiar with the exact details, it's easier to just follow a TN that should just make things work.

What I should have said, is follow my post and ignore any technotes.

However, to be honest, it doesn't really matter, as long as it is working and you are happy.

All that has happened in your case, is that by using -ml 9 it will craete a few extra device files, that aren't really needed, but won't cause an issue - my way is just a bit neater.

So all in, job done.

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Ah45's picture

Yes, I will keep that in mind,

Thanks alot for all your help...