A regression in VMware vStorage APIs for data protection can result in NetBackup erroneously reporting a successful VM backup or restore of an offline LUN when using SAN transport.

Article:TECH208628  |  Created: 2013-07-16  |  Updated: 2014-08-12  |  Article URL http://www.symantec.com/docs/TECH208628
Article Type
Technical Solution


Subject

Issue



Document history:
August 22, 2013: Original publication
August 30, 2013: Updated hotfixes and additional README information attached
December 16, 2013: Updated with "fixed-in" version releases
August 12, 2014: Public hotfixes removed - apply latest maintenance releases or upgrade to resolve this issue

VMware has documented that vStorage APIs for Data Protection could silently fail when using the SAN transport with an offline LUN:

VMware KB 2010428: VDDK 5.0 does not return error for SAN write to offline LUN
 http://kb.vmware.com/kb/2010428

In NetBackup, experiencing this issue could result in an undetected backup or restore failure (status code 0 displayed in Activity Monitor).


Error



No error is reported, yet the VM (virtual machine) backup or restore operation may have failed.


Environment



This issue is limited to NetBackup 7.5 through 7.5.0.6 (which use an affected version of VDDK [Virtual Disk Development Kit]) when using the VMware policy type configured to use the SAN transport during the VM backup or restore operation.  This includes NetBackup 52x0 Appliances using versions 2.5 through 2.5.3, which are based on these NetBackup 7.5.0.x versions.

For VM backups, the vSphere SAN datastore must be in an offline state to be affected by this issue.

For VM restores, the vSphere SAN datastore must be in an offline or read-only state to be affected by this issue.


Cause



As described in this excerpt from the VMware KB article: "VixDiskLib_Read() or VixDiskLib_Write() functions fail to return an error even though they cannot read from or write to an offline or read-only [SAN] LUN."

When operating in the environment described above, SAN reads or writes of the VM can be faulty, yet the backup or restore operation will report success.


Solution



The formal resolution for this issue (Etrack 3263410) is included in the following releases:

  • NetBackup 7.5 Maintenance Release 7 (7.5.0.7)
  • NetBackup 52x0 Appliances 2.5 Release Update 4 (2.5.4)

These releases are now available - more information and download links can be found in the Related Articles linked below.

These versions include a VDDK release which includes a VMware resolution for this issue.

Note: When it is released, NetBackup 7.6 will not be affected by this issue, as it will also contain a version of the VDDK in which this issue will have been resolved.

Workarounds:
Ensure SAN LUNs are not offline or read-only.

Alternately, a transport other than the SAN transport may be used - for example, LAN "network block device" (nbd).

If neither of these workarounds are feasible, please apply the appropriate hotfix attached to this document.

Note: This hotfix is no longer publicly available, as all of its fixes are included in the NetBackup 7.5.0.7 Maintenance Release and NetBackup Appliances 2.5.4 Release Update.  If any of these issues are experienced in a NetBackup 7.5.0.x or NetBackup Appliances 2.5.x environment, the supported resolution is to apply the latest maintenance release OR upgrade to NetBackup 7.6 (NetBackup Appliances 2.6) or above.

NetBackup 7.5.0.6:

Symantec Bug ID: 3256506

Installation Location: client

Installation Instructions:
Please follow the EEB Installer instructions available on http://www.symantec.com/docs/TECH64620

Package Contents:
Please choose the appropriate platform after download:

eebinstaller.3256506.6.linuxR_x86_2.6.18     RedHat x64 Installation
eebinstaller.3256506.6.linuxS_x86_2.6.16     Suse x64 Installation
eebinstaller.3256506.6.AMD64.exe             Windows x64 Installation
eebinstaller.3256506.6.x86.exe               Windows x86 Installation

Checksums:
3817182037 1317200 x86/bedsvmvcb.dll
2160205190 1645648 x86/diskLibPlugin.dll
355425267 1073744 x86/glib-2.0.dll
3346909338 287312 x86/gobject-2.0.dll
208615645 31312 x86/gthread-2.0.dll
1102299390 3685456 x86/gvmomi.dll
883871133 1123920 x86/iconv.dll
3365325793 88656 x86/intl.dll
1040360894 329808 x86/libcurl.dll
2449397781 1025616 x86/libeay32.dll
2198192787 144976 x86/liblber.dll
3566323835 297552 x86/libldap.dll
2393039215 318032 x86/libldap_r.dll
3153606157 1229904 x86/libxml2.dll
833305537 818000 x86/mounter.dll
25827692 218704 x86/ssleay32.dll
3065773753 24903760 x86/types.dll
255369343 4479832 x86/vcredist_x86.exe
2736243406 346960 x86/vddkWrapper.dll
4248570245 1173072 x86/vixDiskLib.dll
682634913 313936 x86/vixDiskLibVim.dll
1283382381 875600 x86/vixMntapi.dll
206010870 3704400 x86/vmacore.dll
1164972322 2385488 x86/vmomi.dll
1276953496 22768 x86/vstor2-mntapi10-shared.sys
834243998 69712 x86/zlib1.dll
968922764 7168 x86/preprocess_install_3256506.exe
10858062 7168 x86/install-3256506.exe
1118760859 830 x86/postprocess_3256506.bat
1792351107 433152 x86/bpVMutil.exe
1692142197 342016 x86/libfi_VMware.dll
199037012 290816 x86/libvix.dll
505953912 271872 x86/rvpext2fs.dll
1754277709 539648 x86/rvpntfs.dll
3879922516 1859 x86/vstor2-mntapi10-shared.inf
2801432938 7168 x86/post_uninstall-3256506.exe
3322295895 1524048 AMD64/bedsvmvcb.dll
368348121 2160208 AMD64/diskLibPlugin.dll
3857378431 1302096 AMD64/glib-2.0.dll
772127665 354384 AMD64/gobject-2.0.dll
1224261400 35920 AMD64/gthread-2.0.dll
2088844692 4816464 AMD64/gvmomi.dll
3079585603 1180240 AMD64/iconv.dll
3204672252 114256 AMD64/intl.dll
3031881913 399440 AMD64/libcurl.dll
2008255510 1299024 AMD64/libeay32.dll
43539793 165456 AMD64/liblber.dll
115672865 350288 AMD64/libldap.dll
2272825741 375376 AMD64/libldap_r.dll
3349693188 1717840 AMD64/libxml2.dll
3215534075 968528 AMD64/mounter.dll
1673003297 268880 AMD64/ssleay32.dll
1126718099 33261648 AMD64/types.dll
2956183000 5207896 AMD64/vcredist_x64.exe
1625799241 397648 AMD64/vddkWrapper.dll
112099186 1430096 AMD64/vixDiskLib.dll
2789043810 414800 AMD64/vixDiskLibVim.dll
2360515301 1224784 AMD64/vixMntapi.dll
2481799074 5753424 AMD64/vmacore.dll
755436839 3724368 AMD64/vmomi.dll
3879922516 1859 AMD64/vstor2-mntapi10-shared-amd64.inf
3848417279 33872 AMD64/vstor2-mntapi10-shared.sys
768405024 81488 AMD64/zlib1.dll
13740541 8704 AMD64/preprocess_install_3256506.exe
516557728 8192 AMD64/install-3256506.exe
2168685094 885 AMD64/postprocess_3256506.bat
2643541616 535552 AMD64/bpVMutil.exe
3296234322 403456 AMD64/libfi_VMware.dll
580628259 342528 AMD64/libvix.dll
2100283455 311808 AMD64/rvpext2fs.dll
3751040348 638976 AMD64/rvpntfs.dll
3128072005 8704 AMD64/post_uninstall-3256506.exe
104278973 1616456 linuxR_x86_2.6.18/libcrypto.so.0.9.8
1134635583 333224 linuxR_x86_2.6.18/libcurl.so.4
2531035222 3723088 linuxR_x86_2.6.18/libdiskLibPlugin.so
1395198451 443551 linuxR_x86_2.6.18/libexpat.so.0
3711320443 1016088 linuxR_x86_2.6.18/libglib-2.0.so.0
3181792990 17952 linuxR_x86_2.6.18/libgmodule-2.0.so.0
28178393 285488 linuxR_x86_2.6.18/libgobject-2.0.so.0
1804881668 22408 linuxR_x86_2.6.18/libgthread-2.0.so.0
3773724789 6403824 linuxR_x86_2.6.18/libgvmomi.so.0
2865668735 342376 linuxR_x86_2.6.18/libssl.so.0.9.8
3235983908 76081224 linuxR_x86_2.6.18/libtypes.so
2301309172 1872488 linuxR_x86_2.6.18/libvixDiskLib.so
2301309172 1872488 linuxR_x86_2.6.18/libvixDiskLib.so.5
2301309172 1872488 linuxR_x86_2.6.18/libvixDiskLib.so.5.1.0
1238265977 169624 linuxR_x86_2.6.18/libvixDiskLibVim.so
1238265977 169624 linuxR_x86_2.6.18/libvixDiskLibVim.so.5
1238265977 169624 linuxR_x86_2.6.18/libvixDiskLibVim.so.5.1.0
1608721486 2114104 linuxR_x86_2.6.18/libvixMntapi.so
1608721486 2114104 linuxR_x86_2.6.18/libvixMntapi.so.1
1608721486 2114104 linuxR_x86_2.6.18/libvixMntapi.so.1.1.0
1633197959 10161656 linuxR_x86_2.6.18/libvmacore.so
2419530967 6361216 linuxR_x86_2.6.18/libvmomi.so
1494362887 111 linuxR_x86_2.6.18/preprocess_install_3256506.sh
4284198479 731984 linuxR_x86_2.6.18/bpVMutil
4189736430 867040 linuxR_x86_2.6.18/libfi_VMware.so
1164639969 432489 linuxR_x86_2.6.18/libvix.so
86866960 433259 linuxR_x86_2.6.18/libvixST.so
3488957371 462591 linuxR_x86_2.6.18/librvpext2fs.so
723116152 462175 linuxR_x86_2.6.18/librvpext2fsST.so
1057137197 934775 linuxR_x86_2.6.18/librvpntfs.so
2534981876 934711 linuxR_x86_2.6.18/librvpntfsST.so
3237701403 98 linuxR_x86_2.6.18/post_uninstall-3256506.sh
104278973 1616456 linuxS_x86_2.6.16/libcrypto.so.0.9.8
1134635583 333224 linuxS_x86_2.6.16/libcurl.so.4
2531035222 3723088 linuxS_x86_2.6.16/libdiskLibPlugin.so
1395198451 443551 linuxS_x86_2.6.16/libexpat.so.0
3711320443 1016088 linuxS_x86_2.6.16/libglib-2.0.so.0
3181792990 17952 linuxS_x86_2.6.16/libgmodule-2.0.so.0
28178393 285488 linuxS_x86_2.6.16/libgobject-2.0.so.0
1804881668 22408 linuxS_x86_2.6.16/libgthread-2.0.so.0
3773724789 6403824 linuxS_x86_2.6.16/libgvmomi.so.0
2865668735 342376 linuxS_x86_2.6.16/libssl.so.0.9.8
3235983908 76081224 linuxS_x86_2.6.16/libtypes.so
2301309172 1872488 linuxS_x86_2.6.16/libvixDiskLib.so
2301309172 1872488 linuxS_x86_2.6.16/libvixDiskLib.so.5
2301309172 1872488 linuxS_x86_2.6.16/libvixDiskLib.so.5.1.0
1238265977 169624 linuxS_x86_2.6.16/libvixDiskLibVim.so
1238265977 169624 linuxS_x86_2.6.16/libvixDiskLibVim.so.5
1238265977 169624 linuxS_x86_2.6.16/libvixDiskLibVim.so.5.1.0
1608721486 2114104 linuxS_x86_2.6.16/libvixMntapi.so
1608721486 2114104 linuxS_x86_2.6.16/libvixMntapi.so.1
1608721486 2114104 linuxS_x86_2.6.16/libvixMntapi.so.1.1.0
1633197959 10161656 linuxS_x86_2.6.16/libvmacore.so
2419530967 6361216 linuxS_x86_2.6.16/libvmomi.so
1494362887 111 linuxS_x86_2.6.16/preprocess_install_3256506.sh
2831904728 744191 linuxS_x86_2.6.16/bpVMutil
3969173196 885459 linuxS_x86_2.6.16/libfi_VMware.so
4175676763 436663 linuxS_x86_2.6.16/libvix.so
2349745239 437273 linuxS_x86_2.6.16/libvixST.so
2452978913 468445 linuxS_x86_2.6.16/librvpext2fs.so
798210303 467997 linuxS_x86_2.6.16/librvpext2fsST.so
2859212700 944141 linuxS_x86_2.6.16/librvpntfs.so
2914166692 940333 linuxS_x86_2.6.16/librvpntfsST.so
3237701403 98 linuxS_x86_2.6.16/post_uninstall-3256506.sh

Recommended service state: Shut down all services on the master server before applying.

NetBackup 52x0 Appliances 2.5.3:
Please download the RPM attached below and access the Related Article linked below for instructions on applying the hotfix on an Appliance.

Best Practices:
Symantec strongly recommends the following best practices:

1. Always perform a full DR backup prior to making any changes to your environment.
2. Always make sure that your environment is running the latest version and patch level.
3. Perform periodic "test" restores.
4. Subscribe to technical articles.

How to Subscribe to Email Notification:
Directly to this Article:
Subscribe to this article by clicking on the Subscribe via email link on this page to receive notification when this article is updated with Release Information.

Software Alerts:
If you have not received this TechNote from the Symantec Email Notification Service as a Software Alert, you may subscribe via email and/or RSS using the links provided at the following page:
 http://www.symantec.com/business/support/index?page=content&key=15143&channel=ALERTS


Supplemental Materials

SourceETrack
Value3263410
Description

Update release to use the VMware 5.1 U1 VDDK.




Article URL http://www.symantec.com/docs/TECH208628


Terms of use for this information are found in Legal Notices