Video Screencast Help

BackupExec 12.5 and 2010 fails on CentOS 5.9

Created: 15 Jan 2013 • Updated: 07 Mar 2013 | 25 comments
AlJ's picture

We have been successfully running 2 Linux backups (CentOS 5) for years. Last week the administrator ran updates and broke the backups. The 12.5 agent aborts when you try to access it. I updated one of the servers to the 2010 agent (all patches), the agent does not abort, but also does not work properly. BE 12.5 can't connect to the Linux agent at all, BE 2010 seems to go into some sort of loop when trying to access the agent. I can see the initial [ROOT] entry but any attempt to display its contents results in a very long wait and eventual abort of BE 2010. I cannot try a kernal rollback until this weekend, so wanted to see if anyone has another idea.

Comments 25 CommentsJump to latest comment

Backup_Exec's picture

Hi

Try uninstalling remote agent and reinstalling 12.5 remote agent back and then check

Also ensure the root is part of beoper group as it should be added automatically when remote agent is installed

Ensure your opt\vrtsralus directory is given more permission and then check if that helps

I presume when you telnet on port 10000 vice versa it is working 

Additionally with BE2010 when using the agent try disabling the ipv6 on the centos and then check to see if that helps

CentOS is not on the compatibility list of BE12.5  and  BE2010:

http://www.symantec.com/business/support/index?page=content&id=TECH137682

Hope that helps

Thanks

Sameer

Don't forget to give a "Thumbs Up" or Mark as "Solution" if someones advice has helped you.

AlJ's picture

Same as Red Hat so compatibility shouldn't be an issue, has been working for years, until the last kernel update.  I did another debug display and found that the directory listing goes into an endless loop, finding the same files and folders over and over and over, I'm sure this is what eventually blows up BE. Will try uninstalling and reinstalling, but if you have any other thoughts please let me know.  Thanks.

david homborg's picture

i can confirm it is a kernel issue. the agent is not working on kernel 2.6.18-348.el5 . after booting into 2.6.18-308.16.1.el5 it is working again. it is reproducible by creating a new backup selection list and browse the server. after clicking [root] the gui freezes. when examining the output of  /opt/VRTSralus/bin/beremote --log-console it looks like there is an infinite loop browsing the contents of / . it starts with VX_FindFirst and will continue to loop through the contents of root with VX_FindNext . it never reaches  FS_NO_MORE .

 

St3phan's picture

I can confirm it, too. With the kernel 2.6.18-348.el5 we have the same problem, Backup freezed at 365 byte.  If you use a 32Bit Kernel 2.6.18-348.el5PAE there is no problem.

AlJ's picture

The problem system is actually RHEL and not CentOS, but the kernel version is the same. I wonder if it is a RHEL or Symantec issue and who gets to fix it?

pkh's picture

BE 12.5 supports RHEL 5.0 to RHEL 5.2.  See the SCL below

BE 12.5 Software (SCL)

If you are running these OS's, you can log a formal support case with Symantec.

However, if you are running newer updates of RHEL, i.e. newer than RHEL 5.2, or CentOS, then you are on your own because these are not supported.  From the title of your post, you are either running RHEL 5.9 or CentOS 5.9, both of these are not supported by BE 12.5.  Neither does BE 2010 support them.. Even BE 2012 does not support RHEL 5.9, only up till RHEL 5.8.  See the SCL's below

BE 2010|2010 R2|2010 R3 Software (SCL)

BE 2012 SCL

david homborg's picture

i think we know there is no official support for rhel/centos 5.9. on the other hand, more and more users will update their 5.8 servers to 5.9 and it would be a positive gesture from symantec to provide a patch (or workaround)  for this issue. in time they will have to support 5.9 anyway and to my opinion the possibility of entering an infite loop is a bug in the software anyway. It will put all the backup jobs on hold as a backup of a 5.9 system will hang infinite on 365 bytes. the bug is in the listing of the root filsystem (as stated earlier) and should be very easy to reproduce and fix.

CraigV's picture

David: Add this in as an idea on the link below:

https://www-secure.symantec.com/connect/backup-and...

Enough votes and it might be considered for a future release.

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

david homborg's picture

Hi,

 

thanks for the suggestion, but i don't think this fits in the category "ideas". I know 5.9 is not supported by symantec, but a simple bugfix would keep customers happy instead of ending up with lots and lots of frustrated customers who end up with  big issues after doing "yum update". what is more simple than debugging an infinite loop issue? it would take a programmer 15 minutes to fix i assume.

 

david

Vivek Jain's picture

If possible to test on a test system, try installing the following path & install RALUS ~

https://bugzilla.kernel.org/show_bug.cgi?id=33992

 

david homborg's picture

I dont' think this will help as the kernel version has not changed, only redhat patches have been applied during the upgrade. This also makes it more difficult to apply this patch to the running kernel. looking through the changelist , maybe this fix is related to the issue:

 

https://bugzilla.redhat.com/show_bug.cgi?id=814418

AlJ's picture

Does anyone know if BE2012 is affected? May just be time for an upgrade.

Captain Bash's picture

We too have this issue.

It's all very well saying that 5.9 is not supported... compatibility list... bla bla bla...

 

Please Symantec listen to your customers and GET IT SUPPORTED!!

 

I could understand if you were only going to support RHEL 6/7 on 2012, but a minor kernel update like this for a currently supported O/S should get support for current BE 2010 r3. Especially seeing as how there's quite a big fuss around the way the new BE 2012 works.

 

If microsoft can prolong support for XP, then you can prolong support for 2010 !

david homborg's picture

a workaround seems to be to use the 32 bits ralus agent (VRTSralus-13.0.5204-0.i386.rpm). I removed the 64bits package and  installed this package, manually created the directory  /opt/VRTSralus/data/ and reestablished the trust relationship.

after this the agent is working. you might have to install some extra compat* packages for the 32 bit agent to work.

it would still be nice if symantec could fix the issue in the 64 bits version... i did some research with strace, but could not find anything. must be some sort of memory issue i assume.

 

Captain Bash's picture

Yes looks like this may be a kernel issue then in fact:

 

Backed up 12522668 files in 1 directory.
 

Yeah. it's backing up the same 2 files in / over and over again.

 

Vicente González's picture

SOLVED

 

You have move library

mv /opt/VRTSralus/VRTSvxms/lib/map/libext2fs.so /opt/VRTSralus/VRTSvxms/lib/map/libext2fs.so.bak

You have create symbolic link to library S.O.

/opt/VRTSralus/VRTSvxms/lib/map/libext2fs.so -> /lib64/libext2fs.so.2

 

 
and restart the agent
 
/etc/init.d/VRTSralus.init restart
 
david homborg's picture

solved indeed! much better than the 32 bits solution.

 

thanks,

 

david

Captain Bash's picture

Whoo hoo!

Confirmed as working! Thankyou Vicente!!

 

Seems a bit of a dirty fix mind, but anyone from Symantec care to comment on why this works??

CraigV's picture

...they might not hey, especially if it is unsupported for now...

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

AJ51's picture

Can you explain the fix, step 2 seems to be referenceing a lib that has been renamed in step 1?

Captain Bash's picture

Step 2 makes a link to a library in /lib64

 

Here's how I applied the fix line by line:

 

~ > cd /opt/VRTSralus/VRTSvxms/lib/map/

/opt/VRTSralus/VRTSvxms/lib/map/ > ll

total 256
-r-xr-x--- 1 root bin  52942 Apr  4  2011 libdisk.so
-r-xr-x--- 1 root bin  72900 Apr  4  2011 libext2fs.so
-r-xr-x--- 1 root bin  60135 Apr  4  2011 libgfsp.so
-r-xr-x--- 1 root bin  52942 Apr  4  2011 librawp.so
 

/opt/VRTSralus/VRTSvxms/lib/map/ > mv libext2fs.so libext2fs.so.old

/opt/VRTSralus/VRTSvxms/lib/map/ > ln -s /lib64/libext2fs.so.2 libext2fs.so

/opt/VRTSralus/VRTSvxms/lib/map/ > ll

total 256
-r-xr-x--- 1 root bin  52942 Apr  4  2011 libdisk.so
lrwxrwxrwx 1 root root    21 Jan 29 11:30 libext2fs.so -> /lib64/libext2fs.so.2
-r-xr-x--- 1 root bin  72900 Apr  4  2011 libext2fs.so.old
-r-xr-x--- 1 root bin  60135 Apr  4  2011 libgfsp.so
-r-xr-x--- 1 root bin  52942 Apr  4  2011 librawp.so

/opt/VRTSralus/VRTSvxms/lib/map/ > /etc/init.d/VRTSralus.init stop

Stopping Symantec Backup Exec Remote Agent ....
Stopping Symantec Backup Exec Remote Agent:                              [  OK  ]

/opt/VRTSralus/VRTSvxms/lib/map/ > /etc/init.d/VRTSralus.init start

Starting Symantec Backup Exec Remote Agent ......
Starting Symantec Backup Exec Remote Agent:                              [  OK  ]

/opt/VRTSralus/VRTSvxms/lib/map/ >

 

Hope that helps

pentium3NL's picture

Dear Everyone,

Thank you for this sollution, this SOLVED our problem.
With symantec helpdesk they were still debugging on mountpoints when we made no changes there:

-- code

/etc/init.d/VRTSralus.init stop; sleep 5; mv /opt/VRTSralus/VRTSvxms/lib/map/libext2fs.so /home/user_admin/libext2fs.so.backup; ln -s /lib64/libext2fs.so.2 /opt/VRTSralus/VRTSvxms/lib/map/libext2fs.so; /etc/init.d/VRTSralus.init start
Stopping Symantec Backup Exec Remote Agent ...
Stopping Symantec Backup Exec Remote Agent: [ OK ]

Starting Symantec Backup Exec Remote Agent ......
Starting Symantec Backup Exec Remote Agent: [ OK ]

A Gilmore's picture

[Edit]

This postponed the issue for me, but did not result in working backups. /lib64/libext2fs might have the same name, but does not provide the same code at all. 

Therefore, I'm having to downgrade my kernel version to get backups again. Boo.

 

[Old post below

This also solved this issue for me, after much hair pulling and concern. Breaking stuff in stable updates is pure suckiness. Vendors packaging obsolete copies of system libraries... also sucks.

It'd be nice if this got more visibility from both Redhat and Symantec for potentially impacted folks.

Kernel version 2.6.18-348.1.1.el5

Redhat Enterprise Linux 5.9

 

I've not explored if the 348.3.1 kernel build fixes this problem.]

 

xeless_it's picture

We had got the infinite loop too with backup exec 2010 R3, and Fedora 19, kernel 3.x .

It started when we give a new drive to the linux. The BE couldn't list the folder.

Solution in our case:

reformat the ext3 fs to ext4 fs.