Video Screencast Help

NetBackup Master Hangs due to abnormal use of swap space.

Created: 12 May 2014 • Updated: 16 Jul 2014 | 9 comments
Bigenthy's picture
This issue has been solved. See solution.

He have one master server in the customer netbackup environment but we we noticed that some days the swap space increased and the master hangs. 
We are obliged to reboot the server. The version of NetBackup Software is 7.1.0.4. The server is a SUN Sparc T5220 (32 GB of Memory). I need solution.

Thanks

Ekra Thierry

Operating Systems:

Comments 9 CommentsJump to latest comment

rk1074's picture

Check if multiple bpdbm processes are running.. They often cause the swap space utilisation to peak.

 

Marianne's picture

Please download and install 'top' utility to see which process (or processes) is top of the list for memory usage.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

SymTerry's picture

It would be nice to see what processes are running. Also check this tech note: NetBackup appears to consume all the available memory in Linux, however this is not the case. It has a great explanation on how Linux uses and manages memory.

Bigenthy's picture

Hi, 

Thanks fot your reply. the are the messages before the server hangs 

Apr 16 19:09:37 Veritasmaster genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 21850 (nbsvcmon)
Apr 16 19:09:38 Veritasmaster genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 21848 (nbsvcmon)
Apr 16 19:09:38 Veritasmaster genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 21850 (nbsvcmon)
Apr 16 19:09:39 Veritasmaster genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 21848 (nbsvcmon)
Apr 16 19:09:41 Veritasmaster genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 21855 (nbsvcmon)
Apr 16 19:09:46 Veritasmaster genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 21862 (nbsvcmon)
Apr 16 19:09:47 Veritasmaster genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 21861 (nbsvcmon)
Apr 16 19:09:49 Veritasmaster sshd[1016]: [ID 800047 auth.error] error: fork: Espace insuffisant
Apr 16 19:09:51 Veritasmaster genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 21867 (nbsvcmon)
Apr 16 19:09:52 Veritasmaster genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 21864 (nbsvcmon)

 

Thanks, 

Ekra Thierry

Marianne's picture

Have you tried the top command yet to monitor memory usage?
What is swap size on the master?

Has shared memory been tuned on the master?

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Bigenthy's picture

Hi Marianne, 

 

The swap space is 320 GB. The server hangs aleatory. 

Marianne's picture

This is why I suggested using 'top' to always run in a terminal session.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

SymTerry's picture

Try swap -s. Your getting "no swap space to grow", this means you're out of memory. The "swap -s" command will list the size of virtual swap. (Physical swap added to the physical memory) On systems with plenty of memory, like yours, "swap -l" will show little or no swap space use but "swap -s" might give a better pictiure of swap space used. 

SOLUTION
Stumpr2's picture

Have you checked your buffers lately?

 

NET_BUFFER_SZ, SIZE_DATA_BUFFERS and NUMBER_DATA_BUFFERS - how they work and how to configure

http://www.symantec.com/docs/TECH1724

 

Also if you are using master as a media server
DOCUMENTATION: Tuning Solaris 10 shared memory for NetBackup Media Server processes (bptm / bpdm)

http://www.symantec.com/docs/TECH62633

Please NOTE: the associated links at the bottom of this technote.

VERITAS ain't it the truth?