Video Screencast Help

NDMP Error Code 174

Created: 08 Oct 2013 • Updated: 15 Nov 2013 | 7 comments
This issue has been solved. See solution.

Hi! I'm back again with more problems. Thanks in advance for the help.

I am having NDMP backup and restore issues and I believe they are related.

The first issue is that after a job shows 100% complete and is no longer writing any data, it takes hours to complete and show that its status is done rather than active.

I am guessing this is a configuration issue.

I'm using NBU 7.5.04 running on W2008 R2. Backing up an EMC VNX 5700. I have allow multiple data streams selected under attributes and have set Media Multiplexing to 5 on the backup schedule. I am backing up to a disk system (EMC Clariion) and I think Multiplexing should be set to 1.

Backup Selections and environment settings are as follows:

set TYPE=tar
SET HIST=Y
SET UPDATE=Y
SET OPTIONS=NT
SET DIRECT=Y
SET SNAPSURE=Y
/STD7FS1/Design/

On this job I get the following from the details. I have removed multiple instances of the "Error ndmpagent" as it appears there are 3 lines of this error per second.

10/07/2013 09:10:21 - Info nbjm (pid=7740) starting backup job (jobid=22390) for client sv07data01, policy sv07data01_design, schedule Cumulative
10/07/2013 09:10:23 - Info bpbrm (pid=7796) sv07data01 is the host to backup data from
10/07/2013 09:10:23 - Info bpbrm (pid=7796) reading file list from client
10/07/2013 09:10:23 - estimated 133671586 kbytes needed
10/07/2013 09:10:23 - Info nbjm (pid=7740) started backup (backupid=sv07data01_1381162223) job for client sv07data01, policy sv07data01_design,         schedule Cumulative on storage unit SU_W
10/07/2013 09:10:23 - started process bpbrm (pid=7796)
10/07/2013 09:10:23 - connecting
10/07/2013 09:10:24 - Info bpbrm (pid=7796) starting ndmpagent on client
10/07/2013 09:10:24 - Info ndmpagent (pid=1756) Backup started
10/07/2013 09:10:24 - Info bptm (pid=6984) start
10/07/2013 09:10:24 - Info bptm (pid=6984) using 30 data buffers
10/07/2013 09:10:24 - Info bptm (pid=6984) using 262144 data buffer size
10/07/2013 09:10:24 - connected; connect time: 0:00:00
10/07/2013 09:10:26 - Info bptm (pid=6984) start backup
10/07/2013 09:10:26 - begin writing
10/07/2013 09:10:27 - Error ndmpagent (pid=1756) sv07data01: Active NDMP backup/restore streams reach the configured concurrent streams 4, but less than the maximum supported concurrent streams 8. No more backup/restore sessions can be started. Wait current backup/restore sessions finish, then increase concurrentDataStreams parameter.
10/07/2013 09:10:27 - Error ndmpagent (pid=1756) sv07data01: Active NDMP backup/restore streams reach the configured concurrent streams 4, but less than the maximum supported concurrent streams 8. No more backup/restore sessions can be started. Wait current backup/restore sessions finish, then increase concurrentDataStreams parameter.

10/07/2013 09:13:17 - Error ndmpagent (pid=1756) sv07data01: Active NDMP backup/restore streams reach the configured concurrent streams 4, but less than the maximum supported concurrent streams 8. No more backup/restore sessions can be started. Wait current backup/restore sessions finish, then increase concurrentDataStreams parameter.
10/07/2013 09:13:26 - Info ndmpagent (pid=1756) sv07data01: SnapSure file system creation succeeds

BTW, this job is still active although it hasn't't written anything for at least 2 hours as of this writing.

I am having similar issues on a separate NBU system. This uses NBU 7.5.04 on a W2008 R2 server running in Vampire.

Details below with numerous lines of "Error ndmpagent" removed. Job ends with a status Code 174.

10/05/2013 22:05:20 - Info nbjm (pid=4924) starting backup job (jobid=303) for client st07caddvnx, policy st07cadvnx_gis, schedule Full
10/05/2013 22:05:23 - Info bpbrm (pid=6232) st07caddvnx is the host to backup data from
10/05/2013 22:05:23 - Info bpbrm (pid=6232) reading file list from client
10/05/2013 22:05:23 - Info bpbrm (pid=6232) starting ndmpagent on client
10/05/2013 22:05:23 - Info ndmpagent (pid=1824) Backup started
10/05/2013 22:05:23 - Info bptm (pid=2536) start
10/05/2013 22:05:23 - Info bptm (pid=2536) using 30 data buffers
10/05/2013 22:05:23 - Info bptm (pid=2536) using 262144 data buffer size
10/05/2013 22:05:23 - estimated 0 kbytes needed
10/05/2013 22:05:23 - Info nbjm (pid=4924) started backup (backupid=st07caddvnx_1381035922) job for client st07caddvnx, policy st07cadvnx_gis, schedule Full on storage unit D7dd01
10/05/2013 22:05:23 - started process bpbrm (pid=6232)
10/05/2013 22:05:23 - connecting
10/05/2013 22:05:23 - connected; connect time: 0:00:00
10/05/2013 22:05:28 - Info bptm (pid=2536) start backup
10/05/2013 22:05:29 - begin writing
10/05/2013 22:05:30 - Error ndmpagent (pid=1824) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.
10/05/2013 22:05:30 - Error ndmpagent (pid=1824) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.
10/05/2013 22:05:31 - Error ndmpagent (pid=1824) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.

10/05/2013 22:29:29 - Error ndmpagent (pid=1824) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.
10/05/2013 22:29:30 - Error ndmpagent (pid=1824) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.
10/05/2013 22:29:30 - Error ndmpagent (pid=1824) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.
10/05/2013 22:30:32 - Info bpbrm (pid=2748) st07caddvnx is the host to backup data from
10/05/2013 22:30:32 - Info bpbrm (pid=2748) reading file list from client
10/05/2013 22:30:32 - Info bpbrm (pid=2748) starting ndmpagent on client
10/05/2013 22:30:32 - Info ndmpagent (pid=5496) Backup started
10/05/2013 22:30:32 - Info bptm (pid=808) start
10/05/2013 22:30:32 - Info bptm (pid=808) using 30 data buffers
10/05/2013 22:30:32 - Info bptm (pid=808) using 262144 data buffer size
10/05/2013 22:30:37 - Info bptm (pid=808) start backup
10/05/2013 22:30:38 - Error ndmpagent (pid=5496) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.
10/05/2013 22:30:38 - Error ndmpagent (pid=5496) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.
10/05/2013 22:30:39 - Error ndmpagent (pid=5496) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.
10/05/2013 22:30:39 - Error ndmpagent (pid=5496) st07caddvnx: Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait current backup/restore sessions finish before start new ones.
media manager - system error occurred  (174)

 

Any suggestions would be sorely appreciated.

 

Thanks.

 

DAN

Operating Systems:

Comments 7 CommentsJump to latest comment

Marianne's picture

I do not have personal experience, but would try to set 'max jobs per policy' attribute to 4.

The Activity Monitor sounds like this 7.5.0.5 issue:   

(ET3106719) <<Fixed in 7.5.0.6>> <<Fix Downloadable>> After upgrading a master server to NetBackup 7.5.0.5..... the Activity Monitor reports inconsistent information on job status for some jobs. Additionally, some jobs may not display in OpsCenter.

 http://www.symantec.com/docs/TECH203521

If not, check Filer logs as well as NBU bptm and bpbrm logs to see if 'completed' status was received.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

SOLUTION
danhatt's picture

Hi Marianne!

I changed the setting for "Limit jobs per policy" (which I think the is "max jobs" you said) to 4.  The 174 errors seem to have stopped, but I am still getting:

Active NDMP backup/restore streams reach the configured concurrent streams 4, but less than the maximum supported concurrent streams 8. No more backup/restore sessions can be started. Wait current backup/restore sessions finish, then increase concurrentDataStreams parameter.

Set logs for bptm and bpbrm at max logging. Full files attached after I started another job.

In the bpbrm log I found the following:

12:57:27.216 [9320.7764] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=C:\Program Files\Veritas\NetBackup\var\host_cache\064\c23d3c64+0,1,a,0,1,0+sv07data03.txt

That was the only thing I saw that looked like it might be a problem in bpbrm.

I found the following which are similar to the above in the bptm log:

12:57:27.528 [8712.7680] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=C:\Program Files\Veritas\NetBackup\var\host_cache\18e\33d6a38e+0,1,a,0,1,0+10.56.27.2.txt
12:57:27.528 [8712.7680] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=C:\Program Files\Veritas\NetBackup\var\host_cache\06e\f17c586e+0,1,a,0,1,0+149.136.252.1.txt
12:57:27.528 [8712.7680] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=C:\Program Files\Veritas\NetBackup\var\host_cache\1bd\839707bd+0,1,a,0,1,0+149.136.252.2.txt
12:57:27.528 [8712.7680] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=C:\Program Files\Veritas\NetBackup\var\host_cache\1f6\bba34ff6+0,1,a,0,1,0+10.112.96.31.txt
12:57:27.528 [8712.7680] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=C:\Program Files\Veritas\NetBackup\var\host_cache\025\c9481025+0,1,a,0,1,0+10.112.96.32.txt
12:57:27.543 [8712.7680] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=C:\Program Files\Veritas\NetBackup\var\host_cache\0d9\2d11c2d9+0,1,a,0,1,0+149.136.253.1.txt
12:57:27.543 [8712.7680] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=C:\Program Files\Veritas\NetBackup\var\host_cache\10a\5ffa9d0a+0,1,a,0,1,0+149.136.253.2.txt

Any ideas of what I should look at next?

Thanks.

DAN

AttachmentSize
bpbrm.txt 7.06 KB
bptm.txt 34.47 KB
danhatt's picture

I'm also having problems with doing restores from one VNX, but not the other.

I'm getting an error:

Active NDMP backup/restore streams reach the maximum concurrent streams supported 4. No more NDMP sessions can be started. Wait

current backup/restore sessions finish before start new ones.

There are no jobs running for either VNX. Anyone have any ideas?

 

DAN

watsons's picture

"set type=tar"  is not recommended for backing up EMC NDMP device:

http://www.symantec.com/docs/TECH202412

I would try with taking out that from backup selection, and use the default (which is "set type=dump").

 

SOLUTION
danhatt's picture

I made the change that watsons recommendend after some additional research. I needed ifo to defend the change if I was challenged.

Opened a case with Symantec, still waiting.

Marianne's picture

I see Symantec TechNotes as official documentation. This is normally the only motivation/proof that is required. 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

danhatt's picture

Worked with Symantec. They could not figure out what happened. Then, while we were working on the issue it went away. No changes were made to anything!

Remains a mystery. Sysmantec tech is looking to find some sort of possible solution in case this occurs again.