Netbackup 7.0 deduplication storage pool not releasing space
Hello All, anyone using NBU7.0 with PDDE (deduplication storage media server) out there?
We're experiencing an annoying problem with the deduplication storage pool on our media server.
Once the dedup pool went beyond the high watermark (95%), of course all backup jobs going to it were failing with error 129.
Day after day, all images stored on it are expiring, one after the other, however the disk space is not released, it's still hanging at 96.3%, about 7TB
Initially we thought that until all segments of similar backups were alive, the space of the dedup pool would not be released. So, we expired manually most of the images in the dedup pool, no change.
Adding the size of all the images in the dedup pool, regardless of the deduplication benefits, add up to roughly 1.5TB. Considering deduplication, I'd expect that they should fit into a half TB or so. Instead, the used space still hangs around 7TB and doesn't move.
I opened a case with Symantec, but I'm still waiting.
If anyone had a similar experience, I'd appreciate your hints
Regs
Comments
Symantec has documented bugs in PDDE deduplication at 7.0. Your issue could be one that is fixed at 7.0.1. It is still in the First Availability stage, but that code should become GA soon.
http://seer.entsupport.symantec.com/docs/350190.htm
http://seer.entsupport.symantec.com/docs/350196.htm
There is a fix for this issue:
http://seer.entsupport.symantec.com/docs/357752.htm
I forgot to mention that Etrack ET1984226 was already applied to master & media servers a month ago when this problem was seen. Apparently it fixed the problem, as dedup pool occupation instantly dropped below 80%.
However, in the meantime we added more clients to the protection domain and saw that the dedup pool occupation was raising again, little by little, but we thought it was correct as the data to backup was growing.
Then, after hitting the high watermark again, we're getting the same problem. No backups are going to the dedup pool, images contained there are expiring day after day, we calculated that the images contained in the dedup pool should total 1.5TB (without keeping in account deduplication...) but the dedup pool occupation is still 7TB and doesn't move.
There must be some other bug. Symantec is helping us with the investigation.
Regs
how did this end up? is it still an issue? I am getting this as well!
Thanks
Evan
Systems Integrator
Emmanuel College
Boston, MA
Also very interested in the outcome of this one.
Principal Learning Consultant with Symantec Education Course Development
I'd be interested to find out if this is a Windows or UNIX PDDE media server.
Thanks.
NetBackup 7 on Solaris 10. Protecting Solaris 10 and Windows.
@ 7.0.1 having the same issue.
It's a test environment 7.0.1, I've expired all backup and run the following:
a. Force fragments and image cleanup:
nbdelete -allvolumes and bpimage -cleanup -allclients
b. Media Server Deduplication Pool cleanup:
crcontrol.exe --processqueue and crcollect -v -m +1,+2 --noreport
But still Disk space is not clear up.
having the same problem
did anyone found any solution about this problem ?
7.0.1 me too
I have the same problem. None of the expired images release space at disk pool.
No news?
Thanks.
I have a similar problem. I
I have a similar problem. I preformed the following:
On the master:
1. bpimage -cleanup -allclients
2. nbdelete -allvolumes
then on the media server I ran the following commands:
3. crcollect -v -m +1,+2
4. crcontrol --processqueue
I repeated step 3 & 4 multiple times. I did see about 20% disk usage reduction but I was expecting more.
thanks
Has anyone actually logged a
Has anyone actually logged a Support call yet?
Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows.
Handy NBU links
Yes... I have a call open,
Yes... I have a call open, they have not found any problems yet.
Problem fixed
OK... I think my problem has been fixed. I had to run the following commands 3 times:
1. crcollect -v -m +1,+2
Wait until it finished, it might take a little while.
2. crcontrol --processqueue
this will take a long time to process everything in the queue that was processed by the previous command. To see if this process is still running, tun the following command crcontrol --processqueueinfo. you should receive no for busy and for pending.
I ran through this sequence 3 or 4 time (it took me 4 days) and not I am down to 3TB usage vs. 8TB when I started 4 days ago.
hope that help.
Thanks
I also have this issue in my
I also have this issue in my test enviroment. The storage pool did not release space to file system. This is what I do.
1. Cause this is test environment so I expire all backup image on the storage pool. That will make all of my container file contain no data and can be remove.
2. I use step as Y_K mention above until there are no SO, DO records available and no queue to process. You can use command "Crcontrol.exe --queueinfo" to check if there are any queue to process.
3. I use command "Crcontrol.exe --dsstat 1" to check the available space that can be reclaim.
4. Run command "Crcontrol.exe --compactreleasesp 100 10 1". This will compact your container file. You can check status with command in step 3. You will see parameter "Space needs compaction" to be 0 MB
5. Run command "Crcontrol.exe --compactreleasesp 100". This will release space by delete empty container file 1 file/time you run command.
hope this help.
Would you like to reply?
Login or Register to post your comment.