Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Space consumption rises massively

Created: 23 Jan 2013 • Updated: 23 Jan 2013 | 21 comments

Hello,

our daily amount of backup jobs has 7 servers which we run as full backups on a deduplication storage. Every data record is to be maintained for 5 days. At the end of the week we save the entire media server on Tape also as complete backup.

On this tape backup we have growth rates of 100 GB per week even though our total data size increases only slightly, if at all.

Am I wrong if I expect that the total amount of data should stagnate or rise only slightly equal to the data on our servers?

The storage space on our servers is stagnating or rises just about 1-2GB per week. Nethertheless the deduplication storage is growing constantly much more than our servers?

Kind regards

Comments 21 CommentsJump to latest comment

CraigV's picture

What sort of data size increase are you seeing on the dedupe folder? Do you have SP1a and any additional patches installed?

Thanks!

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

'Bugs''s picture

All Patches are installed. What do you mean with "sort of data size increase"? Sorry for my bad english

Backup Exec 2012 SP1a | Advanced Disk Based Backup | Deduplication | Enterprise Server | W2K8 R2 | 24 GB RAM

CraigV's picture

By how much is your dedupe folder increasing on a weekly basis? 100GB? 5GB?

Alternative ways to access Backup Exec Technical Support:

https://www-secure.symantec.com/connect/blogs/alte...

'Bugs''s picture

Like the tape about 100GB per Week

Backup Exec 2012 SP1a | Advanced Disk Based Backup | Deduplication | Enterprise Server | W2K8 R2 | 24 GB RAM

pkh's picture

How long have you been using the dedup folder?  It might be that you need a couple more runs before the expansion stabilises.

The dedup should reclaim dead space automatically.  In case, this process is not working.  You can follow the procedure in the document below to reclaim dead space in the dedup folder

How to force a garbage collection of the deduplication folder

'Bugs''s picture

Since the beginning in early 2011. I will try your sugfgestion and report, thank you :)

ok, i did the manual garbage collection, doesn`t help unfortunately

Backup Exec 2012 SP1a | Advanced Disk Based Backup | Deduplication | Enterprise Server | W2K8 R2 | 24 GB RAM

16ris10's picture

what kind of disk is it? aram assuming here, the the backups go from disk to tape right?

'Bugs''s picture

Yes thats right. The HDD is a RAID5 Array of six 2TB-HDDs SAS 7200k

Backup Exec 2012 SP1a | Advanced Disk Based Backup | Deduplication | Enterprise Server | W2K8 R2 | 24 GB RAM

16ris10's picture

ok, i dont have much experience in this field but im guessing, its pretty normal that deduped backups going to tape would occupy more space on tape than on disks.

16ris10's picture

here you go with the query. concept of hyderation and best practise:

Best Practice for Backing up Dedupe to Tape
https : // www-secure . symantec . com /connect/forums/best-practice-backing-dedupe-tape

'Bugs''s picture

The consumption of space is not higher on tape. The problem is the constant increase of space consumption on the deduplication storage without having the same increase of live-data on our servers

Backup Exec 2012 SP1a | Advanced Disk Based Backup | Deduplication | Enterprise Server | W2K8 R2 | 24 GB RAM

16ris10's picture

i don't know how your disk work, but i have played with symantec puredisk for a while now, had the same issue, not sure if the procedure would be same. but you know what, you can check if the garbage collection and reclaimation of space is working or not by just sparing diskpool for some time (diverting to others) and see if the diskpool space consumption goes down. if it does, then obviously its automatically reclaiming space. if not, then defnitely you have a problem.

teiva-boy's picture

Is ANY of the data encrypted or compressed on that server?  Or any high change rate data?

What causes a dedupe store to grow are a couple of factors.  I wont get into the short comings of Symantec's Dedupe technology, though you cna read up on their stream handlers on your own...

1.  High change rate - e.g. databases that perhaps do a large number of writes or re-indexed for maintenance purposes

2.  Exchange 2010 and it's always on 24/7 online defrag

3.  Retention levels.  Too short of a retention, negates the benefit of dedupliction.  About 2weeks is minimum, 4+ weeks is ideal.

4.  Data that is encrypted and/or compressed repeatedly.  If it's encrypted and backed up thats fine.  If it gets opened and closed and reencrypted, thats not fine.

5.  An application that is not standard or widespread (SQL is common, MySQL is not.  Postgres is also no common)

Do you think any of your data sets fall into this?  

There is an online portal, save yourself the long hold times. Create ticket online, then call in with ticket # in hand :-) http://mysupport.symantec.com "We backup data to restore, we don't backup data just to back it up."

'Bugs''s picture

Hello again,

first i want to thank you all for your effort and time.

No encrypted or compressed data nor do we have high data change rate.

We have no big databases, and no exchange server.

Our data consist largely of many small immutable files and a small Active Directory with server based  user profiles. In addition there are about 50 pst-Files which i assume to have the highest data change rate.

@16ris10:

Do you mean that i should create a second deduplication storage?

Backup Exec 2012 SP1a | Advanced Disk Based Backup | Deduplication | Enterprise Server | W2K8 R2 | 24 GB RAM

pkh's picture

You can only have one dedup folder per media server.

'Bugs''s picture

Then i sadly don`t understand what 16ris10 means in his last post

Backup Exec 2012 SP1a | Advanced Disk Based Backup | Deduplication | Enterprise Server | W2K8 R2 | 24 GB RAM

16ris10's picture

yes sorry i thought you had more storage server as like in my experience we had 5 other dedupe appliances, for which we diverted the backups and saw if the comsumption on the affected was going down or not by this we were able to verify if the garbage collection and reclaimnation of space was working automatically or not..

'Bugs''s picture

According to this Thread: http://www.symantec.com/business/support/index?pag... Symantec writes that there is no space freed up on the file system at all?

Please note that Backup Exec will only free the space within the Deduplication Folder containers and will not release space to the file system.  That is, at the end of this process, the properties of the Deduplication Folder should show that space has been made available.  However, looking at the volume from the Windows Explorer view will not show space being freed up

Can this really be true? What if the deduplication folder grows and grows until the hdd is full?

This would mean that i save totally unefficient every week a large amount of empty containers because these containers cannot be removed?

Backup Exec 2012 SP1a | Advanced Disk Based Backup | Deduplication | Enterprise Server | W2K8 R2 | 24 GB RAM

Gurvinder Rait's picture

if you run crcontrol.exe --compactstart, that would release space to file system. You can check the dedupe stats by running crcontrol.exe --dsstat and if that shows you have a lot of space within the space available within containers then that space can also be given back to the file system but later when more containers have to be created BE/Dedupe would take up that space.

'Bugs''s picture

crcontrol.exe --dsstat shows me this:

************ Data Store statistics ************
Data storage      Raw    Size   Used   Avail  Use%
                   9.1T   8.7T 848.5G   7.9T  10%

Number of containers             : 12735
Average container size           : 255419471 bytes (243.59MB)
Space allocated for containers   : 3252766966063 bytes (2.96TB)
Space used within containers     : 3252532086155 bytes (2.96TB)
Space available within containers: 234879908 bytes (224.00MB)
Space needs compaction           : 2411773786713 bytes (2.19TB)
Reserved space                   : 447292252160 bytes (416.57GB)
Reserved space percentage        : 4.5%
Records marked for compaction    : 98905951
Active records                   : 17987054
Total records                    : 116893005

I calculated the total amount of data of our servers today morning and i get ~1300GB total space consumption. We have more or less just three backup sets of every server (one of them has five) and a deduplication rate of 3,4:1

Backup Exec 2012 SP1a | Advanced Disk Based Backup | Deduplication | Enterprise Server | W2K8 R2 | 24 GB RAM

16ris10's picture

Space needs compaction : 2411773786713 bytes (2.19TB)

Are you serious with this much space needing compaction? :O.

Follow these two technotes to reclaim a considerable amount of this 2.19TB.

TECH124914
TECH180659