Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Number of file is more than 4113406

Created: 01 May 2013 | 5 comments

Hi Folks,

I am taking linux client.One file system have nearly 4113406 files.My backup is speed is reducing to below 10Kb/Sec once started taking in particular file system.Backup is alreay running on multi stream mode.My remaining file system is getting completing good speed.

Why this particular file system is not happening..Any issue with large number of files??

Please help on this.

Thansk,

Susi.S

Operating Systems:

Comments 5 CommentsJump to latest comment

revaroo's picture

there are issues backing up (and restoring) thousands and millions of small files. There is overhead when capturing the inode/file attributes of millions of small files.

The way to avoid this is to use flashbackups that backup at a block level rather than a file level.

See the following forum posts, where others have had the same issue:

https://www-secure.symantec.com/connect/forums/lar...

https://www-secure.symantec.com/connect/forums/mil...

https://www-secure.symantec.com/connect/forums/bac...

Also check out this technote, specific to Windows

http://www.symantec.com/business/support/index?pag...

Nicolai's picture

Qustion number 1 - Is the file system the cause of the issue (can be).

See http://www.symantec.com/docs/HOWTO56131 titled Measuring disk performance with bpbkar

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

Marianne's picture

In addition to above excellent posts, have a look at this similar discussion:

https://www-secure.symantec.com/connect/forums/best-practice-linux-file-services-performance-backups

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

mph999's picture

Hasten to add, all backup products will have an issue with this, not just NetBackup.

At least one of the reasons Flashbackup was produced, is to backup filesystems with millions of small files.

Martin

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Mark_Solutions's picture

Lots of excellent advice above - just one thing to add to the rest ....

Check that clients exclude list - if it has any wild card exclusions (*.tmp etc) then get rid of them and at least make them more specific

If you have such a wild card exclusiin the client will have to scan the entire system for such files before it can even think of starting its backup, which could take hours.

Hope this helps - flashbackup is really the way to go but if you dont have an Enterprise Client license check out my suggestion

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.