Video Screencast Help

Scan Engine Performance Testing

Created: 10 Jan 2012 | 4 comments

Not specifically hunting for a solution here, more looking for information.

We have a pair of SSE servers (HP DL360 G5's quad core 1.86Ghz Xeon CPU, 5Gb RAM [yes a strange amount of RAM !!], gigabit ethernet over a PVLAN [on a shared switch that carries other VLAN trunk traffic] to a pair of NetApp filers]

Is there any way of achieving a benchmark that will demonstrate the ceiling that this kit can reach before performance becomes impacted.

Looking at average day performance the LAN flies all over the place from idle to 50-60%, CPU averages 30-40% occasional peaks ~70-80% and the disk access seems negligible....

I appreciate that there are so many factors to consider network load on other trunks, volume of file requests, size of files etc etc.....

I'd appreaciate if anyone can suggest any ways of benchmarking it such that we can capacity plan and monitor this solution (there is plan to move a very large chunk of CIFS data to these filer heads which will undoubtedly incur extra load on the SSE boxes.....)

regards

Rob

Comments 4 CommentsJump to latest comment

kirsche's picture

Hello,

as you already mentioned, its tricky to estimate the load it can handle. Especially with the filer also playing part of this (file types, file sizes, access patterns...). When you move a lot of file to the heads - are you planning to do this via CIFS or some mirroring? The copy process itself shouldnt be affected if you are on ONTAP 7.3 or higher since the vscan process happens separated from the client process. You would just need to keep an eye on the performance and maximum scan requests.

As for monitoring of the SSE - you can see how busy the box is in the UI - how many threats are active or idle. Also ensure that the values for the in-memory-filesystem have been increased from the defaults.

Probably the best way to monitor is on the filer side - since this will be the system who sends the requests and keeps track on the progress (check the vscan_stats counters for this).

robskij's picture

The data will be robocopied from our legacy storage onto the NetApp, in differential batches overnight.

Some of this data is roaming profile information for some 6,000 users... data which will be actively scanned the next time they login...

I'm guessing each day following a batch migration we may see an initial load increase as the new content gets its first scan, then should tail off as only changes/updtes are scanned.

On the scan engin UI the load looks to be well within the boxes tolerances, of the 30 active threads (15 per ethernet link) each box cites they rarely get above 5 on each scanner....

The scans per second at peak load show ~20-25  (during testing with the command line tool I was able to push this to ~140 on each of the 2 scan servers)

We are already watching the filers via the NetApp management layer, There seems to have been a marginal increase in load from applying the scan engines but nothing that would make you air any concern on load.

I was just worried that the additional CIFS content might cause problems as its using the storage for homes and profiles where previously it was just for filestore.

The filers also run NFS and iSCSI volumes which consume more of the NetApp processing power than the current CIFS data.

Probably worrying about nothing but our service assurance manager has some concerns that we are simply dumping data onto a service without adequate capacity planning!

BenDC's picture

In Monitors you can monitor requests. You can also enable statistics reporting in monitors and logging. Then review the statistics data in reports -> Statistic view.

Davinci_uk's picture

If you have a lot of  NAS headers, you may need to create some SSE pods as descibed in the 'Getting Started Guide'