The release notes document for RU5 version has all the details.
[ ftp://ftp.symantec.com/public/english_us_canada/products/symantec_endpoint_protection/11.0/manuals/ru5/Release_Notes_for_Symantec_Endpoint_Protection_11_0_5.pdf ]
I think this fix could be related to what you have mentioned in your post.
RTVScan.exe does not release memory until after the scan
completes
Fix ID 1427192
■ Symptom:Whenvery large containers are scanned, memory continues to grow
until the scan completes.
■ Solution: Memory usage is reduced by not storing unnecessary data during
the scan.
However, we would be eager to know the results in your environment as far as scan times are concerned.
Aniket