Creating new IIS application pools - the magic solution to SMP 7 slow console operations?
I was on-site to a new customer last week and we worked on resolving some problems with their production SMP 7 installation that was consuming an average of 20% of CPU whilst managing a meager 40 machines.
In order to check where the CU cycles were wasted we decided to implement 2 new application pools: one for the InventoryRuleManagement web-service and one for the Altiris/NS/Agent interfaces.
This resolved the issue so this morning, I decided to implement the same solution on my test lab.
And my SMP console has been operating in conditions much worse than what my customer encountered. I started with a single VM (SQL + NS and 4 GiB of RAM) and IIS was constantly taking 100% CPU when I used the console (specially when loading the Patch Management Solution worker view) for a very long time.
Such a long time... that I gave up in the end and built up a 64-bit Windows Server to host the SQL database (with 4 cores and 4GiB of RAM). I migrated the DB over to the new server, left the NS with 3.5GiB and tested. Still the same. IIS was still consuming all available resources and the console was not really usable.
Since then I dropped the SQL Server memory and the NS memory to 2GiB each, and it worked pretty much the same.
But as I am building up my SMP 7 environment to test more features for my customers I need my SMP lab to be really operational, so I went for the same solution, adding application pool isolation to my server this morning. And dear reader, did that help?
YES!!!
It's amazing. I created three new application pools as shown below and after an IIS reset all my problems with the sluggisher-than-bearable console went away. I have loaded the console many times over, switched views and loaded the PMS worker view and Patch Remediation center with no major troubles.
Still this is quite surprising, as the server memory hasn't really changed, but it seems that the IIS processes (w3wp.exe) are consuming more memory and much less CPU. So is the performance impact due to the additional head room provided by the application isolation's?
It must be.
About Endpoint Management Community Blog
The Endpoint Management Community Blog is the perfect place to share short, timely insights including product tips, news and other information relevant to the Endpoint Management community. Any authenticated Connect member can contribute to this blog.
Comments 21 Comments • Jump to latest comment
Wow, very nice.
Nice! I've had to change the app pool for service desk to get the process manager working on my laptop.
An article was published on KB.altiris on moving the Patch management pools to a seperate pool.
http://www.symantec.com/business/support/index?page=content&id=HOWTO4592&actp=search&viewlocale=en_US&searchid=1287081280447
Have you verfied everything is intact after upgrade?
Note: Customers should reverse the application pool assignment on the InventoryRuleManagement virtual directory, prior to performing a repair or upgrade of any Patch Management based solution. This concern will be negated upon final confirmation from development. (This step may not be necessary in some upgrade cases, after the upgrade has completed check the Application Pool to see if has changed to Default. If it has change it back to Inventory Rule Management.)
Note for Patch Management 7.0 users: This has not been tested in 7.0 but there is no known reason that it would cause a problem. Symantec would suggest that customers start without setting this up and it could be added at a future time if needed.
Steve Petrasek
...we'll check this.
Hopefully now that SP5 is out it'll be a while before anything major is release for Patch or SMP.
We shall see.
Ludovic FERRÉ
Principal Remote Product Specialist
Symantec
Need help with IIS log files? Check out the self-service portal on http://aila.15-cloud.fr/
For a view
because of an issue and forgot to set the inventory rule management back to the defualt application pool. Did not encounter any problems, still running smooth.
@Ludovic: Can you specify which parts you set to which application pools? I've noticed this same behaviour a while back and did the same changes. I want to compare to yours to see if i missed any :-) Performance is ok now but it can always be better :)
Peter van Leeuwen
If your question has been resolved, please be sure to click Mark as Solution! Thank you.
Reading the text for the second time i see that you changed the inventory rule mgmt and the ns agent. I've did the same with the same results. :-)
Peter van Leeuwen
If your question has been resolved, please be sure to click Mark as Solution! Thank you.
Ga eens aan het werk ;)
Martijn Groothuis
Technical Consultant
If your question has been resolved, please be sure to click Mark as Solution! Thank you.
ludovic_ferre
Do you think you could write an "Article" on how to do this?
The article is already written!
http://www.symantec.com/business/support/index?page=content&id=HOWTO4592&actp=search&viewlocale=en_US&searchid=1287081280447
Steve Petrasek
yeah but this is for IIS 6
Not being an IIS guy, I am having some issues doing this with IIS 7 and I can't seem to even google anything that helps me configure the isolation for select sites
I think I have it but unsure if it is correct.
When I created the application pools for the sites I chose, I used "Classic" mode
Is this correct?
Here is my application pool list after I performed the tasks
I chose them randomly so there is no reason for them other than to test.
I wouldn't know how to check for the performance gains either but since my NS is now responsive, I suspect this is helping
I have searched for "integrated application pool" and Google kinldy pointed me to Microsoft technet:
http://technet.microsoft.com/en-us/library/cc753449(WS.10).aspx
And I found the following information on the classic and integrated application pools:
This points to internal processing changes. Now may I ask how you managed to install SMP 7.0 on a Windows Server 2008? Or did I miss something ;).
[1] http://technet.microsoft.com/en-us/library/cc725564(WS.10).aspx
Ludovic FERRÉ
Principal Remote Product Specialist
Symantec
Need help with IIS log files? Check out the self-service portal on http://aila.15-cloud.fr/
For a view
Bill,
You would want to use the "Classic" application pool for all 7/7.1 related communications at this time. This may change for now, but IIS6 compatability modes and classic application pools are required for 7/7.1.
Regards,
Rasmus
I am using the Beta :)
Sorry to get you excited there. I can imagine you were thining "sweet...I can use x64 now!!!"
Thanks for the input and sorry it took so long to respond. was at the conference last week now onsite at a client in another city this week.
Bill
So, I wrote that original KB article many years ago (HOWTO 4592). It was intended to solve performance issues for a very large customer with a 8+ languages enabled for Patch Management 6. Each PM Inventory Rule processing request was an expensive operation, and it was overwhelming all other operations on the NS.
Official support for the technique was added in a later version of PM6 and covered in the release notes. As someone stated earlier, I don't know if it's officially supported for PM7.
The technique of defining additional application pools in that scenario enabled IIS to restrict the backlog of Inventory Rule requests to a manageable size (and avoid running out of memory). The other benefit was that it provided a an express lane for incoming NS console requests, as they didn't have to wait in the queue behind all those client's Inventory Rule requests.
If this is making a measurable difference in a lab environment with just a few clients, then it's a bit of a surprise to me. However, I know virtually nothing about the changes in PM7 so don't take this as a dismissal of your results :) Did you enforce any limits on the new app pools?
Scott Wedekind
.. this relates to the orginal (PMS) issue.
For some strange reasons my server was pretty much unusable for minutes when interacting with the console, with w3wp.exe taking all cpu resources.
Then I went to a customer and we resolved some IIS performance problems with the same solution.
It's worth stating that the difference between our fix implementation and the PMS application pool fix is that we are not curring a workload issue but rather some base memory consumption.
Before we implemented the 3 application pools memory consumed by the w3wp service was limited 500~520 MiB. After implementation it went up to 700~800 for the 3 pools with a noticeable impact on performance.
The additional memory connsumption could be bad overhead or more likely needed head room for the various web-applications.
Back to my poor test system, the same solution had pretty much the same results, alhough I still get some spikes here and again it made a huge difference (I actually turned from hating my test system to using it and linking CMS 7, a huge difference).
Ludovic FERRÉ
Principal Remote Product Specialist
Symantec
Need help with IIS log files? Check out the self-service portal on http://aila.15-cloud.fr/
For a view
https://www-secure.symantec.com/connect/forums/ns7-systemoutofmemoryexception-global-mutex-timed-out
One of the problems we had with 7.0 was the performance of the console so we put that on hold, carried on with 6 but are now trying 7.1 beta. Having a separate AppPool for different web applications seems to be a good idea, but we have had little difference in performance. We included the ones that are specified and then also added pools for:
I am also looking at adding one for Workflow.
These applications were chossen based on the IIS log. It is interesting to see them being used and the total memory of all the AppPools is slightly more than using the single one.
We currently only have three computers in the database and one of those is the server itself. The console performance is now making the rollout of 7.1 to look impractical and we stil need to get on to functionallity.
The server has 8GB RAM and four cores.
I am going to give this a try as well. What about increasing the web garden from 1 to more? Allowing the W3 to use the multiple cpu's and take advantage?
Also,
Does any one have any advice on a 64 bit service desk IIS. I have increased caching on the pools and recycle times. I tried the web garden moving it to 8 at first for the 8 Virtual processor and I never seen the system move that fast but, after a while we were getting "session id" not found and switched back.
Caching has help somewhat, but We want more speed!
thanks in advance
Official support now :)
How to implement custom application pools on the SMP to improve web-console performance
http://www.symantec.com/docs/HOWTO82344
Was this script from You Ludo ?
~Pascal @ Kotte.net~ Do you speak French? Et utilisez Altiris: venez nous rejoindre sur le GUASF
I updated the script with my usual web-application folders, but I got it from Carsten Giese originally.
Ludovic FERRÉ
Principal Remote Product Specialist
Symantec
Need help with IIS log files? Check out the self-service portal on http://aila.15-cloud.fr/
For a view
Thanks both :)
~Pascal @ Kotte.net~ Do you speak French? Et utilisez Altiris: venez nous rejoindre sur le GUASF
Would you like to reply?
Login or Register to post your comment.