Video Screencast Help
Endpoint Management Community Blog

Creating new IIS application pools - the magic solution to SMP 7 slow console operations?

Created: 14 Oct 2010 • Updated: 14 Oct 2010 • 24 comments
Ludovic Ferre's picture
+5 5 Votes
Login to vote

I was on-site to a new customer last week and we worked on resolving some problems with their production SMP 7 installation that was consuming an average of 20% of CPU whilst managing a meager  40 machines.

In order to check where the CU cycles were wasted we decided to implement 2 new application pools: one for the InventoryRuleManagement web-service and one for the Altiris/NS/Agent interfaces.

This resolved the issue so this morning, I decided to implement the same solution on my test lab.

And my SMP console has been operating in conditions much worse than what my customer encountered. I started with a single VM (SQL + NS and 4 GiB of RAM) and IIS was constantly taking 100% CPU when I used the console (specially when loading the Patch Management Solution worker view) for a very long time.

Such a long time... that I gave up in the end and built up a 64-bit Windows Server to host the SQL database (with 4 cores and 4GiB of RAM). I migrated the DB over to the new server, left the NS with 3.5GiB and tested. Still the same. IIS was still consuming all available resources and the console was not really usable.

Since then I dropped the SQL Server memory and the NS memory to 2GiB each, and it worked pretty much the same.

But as I am building up my SMP 7 environment to test more features for my customers I need my SMP lab to be really operational, so I went for the same solution, adding application pool isolation to my server this morning. And dear reader, did that help?

YES!!!

It's amazing. I created three new application pools as shown below and after an IIS reset all my problems with the sluggisher-than-bearable console went away. I have loaded the console many times over, switched views and loaded the PMS worker view and Patch Remediation center with no major troubles.

Still this is quite surprising, as the server memory hasn't really changed, but it seems that the IIS processes (w3wp.exe) are consuming more memory and much less CPU. So is the performance impact due to the additional head room provided by the application isolation's?

It must be.

Comments 24 CommentsJump to latest comment

dfnkt_'s picture

Nice! I've had to change the app pool for service desk to get the process manager working on my laptop.

-1
Login to vote
lotsill's picture

An article was published on KB.altiris on moving the Patch management pools to a seperate pool. 

http://www.symantec.com/business/support/index?page=content&id=HOWTO4592&actp=search&viewlocale=en_US&searchid=1287081280447

Have you verfied everything is intact after upgrade?

Note: Customers should reverse the application pool assignment on the InventoryRuleManagement virtual directory, prior to performing a repair or upgrade of any Patch Management based solution. This concern will be negated upon final confirmation from development. (This step may not be necessary in some upgrade cases, after the upgrade has completed check the Application Pool to see if has changed to Default. If it has change it back to Inventory Rule Management.)

Note for Patch Management 7.0 users:   This has not been tested in 7.0 but there is no known reason that it would cause a problem.  Symantec would suggest that customers start without setting this up and it could be added at a future time if needed.

+2
Login to vote
Ludovic Ferre's picture

...we'll check this.

Hopefully now that SP5 is out it'll be a while before anything major is release for Patch or SMP.

 

We shall see.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

-1
Login to vote
P. van Leeuwen's picture

because of an issue and forgot to set the inventory rule management back to the defualt application pool. Did  not encounter any problems, still running smooth.

 

@Ludovic: Can you specify which parts you set to which application pools? I've noticed this same behaviour a while back and did the same changes. I want to compare to yours to see if i missed any :-) Performance is ok now but it can always be better :)

Peter van Leeuwen

If your question has been resolved, please be sure to click Mark as Solution! Thank you.

+2
Login to vote
P. van Leeuwen's picture

Reading the text for the second time i see that you changed the inventory rule mgmt and the ns agent. I've did the same with the same results. :-)

Peter van Leeuwen

If your question has been resolved, please be sure to click Mark as Solution! Thank you.

-1
Login to vote
Martijn Groothuis's picture

Ga eens aan het werk ;)

Martijn Groothuis
Technical Consultant
If your question has been resolved, please be sure to click Mark as Solution! Thank you.

-3
Login to vote
billasullivan's picture

ludovic_ferre

 

Do you think you could write an "Article" on how to do this?

+1
Login to vote
billasullivan's picture

yeah but this is for IIS 6

Not being an IIS guy, I am having some issues doing this with IIS 7 and I can't seem to even google anything that helps me configure the isolation for select sites

 

I think I have it but unsure if it is correct.

When I created the application pools for the sites I chose, I used "Classic" mode

Is this correct?

 

Here is my application pool list after I performed the tasks

 

 

I chose them randomly so there is no reason for them other than to test. 

I wouldn't know how to check for the performance gains either but since my NS is now responsive, I suspect this is helping

+3
Login to vote
Ludovic Ferre's picture

I have searched for "integrated application pool" and Google kinldy pointed me to Microsoft technet:

http://technet.microsoft.com/en-us/library/cc753449(WS.10).aspx

And I found the following information on the classic and integrated application pools:

"In IIS 7, there are two request-processing modes for application pools: integrated mode and classic mode. When you configure an application pool with integrated mode, IIS processes requests for managed content with the new integrated IIS and ASP.NET request processing pipeline. When you configure an application pool with classic mode, IIS continues to process requests for managed content using the separate IIS and ASP.NET request-processing pipelines. Use classic mode only for applications that cannot run in integrated mode." [1]

This points to internal processing changes. Now may I ask how you managed to install SMP 7.0 on a Windows Server 2008? Or did I miss something ;).

[1] http://technet.microsoft.com/en-us/library/cc725564(WS.10).aspx

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

+2
Login to vote
Rasmus R's picture

Bill,

You would want to use the "Classic" application pool for all 7/7.1 related communications at this time. This may change for now, but IIS6 compatability modes and classic application pools are required for 7/7.1.

Regards,
Rasmus

+3
Login to vote
billasullivan's picture

I am using the Beta :)

Sorry to get you excited there.  I can imagine you were thining "sweet...I can use x64 now!!!"

Thanks for the input and sorry it took so long to respond.  was at the conference last week now onsite at a client in another city this week.

Bill

-1
Login to vote
scottwed's picture

So, I wrote that original KB article many years ago (HOWTO 4592).  It was intended to solve performance issues for a very large customer with a 8+ languages enabled for Patch Management 6.  Each PM Inventory Rule processing request was an expensive operation, and it was overwhelming all other operations on the NS. 

Official support for the technique was added in a later version of PM6 and covered in the release notes.  As someone stated earlier, I don't know if it's officially supported for PM7. 

The technique of defining additional application pools in that scenario enabled IIS to restrict the backlog of Inventory Rule requests to a manageable size (and avoid running out of memory).  The other benefit was that it provided a an express lane for incoming NS console requests, as they didn't have to wait in the queue behind all those client's Inventory Rule requests.

If this is making a measurable difference in a lab environment with just a few clients, then it's a bit of a surprise to me.  However, I know virtually nothing about the changes in PM7 so don't take this as a dismissal of your results :)   Did you enforce any limits on the new app pools?

Scott Wedekind

 

-1
Login to vote
Ludovic Ferre's picture

.. this relates to the orginal (PMS) issue.

For some strange reasons my server was pretty much unusable for minutes when interacting with the console, with w3wp.exe taking all cpu resources.

Then I went to a customer and we resolved some IIS performance problems with the same solution.

It's worth stating that the difference between our fix implementation and the PMS application pool fix is that we are not curring a workload issue but rather some base memory consumption.

Before we implemented the 3 application pools memory consumed by the w3wp service was limited 500~520 MiB. After implementation it went up to 700~800 for the 3 pools with a noticeable impact on performance.

The additional memory connsumption could be bad overhead or more likely needed head room for the various web-applications.

Back to my poor test system, the same solution had pretty much the same results, alhough I still get some spikes here and again it made a huge difference (I actually turned from hating my test system to using it and linking CMS 7, a huge difference).

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

0
Login to vote
JohnEBLP's picture

One of the problems we had with 7.0 was the performance of the console so we put that on hold, carried on with 6 but are now trying 7.1 beta. Having a separate AppPool for different web applications seems to be a good idea, but we have had little difference in performance. We included the ones that are specified and then also added pools for:

  • ActivityCenter
  • Console
  • PublishingContoller

I am also looking at adding one for Workflow.

These applications were chossen based on the IIS log. It is interesting to see them being used and the total memory of all the AppPools is slightly more than using the single one.

We currently only have three computers in the database and one of those is the server itself. The console performance is now making the rollout of 7.1 to look impractical and we stil need to get on to functionallity.

The server has 8GB RAM and four cores.

+5
Login to vote
Harold.Moore's picture

I am going to give this a try as well.  What about increasing the web garden from 1 to more?  Allowing the W3 to use the multiple cpu's and take advantage?

Also,

Does any one have any advice on a 64 bit service desk IIS.  I have increased caching on the pools and recycle times.  I tried the web garden moving it to 8 at first for the 8 Virtual processor and I never seen the system move that fast but, after a while we were getting "session id" not found and switched back.

Caching has help somewhat, but We want more speed!

 

thanks in advance

0
Login to vote
Pascal KOTTE's picture

Official support now :)

 

How to implement custom application pools on the SMP to improve web-console performance

http://www.symantec.com/docs/HOWTO82344 

Was this script from You Ludo ?

~Pascal @ Kotte.net~ Do you speak French? Et utilisez Altiris: venez nous rejoindre sur le GUASF

+1
Login to vote
Ludovic Ferre's picture

I updated the script with my usual web-application folders, but I got it from Carsten Giese originally.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

+1
Login to vote
Pascal KOTTE's picture

Thanks both :) yes

~Pascal @ Kotte.net~ Do you speak French? Et utilisez Altiris: venez nous rejoindre sur le GUASF

-1
Login to vote
noodleNT's picture

Is the "Web Garden" option supported? I was thinking about increasing it to 4 for both the TaskManagement and Altiris-NS-Agent app pools since then seem to work the hardest.

I have also noticed in the Classic.NET AppPool that /Altiris/PuggableProtocols/NSAccess/NSAccess.aspx is very active. It might make sense to move this into its own App Pool.

+1
Login to vote
network101's picture

Hi, does this work in cms 7.1 sp1? My console is painfully slow.

-2
Login to vote
Ludovic Ferre's picture

I can't officially comment, at the risk of getting another of my post suppresed from Connect for sharing good information with out customers.

But you can work things out on your own and wigh the cost benefit of the potential solutions.

It's very easy to implement and reverse as well.

I am currently off-net, on a retreat of some kind. I'll be back real soon, and you sure will hear from me then ;-).

Ludovic FERRÉ
Principal Remote Product Specialist
Symantec

-4
Login to vote