Symantec Management Platform (Notification Server)

 View Only
  • 1.  Filledup Table slows down NS7

    Posted Jun 03, 2009 07:44 AM
    Hello,

    i am new here in this forum. I work with Dell Client Manager 2.x/3.x and manage about 1000 PCs.
    The NS7 runs in a VM on a ESX-Cluster and the Database is located on a phys. machine.

    I noticed that the network-traffic between NS7-VM and the db-server increases steadily over weeks. This is the same with the cpu-usage: it grows steadily over weeks.
    Aftter some investigation with sql enterprise manager i found that the NS7 periodically sends the query "select Id, CollectionGuid, ResourceGuid, Removed from [dbo].CollectionMembershipChanges where Id > @LastChangeID order by [Id] asc" and this query creates alot if traffic.
    The relevant table CollectionMembershipChanges on my system contains about 2 600 000 lines after about 1month NS7 being productive. I pruned the table and now the perfomance of the system and the network traffic went back to "normal".


    My question is: Is it ok to clear the table? Is it intentional, that the table mentioned above is filling up? I did not find any settings to prune the table or limit the count of lines.

    Greetings, Th0u



  • 2.  RE: Filledup Table slows down NS7

    Posted Jul 08, 2009 09:55 PM
    I would also like to know if it's safe to prune this table.  We've got over 93 millions rows right now and performance is horrible.  I received a e-mail from Symantec support that this is a known issue (CollectionMembershipChanges and ResourceKeyChanged growing excessively) and was told it would be addressed in SP3. 

    It just occurred to me that we haven't seen SP2 yet, so I hope that was a typo.


  • 3.  RE: Filledup Table slows down NS7

    Posted Jul 10, 2009 11:20 AM
    Here's what I know so far.  Symatec support has recommended that I NOT prune the CollectionMembershipChanges table, as they are not sure what negative impact that may have.  This issue has been escalated to their highest priority and I'm supposed to be getting a call from their development team soon.  Our NS is becoming more and more unusable over time, with that table growing by about 1 million rows every four hours.  This is causing a lot of random timeouts, OutOfMemory exceptions, and other errors on the NS.  I have to put any further NS6 to NS7 migration plans on hold until this gets resolved, and I'm about a third of the way through the migration.  Not a great situation to be in.


  • 4.  RE: Filledup Table slows down NS7

    Posted Jul 10, 2009 11:55 AM
    if you like, we can probably scare up a blindfold and\or cigarette. (just kidding). I think this is something that I could pin to the top, except I can't do that anymore.


  • 5.  RE: Filledup Table slows down NS7

    Posted Jul 10, 2009 03:02 PM
    I spent some time on the phone with Symantec again today.  We basically watched the memory usage to confirm that services were releasing memory like they should, and they were.  This was to confirm that the problem was due to the size of these tables.  They are now figuring out what can be done with all the data in CollectionMembershipChanges and should be calling me back on Monday, possibly with a fix.

    They are also well aware of the issue with ResourceKeyChanged and that should be resolved in the next service pack.


  • 6.  RE: Filledup Table slows down NS7

    Posted Jul 13, 2009 04:22 PM
    You're going to see a new KB article on this issue soon - it'll be 48036.  They did have me truncate this table in the end, and a fix is scheduled for SP3, although a hotfix before then might be possible.

    Here are the steps that they wanted me to run through:

    Backup your DB. (as you should always do before something like this)
    Stop the Altiris services.
    Stop IIS
    Truncate the CollectionMembershipChanges table. (Do this directly in SQL using the Symantec_CMDB database with the query "truncate table CollectionMembershipChanges")
    Restart IIS
    Restart services.



  • 7.  RE: Filledup Table slows down NS7

    Posted Aug 13, 2009 08:13 AM
    The KB article Jessek speaks of is now available: http://kb.altiris.com/article.asp?article=48036&p=1


  • 8.  RE: Filledup Table slows down NS7

    Posted Aug 13, 2009 05:16 PM
    Yes, and this problem has gotten even worse for me.  To help keep me sane I set up a batch file on my SQL server that looks like:

    shutdown /m \\nameofNS /r /t 01
    c:\sleep 90
    osql -E -S nameofSQLServer\altiris -d Symantec_CMDB -Q "truncate table CollectionMembershipChanges"
    exit

    And then I have this run as a scheduled task from the SQL server every morning at 6AM.  And even then I've noticed that the OutOfMemoryException errors start in the early morning hours.  I know that I just need to restart the Altiris servers, but I decided to just reboot the server altogether.