Inventory forwarding is the very cool process of copying inventory information from one or more (source) Notification Servers to a single (destination) Notification Server. Brent Ring covers the process from top to bottom and even tosses in a section on best practices and another that covers solutions to common problems.
- Inventory Forwarding - Executive Summary
- Inventory Forwarding Process – Simple Overview
- Inventory Forwarding – A Deeper Look
- Step 1: Assure that Client Computers have Data to be Forwarded
- Step 2: Configure the Inventory Forwarding Rule
- Configuring the Rule – Naming and Describing the Rule
- Configuring the Rule – Selecting Resources
- Configuring the Rule – Selecting Inventory Classes
- Configuring the Rule – Selecting a Destination (Reporting) Server
- Configuring the Rule – Credentials
- Configuring the Rule – Replication Schedule
- Configuring the Rule – Verification Schedule
- Inventory Forwarding Optimization and Best Practices
- Preparation for Forwarding - Available Disk Space
- Forwarding Specific Data
- Purging Resources from the Destination Notification Server
- Inventory Accuracy
- Data Refresh
- Intermediate Reporting Servers
- Include All Destination Servers
- Properly Configure Forwarding Schedules
- Correctly Configure Verification Rules
- Optimize SQL
- Anti-Virus Configuration
- Notification Server Queue Optimization
- Match Solutions
- Adjust Top Tier Queue Size
- Adjust Forwarding Threads
- Adjust Collection Refresh Scheduled Tasks
- Stop Processing of NSE's
- Inventory Clean Up
- License Refresh
- Extra Index
- Appendix A: Common Issues and Solutions with Inventory Forwarding
- Appendix B: Registry Information
- Appendix C: File Processing thru the Queues
Altiris® Notification Server – Inventory Forwarding : PDF Version.Inventory Forwarding - Executive Summary
This white paper is intended to cover the details surrounding how the Inventory Forwarding feature functions inside of the Altiris® Notification Server platform. Inventory forwarding is the process of copying inventory data from one or more source (managing) Notification Server's to a destination Notification Server, also known as a reporting or roll-up server. Once both Notification Servers are configured and scheduled to perform their respective roles, the process begins. A complete cycle to populate complete data on the destination Notification Server involves forwarding, data replication, and verification. The combination of these processes can be potentially time-consuming, depending on the existing load of the Notification Servers involved. The overall process is summarized in Figure 1.
Figure 1 – Inventory Forwarding Diagram
Inventory forwarding is performed at a dataclass level for a selected group of resources. Once the initial dataset is forwarded, the subsequent scheduled tasks only forward delta or changed sets of data. If nothing has changed, nothing gets forwarded.
Multiple source Notification Servers can forward selected dataclass information to a destination Notification Server using the Inventory Forwarding rules. You can find these rules in the Inventory Forwarding section on the Configuration page, as shown in Figure 2.
Figure 2 – Inventory Forwarding Console Page
Inventory Forwarding features are included with Notification Server. There are no additional licensing requirements; however, if dataclasses from other solutions beyond basic inventory are forwarded, BOTH the source and destination Notification Servers must have a corresponding, valid solution installation and license. The same requirement exists for all solutions added in the future.
Inventory Forwarding to SMS
Notification Server can exchange data with Microsoft* SMS by using Altiris® Connector for Microsoft SMS. The use and function of that connector is beyond the scope of this document and will be addressed in a separate article.Inventory Forwarding Process – Simple Overview
The following steps relate to Figure 3. Each configured Inventory Forwarding rule uses the details specified in the figure below to determine which resources (computers, users, and so on) to forward, what corresponding inventory dataclasses to send, and the name of the destination Notification Server. Scheduling information is also provided for when and how frequently this data is forwarded.
Figure 3 – Inventory Forwarding Simple Overview
Steps 1 and 1a – Inventory data is collected from the managed computers.
Step 2 – Create the forwarding rule by right-clicking the Inventory Forwarding folder and selecting New > Inventory Forwarding Rule. You can also clone an existing rule and modify its parameters. Configure the rule to use the required resource collections, and choose the appropriate inventory classes, select a destination server, and provide any credentials necessary to connect the respective Notification Servers. You then configure the replication schedule (required) and the data verification schedule (optional). Details surrounding this configuration are discussed later.
Step 3 – When the scheduled time arrives, or if you select the Run Rule button (see Figure 4), both the Inventory Forwarding AND the Item Replication scheduled tasks run. The data forwards and item replication occurs simultaneously.
Figure 4 – The Run Rule Button
Steps 4 and 4a – The Inventory Forwarding process compiles the data to be forwarded by collecting the required information for each selected dataclass and generating a Notification Server Event (NSE) file. This file is posted to the destination server, the file is parsed, and resulting data is either inserted or removed in the correct database tables. While this is occurring, the item replication schedule also runs from the destination server, completing resource associations, comparing information, and generating and comparing snapshots. This process allows data that may have been skipped or missed to be able to be forwarded at a later time.
Step 4b – Data verification is run as a separate scheduled activity. This process simply verifies the forwarded data through a hash comparison on a per dataclass basis. If the hashes don't match, the data is flagged to be re-sent. Data is verified every 30 days, by default.
Step 5 – The resource associations are created at this point, and forwarding is complete.Inventory Forwarding – A Deeper Look
This section will take the simple process outlined in the previous section and provide a deeper analysis into the procedures that are working behind the scenes, some of the tables they affect, and some ideas on how to troubleshoot problems in each area. Some assumptions are made here. First, client computers are generating inventory information AND communicating that information to the managing Notification Server. Second, the actual forwarding mechanism is working correctly, and the destination or roll-up server is receiving forwarded data from the managing or collecting servers. These assumptions allow the remainder of this document to concentrate on details of the inventory forwarding process itself.
Step 1: Assure that Client Computers have Data to be Forwarded
Reporting is a key purpose for inventory forwarding. The reports required to make the necessary business decisions are useless without data. That data comes from managed client workstations and, correspondingly, the first step is making sure that there is actually data to be forwarded, which you can do by assuring that the managed client computers reporting to a managing Notification Server have good Altiris Agent communication. For information on troubleshooting Altiris Agent communication, see Altiris Knowledgebase article 30757, found here.
Assuming that agent communication is functioning, the managing Notification Server should also have data stored for that dataclass. Figure 5 shows the Inventory tab of a Resource Manager page, from a client workstation. The highlighted Inventory tab shows some of the dataclasses available for forwarding. This client workstation has data ready to go.
Figure 5 – Resource Manager Inventory Page
Deeper Details – Forwarding Resources and Data
When a managed client computer sends inventory to its managing Notification Server, the Last Inventory Received time is recorded in the database. After the first initial forward for a resource, a record for the inventory forwarding event is written to the database. On a subsequent forwarding process, each resource's Last Inventory Received time is compared against the Last Inventory Sent time. If the received time is more recent than the send time, the inventory data defined in the forwarding rule is sent to the destination Notification Server. For the most accurate reporting, ensure that all client computers are updating their inventory at regular intervals.
Inventory forwarding creates Notification Server Events (NSE), and each NSE contains one dataclass—the corresponding data for each relevant resource is included in the NSE. For example, a forwarding rule specifies forwarding of resources contained in Collection A, and Inventory Classes B and C. When the rule is run, collection A contains Resources 1 and 2. Resource 1 has data in inventory classes B and C. Resource 2 only has data in inventory class B. Thus, when the rule is run, two NSEs will be posted to the target computer. One NSE will contain data for inventory class B and include resources 1 and 2. The other NSE will contain data for inventory class C and include resource 1 only. See Figure 6.
Note: Resource 3 data does not get forwarded.
Figure 6 – NSE File Creation
Additionally, inventory forwarding can use multiple threads. This is controlled by the MaxInvForwardThreads variable in coresettings.config. The default value is 2.
Inventory Forwarding is fault tolerant in the sense that if the forwarding event is interrupted or terminated before completion, then it will pick up from where it left off. It also forwards the resources contained in the collections specified in the rule, not the collections themselves.
The database tables involved in forwarding are ResourceUpdateSummary, ForwardingHistorySummary, and ForwardServer.
Step 2: Configure the Inventory Forwarding Rule
Once you are ready to forward data, the next step is configuring the forwarding rule. This is done by right-clicking the Inventory Forwarding folder and selecting New > Inventory Forwarding Rule. An existing forwarding rule can also be cloned and you would only need to modify the parameters that need to be changed.
Figure 7 – Naming the Rule
Configuring the Rule – Naming and Describing the Rule
The first task at hand is to name the rule. Provide a name that is descriptive of the focus of the rule.
Example: Local Security Solution Data to Reporting NS (Reporter)
In the example shown in Figure 6, providing a detailed name tells others who view the rule that it is forwarding all Local Security Solution dataclasses to the reporting Notification Server (In this case called REPORTER).
Also provide a description of the rule as well. For example, provide a list of the prefixes for the included dataclasses or the solution from which the dataclasses are coming from. Use something that accurately describes the purpose of the rule.
Configuring the Rule – Selecting Resources
After naming the rule, you need to select the required Resource collections so that the rule can know to which resources this rule will apply. Computer or User resources may be selected here, or a combination of both. Custom SQL to define the collections can also be used. See Figure 8.
Figure 8 – Resource or Collection Selector
Deeper Details – Selecting Resources
If you want to forward resource inventory data to another Notification Server, it should only be present in one Inventory Forwarding rule. Having multiple forwarding rules applying to the same collection will result in inconsistent reports on the destination Notification Server. Assure that the resources used are not included in multiple separate forwarding rules.
Because you can specify one or more collections from which inventory resources are forwarded, we recommend using collections that contain the bulk of the resources. Membership of collections is updated on user configurable intervals, and the membership of any collection that has any policy applied to it will be updated more frequently. If you use a collection that does not have a policy (either disabled or enabled) applied to it, the membership of the collection will only be updated by viewing the collection membership in the console. After an initial forward, only changes are forwarded, and thus selecting a collection that is not being updated results in little or no information being forwarded. The key here is to select a "high use" collection or collections.
Configuring the Rule – Selecting Inventory Classes
Configure the rule to use the required Resource collections, and choose the appropriate Inventory classes.
Figure 9 – Dataclass Selector
Deeper Details – Selecting Inventory Classes
Dataclasses sent by the source Notification Server that do not exist on the destination Notification Server are created on the destination provided the coresettings.config value 'AutoClassRegistration' is set to true (on the destination server). Currently, event data cannot be forwarded.
Configuring the Rule – Selecting a Destination (Reporting) Server
Selecting this option lists all of the available Notification Servers to forward to. If none are listed, simply enter the name of the destination Notification Server. The Notification Server Web URL will be populated automatically. You can also use IP addresses, fully qualified domain names, and NETBIOS names.
Figure 10 – Destination Server Selection Box
Configuring the Rule – Credentials
In order for data to be forwarded, authentication between the two servers is necessary and, thus, proper credentials must be given. Select Credentials and the following screen appears, as shown in Figure 11.
Figure 11 – Credentials Page
Configuring the Rule – Replication Schedule
Next, you need to configure the replication schedule. After you click the Apply button, a forwarding scheduled task is created on this managing Notification Server, and an item replication scheduled task is created on the destination Notification Server. Please note that this is a required step. Inventory Forwarding rules cannot be created without assigning them to a schedule.
Deeper Details – Item Replication and its Schedule
Item Replication is the process of replication of resources from the source to the target computer. It does NOT replicate inventory data—it merely replicates bare details (its Guid, name, and OwnerNS Guid)—for that resource only.
The first time a forwarding rule runs, the resource is created on the destination Notification Server and all relevant dataclasses are forwarded. Item replication causes the resource associations to be created. Each subsequent time the Replication Schedule runs, a manifest is generated on the source server. A current snapshot is taken and compared to the previous snapshot. Any differences (missing items) are marked as missing items. The hash entries are compared against the local item table hash entries. The mismatched items have an export request generated. This request is then sent back to the source server, and the mismatching items are exported and imported by the destination server.
Also, during the item replication process, the resources on the source and the target computer are compared. Items on the source computer that do not exist on the target computer are imported. Items on the target computer are re-imported if they are different to those on the source. Items on the target computer that do not exist on the source computer are deleted. This communication is all done as an MD5 hash of the .xml data file.
Forwarding rules are executed based on their defined schedule. If a forwarding process is currently running when the next scheduled process time arrives, the second process does not start. Only one instance of a forward rule is active at a time. A daily schedule repeated at a regular interval, such as "Every 6 hours from 7:20AM for 19 hour(s), every day," would ensure the destination Notification Server is kept up-to-date.
The database tables involved in item replication are ItemReplication and RemoteToLocalHash.
Configuring the Rule – Verification Schedule
This menu choice allows you to set the optional data verification schedule. Data verification uses a stored data hash for each dataclass forwarded, on a per resource basis. This hash is stored on both the managing Notification Server and the destination Notification Server. When the verification schedule runs, if the hashes match, the related data is considered verified. If the hashes do not match, the data is marked and will be re-forwarded on the next forwarding schedule. For the page location to set the Data Verification schedule, see Figure 12.
Also note that verification of data is set to the default of 100%. This default setting means that all of the data configured to be forwarded will be verified. This can potentially cause additional load on the servers and may be reduced to meet specific Notification Server performance needs.
Deeper Details – Verification Schedule
To assure that the data forwarded is accurate, data verification is performed by comparison of data hashes. A per resource, per dataclass hash is stored in the ResourceUpdateSummary table on both the source and the target computers. The verification schedule compares these hashes, and if they mismatch, then the associated data is flagged for re-forwarding on the source server at the next scheduled forwarding event.
The data verification schedule will only perform the hash comparison if a minimum period has elapsed since the last forwarding event. This period is defined by the ForwardInventoryIntervalMins setting in coresettings.config on the source computer and is set to "1440" (1 day) by default.
Data that has been verified once is ignored on subsequent runs of the verification schedule until an interval—specified by 'InvForwardingVerifyDaysThreshold' in coresetting.config—has passed. It is then re-verified at the next run of the verification schedule. 'InvForwardingVerifyDaysThreshold' is set to"30" by default.
Tables involved in data verification:
ForwardingHistorySummary and ResourceUpdateSummary.
Figure 12 – Replication and Data Verification Schedules
Preparation for Forwarding - Available Disk Space
If your environment is larger, or you track a lot of items as part of inventory, you may run the risk of running out of temporary disk space, while the xml files (NSE's) are being generated. Until these files are created, disk space is used to prepare for the NSE file generation. This space gets consumed from the configured TEMP variables. These temp variables are controlled by the particular user defined. This user is set when configuring forwarding credentials for the source and destination servers.
Assure that these variables are pointing to disk drives that have plenty of available and open disk space. Usually 20-30 GB is a good number. Also, familiarize yourself with some of the larger dataclasses that you will be forwarding, (such as Aex SW Audit Software) and review those data classes in a Resource Manager window, for a few machines to get a feel of the data size for that class.
Forwarding Specific Data
By default, all classes are selected to be forwarded. Again, remember that all solutions having data forwarded must be installed on both the source and destination servers. (Transfer time and database size can be reduced on the destination Notification Server by selecting only the necessary classes.) A minimum recommendation is to always include the Basic Inventory and the AeX ID Computer dataclasses in the forwarding rule. Keep the final reported data in mind as choices are made for the classes selected for each forwarding server. The Hardware, Operating System, and Software groups of classes are the most common groups to include in a forwarding rule. The Software group will generally contain the most data and consume the most database space on the destination Notification Server.
If you need all data, then consider creating multiple forwarding rules. An example would be a rule to forward the basic inventory classes, a rule to forward hardware classes and a rule to forward software classes. These rules would need to be staggered so that they do not overlap each other.
The single report provided for inventory forwarding is the Inventory Forwarding Analysis report. This report is found under the reports tab at Reports > Notification Server > Infrastructure > Server > Inventory Analysis Reports. This report shows data discrepancies due to the latency in the processes and is based on data found in Inv_Aex_AC_Resource_Inventory_Summary. See Figure 13.
Figure 13 – Inventory Forwarding Analysis Report
Because forwarding processes are on independent schedules, you can use the report to view which processes have not caught up yet. Data for this report is generated daily by the NS.Resource Inventory Summary Update scheduled task.
For this report to be accurate, the scheduled task must run at the source server, there must be data forwarded to the destination Notification Server, and the scheduled task must then run at the destination server.
Purging Resources from the Destination Notification Server
When client computers or resources are removed from a source Notification Server database, no data is sent to the destination Notification Server to indicate that it is no longer a valid resource. Essentially the destination Notification Server does not know that the client computer does not exist anymore. Purging must be enabled on the destination Notification Server to remove any resources that have not updated their data in a defined period. Purging is the only way that you can remove computers from the destination server. This functionality is by design, thus allowing historical reporting on older, expired client computers.
Class changes are deleted on the destination database on data forwarding. Deleted computers will show up as unmanaged in the item replication process. You can specify the classes that you want forwarded in the collection picker. Every verification tag on the client computer that is older than 30 days is removed. With the tag gone, the procedure re-verifies the data and retags it with the new date.
The inventory forwarding process events—Begin, Status and End—are logged as information-level events on the source Notification Server. Received confirmations for inventory and data are logged as information-level events on the destination Notification Server when data is successfully imported to the database. All unexpected and critical errors are logged under Error- or Warning-level events.
While configuring and tuning inventory forwarding in your environment, you should enable Information level logging on both source and destination Notification Server. After tuning, you should raise the logging level to Error and Warnings only to avoid unnecessary logging. You should only enable Trace level logging when requested by Altiris support for troubleshooting purposes.
Make sure your lower level NSes have accurate inventory. Items such as missing serial numbers, duplicate computer names, GUID's etc. can cause some big problems with data once it reaches the top level NS.
Refreshing your data on the lower tiered servers more frequently will provide cleaner data results on the top tiered server. CleanBeforeRun inventories should NOT be run more frequently than once per week and then during off peak times unless you can guarantee that the NSEs from all clients have processed prior to the inventory forwarding task running. Having an overloaded Notification Server will cause SQL deadlocks to occur, causing data classes to fail to forward. Monitor the A.log files on your lowered tiered Notification Server for these forwarding errors and take steps to reduce the load on your server to prevent these from occurring.
Intermediate Reporting Servers
Make sure you forward your data from your lower Notification Servers to ALL top level servers at the same time. DO NOT forward from one top level to another top level, and then to another top level. That middle server will be overworked trying to process all the incoming forwarding data and then forward it all back out again.
Include All Destination Servers
Make sure you include ALL servers you are forwarding to in a single forwarding rule. It will cause undue burden on the Notification Server to mitigate multiple forwarding rules, when you are forwarding to multiple servers. Otherwise, this causes issues with contention for system and DB resources as both rules could be vying for the same tables and data classes. As shown in Figure 14, Make sure all of the necessary servers are checked.
Figure 14 – Location to Select Multiple Servers
Properly Configure Forwarding Schedules
Make sure you offset your forwarding schedule on the lower Notification Servers from each other. If you don't, your top level or reporting Notification Server will get hammered with too many events to process from too many client-facing Notification Servers.
Forwarding schedules will vary with each customers needs. Some customers may need to forward inventory nightly. You may want to customize your forwarding to coincide with your inventory. You may not require all data classes to be forwarded each day. You may only want to forward those data classes that you have updated inventory for, especially in large environments. This helps alleviate unnecessary network traffic as well as the processing necessary on the both lower and top-level NSes.
Correctly Configure Verification Rules
Make sure your verification rules are set far out enough so that your forwarding is complete and all the resulting NSE's have been processed. Data inconsistencies can result if your checks are occurring before all the data is properly posted in the DB.
Optimize SQL. Review the recommendations in the "SQL Tuning and Maintenance" document, KB 17079. It may be helpful to adjust the processing and memory allocation in SQL to help process the events depending on the role of the server. Make use of /3GB, PAE, /USERVA and AWE if your system has the capability of utilizing these. On the top tiered Notification Server, with on-box SQL, you may consider de-allocating a processor from SQL. Also, if RAID is in your environment, consider configuring the raid controller for the drive set used by SQL to 25/75 read/write.
There are 2 things that will help the efficiency of the system and should be part of your maintenance plan: 1) defrag your disks and 2) defrag the database indexes.
Verify that your Anti-Virus scanner is set to exclude the NSCAP queues as that can slow event processing as well
Notification Server Queue Optimization
Adjust your queue settings to make sure you are processing events using the right throttling settings. Threads are allocated to the EvtQueue and EvtQFast folders are set based upon the number of processors in the server. MaxConcurrentFastMsgs are set to the number or processors in the NS and MaxConcurrentSlowMsgs are set to one half the number of processors in the NS. The EvtQSlow queue has 1 thread assigned. Each of these queues has a registry setting that determines the size of NSEs these Queues process. FastQueueThreshold is set to 15KB, SlowQThreshold is set to 1 GB. The EventQ Threshold will be above the FastQueueThreshold setting and below the SlowQThreshold setting. Since the top tiered NS shouldn't have any clients reporting to it (except itself) there may be some some efficiencies gained by adjusting these settings. For example, you may find that adjusting the thresholds up allows the EvtQueues and EvtFastQueue to handle more of the load.
Make sure ALL the solutions installed on the lower level Notification Servers are installed at the top level so the data classes can forward properly.
Adjust Top Tier Queue Size
If necessary, modify the queue size on the top tiered server. The default queue sizes can cause the event queues to fill too quickly, causing the NSE's being forwarded to be rejected.
Adjust Forwarding Threads
Inventory Forwarding by default utilizes two threads. These are controlled by the coresettings.config value MaxInvForwardThreads. If you are seeing deadlocks on the forwarding server causing Inventory Classes to fail to forward due to SQL deadlocks, reduce this value down to 1 thread. You should see substantial numbers of deadlock victims, prior to adjusting this. Do not adjust only for deadlock warnings.
Adjust Collection Refresh Scheduled Tasks
If possible, stagger the collection update and policy update intervals so that they don't run during the time that inventory forwarding is occurring.
Stop Processing of NSE's
It may be necessary to stop the processing of NSEs on the client-facing Notification Server while inventory forwarding is occurring. If the log files on the client-facing Notification Server have a substantial amount of deadlock victim error messages, this would be appropriate. This can be done by stopping the "Altiris Client Message Dispatcher" Service. KB 31616 provides a VBScript and process for doing this automatically.
Inventory Clean Up
The spInventoryCleanup stored procedure is associated with the ns.clean-up old inventory data schedule. It is scheduled by default to run every night at 12:00 AM. Currently, the query contained in this procedure has not been optimized fully and can cause deadlocks to occur. Reschedule this process to run when forwarding is not occurring. The state data in the item table for this schedule will need to be modified to do this.
The NS.Internal Licensing Refresh Item is scheduled to run every half hour. Certain solutions have licensing queries that can cause deadlocks. Increase the time interval so that this runs on up to every 4 hours. Stagger this task to run on quarter or half hour intervals, so that it doesn't run at the same time as the ns.clean-up old inventory data schedule. The state data in the item table for this schedule will need to be modified to do this.
There's an extra index on ForwardingHistorySummary that negatively affects concurrency. Run the ForwardingHistorySummary SQL script, found in KB 32719. It will drop the problem index, and optimize the stored procedure used to insert into the ForwardingHistorySummary table.Appendix A: Common Issues and Solutions with Inventory Forwarding
Inventory forwarding is running but not completing on NS 6.0.5287 (SP2)
Problem: Inventory forwarding is shown as running for a given forwarding rule, yet it does not complete after running for a number of days. Why?
Cause: If the database becomes unavailable during inventory forwarding, the process can hang and fail to complete, thus preventing other inventory forwarding schedules to run.
Resolution: Inventory forwarding (unlike other scheduled tasks) runs in the ScheduleProcessor.exe process for only a short time and completes its main process within the AeXSvc.exe process space (the "Altiris Service" Windows service). To recover from this inventory forwarding problem you need to restart the "Altiris Service" (AexSVC.exe, in Services).
Inventory forwarding causes error 503 on the sending Notification Server and "EvtQSlow is full--returning busy" on the receiving Notification Server
Problem: Inventory forwarding is causing "HTTP error: 503 Service overloaded (-2147213300)" errors on the sending Notification Server and "NS Event Queue (EvtQSlow) is full—returning busy errors on the receiving Notification Server.
Cause: Notification Servers with high client count and/or frequent inventory changes can produce more data during inventory forwarding than the default configurations on the receiving Notification Server will accept. By default, the maximum file queue size is set to 512000 KB (500 MB).
Resolution: Increase the value of HKLM\SOFTWARE\Altiris\eXpress\Notification ServerMaxFileQSize(KB) in the registry on the receiving Notification Server to resolve the issue. Setting the value somewhere between 2 and 5 GB (converted into KBs) will address the needs of most enterprise environments.
Memory leak after Inventory Forwarding schedule runs
Problem: After inventory forwarding runs, memory usage grows rapidly with some systems experiencing OutOfMemory exceptions and Altiris service crashes.
Environment: Notification Server 6.0 SP3 (6.0.6074)
Cause: Inventory forwarding on Notification Server 6.0 SP3 utilizes the Item Replication framework, a technology which will form the foundation for Hierarchy and is part of Altiris 7. Item replication enables users to distribute not only Notification Server resources, but all data associated with a resource. The initial revision of inventory forwarding could only deal with Inventory data, but with the advent of item replication we can now forward Event data.
If a Notification Server resource is forwarded as part of an Inventory Forwarding rule all Event data associated with the resource (such as Client Config Generation and Client Package Info) is loaded into memory by a background task. This is an unintended regression from SP2 and will be addressed in a hotfix. On a large system this can result in the Notification Server attempt to load several gigabytes of data, eventually resulting in all available system memory being consumed causing the Altiris service to crash.
The following registry keys control the inventory forwarding process. All keys are found under HKLM\SOFTWARE\Altiris\eXpress\Notification Server. Before modifying any settings, carefully consider the effect on the forwarding process.
|InvForwardFullLoggingEnabled||REG_DWORD||0||Enables full logging when Notification Server to Notification Server inventory forwarding is taking place. There are two valid values for this key: 0 and 1.
1 = enable extra logging
0 = disable extra logging
Note: This registry setting does not get added at installation time. You must create it when you need it.
|ForwardInventoryIntervalMins||1440||The time that the Inventory Forwarding is expected
to take. If data is not seen at the destination Notification Server within this interval, you can assume it was lost in transit.
The data verification schedule will then force the data to be sent again.
The default value is 1440 (one day).
|MaxInvForwardThreads||REG_DWORD||2||The number of threads per forwarding rule. This key is used by the component that handles Notification Server to Notification Server inventory forwarding. Using this key can help to decrease forwarding times. This is set on the source Notification Server.|
|MaxFileQSize(KB)||REG_DWORD||512||[destination Notification Server] By default, the file based queue size is 512 KB. Although increasing this value can help the destination Notification Server to reject data, balancing the processing thread counts on the source and destination Notification Server is preferred.|
Because inventory forwarding makes use of the normal data loading mechanism of the Notification Server (NSE files), a short discussion of how that occurs is included here for reference.
- The Altiris Agent message Dispatcher Service monitors the file-based queues for NSE files. Stopping the Dispatcher Service stops the processing of files from the queues.
- The Dispatcher Service is multi-threaded and processes multiple NSE files at once.
- The EvtQFast queue has 10 threads allocated to processing NSE files. The number of threads is defined by the MaxConcurrentFastMsgs element of the coresettings.configfile.
- The EvtQueue queue has three threads allocated to processing NSE files. The number of threads is defined by the MaxConcurrentSlowMsgs element of thecoresettings.config file.
- The EvtQSlow queue is limited to a single processing thread. This can't be modified, and protects against excessive memory usage by Notification Server.
- The EvtQLarge queue is NOT monitored by the Dispatcher Service and NSE files sent to this queue aren't processed. We don't recommend files greater then 20 MB, as they consume too much memory. However, you can process them manually.
- The processing threads extract the XML from the NSE and execute the dataloader process to write the data into the Notification Database. After the XML data has been successfully written to the Notification Database, the NSE file is deleted.
Altiris® Notification Server – Inventory Forwarding