Role assignment changes do not save while ADSynchroniser processing is synchronizing users or groups.

Article:TECH195382  |  Created: 2012-08-22  |  Updated: 2013-07-23  |  Article URL http://www.symantec.com/docs/TECH195382
Article Type
Technical Solution


Environment

Issue



Role assignment changes in a Compliance Accelerator environment with over 100,000 Monitored Employees do not save while the ADSynchroniser process is actively synchronizing users and groups.


Error



The symptom is a role assignment will be selected and saved, only to disappear from the assigned pane and still show in the available roles to be assigned pane.

Equivalents to the following line pair can be found in a dtrace of the AcceleratorService process on the Compliance Accelerator server while this issue is being experienced:

(AcceleratorService)     <Cache polling thread:7440>     EV-L     {-}{CUSTOMERCACHE.EN_US}{C8.EN_US} ServerCache: Received cache update for object 'BO_Principals' for customerID '2'

(AcceleratorService)     <Cache polling thread:7440>     EV-L     {-}{CUSTOMERCACHE.EN_US}{C8.EN_US} ServerCache: object 'BO_Principals' for customerID '2' found in the cache and invalidated


Environment



- Compliance Accelerator 8.0 SP5 on Windows Server 2003 SP2 Enterprise Edition

- Multiple Active Directory domains with a combined user count over 100,000 that is synchronized to Compliance Accelerator as Monitored Employees.

 


Cause



The ADSynchroniser process uses some of the same resources, i.e., database tables, as the Role Assignment process.  When synchronizting thousands of Monitored Employees, either through Employee Groups or individually, the ADSynchroniser process temporarily locks those resource and does not release them within the default 2 second policy cache update period.  The result is that the cache update with any role assignment change is ignored and reset as though the role assignment change was never attempted while the ADSynchroniser process is actively synchronizing account information.
 


Solution



Workarounds

Two possible workarounds exist for this issue.  If Workaround 1 does not work, Workaround 2 should be implemented.

Workaround 1

Increase the Policy Cache timeout value from the default of 2 seconds to 10 seconds or greater using the following steps.

  1. Access the Configuration tab, then the Settings sub-tab in the Compliance Accelerator (CA) Client.
  2. Hold the Ctrl key and click on the Configuration Settings banner.  This will allow viewing and changing hidden configuration settings.
  3. Expand the Security section.
  4. Change the value for Permission Cache Option (may be in the only gray line in that section) from 2 to 0.  Do not click the Save button as yet.
  5. Change to and expand the System  section.
  6. Locate the Cache Policies File line (may be the 3rd gray line).
  7. Click on the Save As... option.
  8. Save the Cache Policies to a file on the CA Server's local disk (anywhere will do) - default file name is "Cache Policies File.txt".
  9. Edit the "Cache Policies File.txt" file to change the entry PollingInterval="2" to PollingInterval="10".
    1. Open this file in NotePad and use the Find feature to find PollingInterval=.  This will get to where the change needs to be made quickly.
    2. Save and close the file once the change has been made.
  10. Click the Browse option in the Cache Policies File option line (to the right of the Save... link).
  11. Browse to, select, and open the "Cache Policies File.txt" file.
  12. Click the Save button.
  13. Click the OK button to acknowledge the need to restart:
    1. Remoting
    2. Customer Background Tasks
    3. Client Application
    4. Journal Connector
    5. Process Manager
  14. Close the CA Client.
  15. Restart the Enterprise Vault Accelerator Manager Service (EVAMS) using the Services MMC snap-in on the CA Server.
  16. Restart each Journal Task where the Journal Connector is enabled.
  17. Restarting EVAMS with the new Cache Policy loaded does not cause the hidden setting Permission Cache Option to reset back to 2, so this must be done manually by repeating Steps 1 through 4.  Change the 0 to 2 and click the Save button this time.  A restart of either the Customer Background Task (CBT) through the EVBAAdmin site or EVAMS is required.

Workaround 2

Stop the automatic synchronization and run a batch job during off-work hours to synchronize with Active Directory or Lotus Domino directory using the following steps.

A.  Create a batch file to trigger the ADSynchroniser processing.

*** On the SQL Server hosting the CA Configuration database, logged on with an account that has proper permissions to read the data in the CA Configuration database ***

   1.  Run the following SQL query against the CA Configuration database to obtain the CustomerID of the CA Customer to be synchronized:

  • SELECT * FROM tblCustomer WHERE ATStatus = 21

   2.  Note the CustomerID associated with the CA Customer that is to be synchronized with Active Directory or Lotus Domino.
   3.  Run the following SQL query also against the CA Configuration database to obtain the ServerID of the CA Server (should be 1 if only 1 CA server exists)

  • SELECT * FROM tblServer

   4.  Note the ServerID associated with the name of the CA server hosting the CA Customer.

*** On the CA Server hosting the CA Customer, logged on as the Vault Service Account ***

   5.  In a folder that is to contain scheduled job batch files, create a batch file to contain the ADSynchroniser command and arguments.
   6.  Edit the batch file to contain the the ADSynchroniser command in the following format:

  • "CA Install drive\CA Install path\ADSynchroniser.exe" X Y a
    • For a CA server on a 32-bit Operating System, the default installation drive\path is "C:\Program Files\Enterprise Vault Business Accelerator
    • For a CA server on a 64-bit Operating System, the default installation drive\path is "C:\Program Files (x86)\Enterprise Vault Business Accelerator\ADSynchroniser.exe" X Y a

   7.  Replace drive\path with that of the CA installation on the CA server.
   8.  Replace X with the CustomerID obtained in Step 2 above.
   9.  Replace Y with the ServerID obtained in Step 4 above.
 10.  Leave the letter "a" argument alone as this denotes synchronizing all domains, groups, and users. For example, a CA server with the 32-bit OS default installation path, CA CustomerID of 5, and ServerID of 1 would have the following command line in the batch file:

  • "C:\Program Files\Enterprise Vault Business Accelerator\ADSynchroniser.exe" 5 1 a

 11.  Save and close the batch file.
 12.  Use a scheduling program to schedule this batch file to run during after work hours.

B. Configure Active Directory synchronization to not run automatically.

*** On a workstation with the CA Client installed, using an account with proper Application level permissions (such as the Vault Service Account) ***

 13.  Access the Configuration tab, then the Settings sub-tab in the Compliance Accelerator (CA) Client.
 14.  Expand the Profile Synchronization folder.
 15.  Locate the option Synchronize profile.
 16.  Uncheck the check box located under the Value column for this option.
 17.  Click the Save button to save this change.
 18.  Click the OK button to acknowledge the requirement to restart the Customer Background Tasks.
 19.  Close the CA Client.

*** On the CA Server, logged in as the Vault Service Account ***

 20.  Execute either of the following 2 options

a.  Restart the Enterprise Vault Accelerator Manager Service (EVAMS)

1)  Launch the Services MMC snap-in.
2)  Select the Enterprise Vault Accelerator Manager Service (EVAMS).
3)  Click on the Restart toolbar button.

b.  Restart the Customer Background Task (CBT)

1)  Launch Internet Explorer to the EVBAAdmin site (http://localhost/EVBAAdmin).
2)  Right click on the CA Customer in the left pane.
3)  Select the Customer Background Task option to remove the check mark to its left.
4)  Monitor the status of the CA Customer in the right pane to see it change from Running to Stopping to Stopped.
5)  Once stopped, right click on the CA Customer in the left pane again.
6)  Select the Customer Background Task option again, this time to place a check mark to its left.
7)  Monitor the status of the CA Customer in the right pane to see it change from Stopped to Starting to Running.

  21.  Allow the scheduled batch job to run during off-work hours to synchronize all Monitored Employees and Employee Groups in all domains.

 Note: an issue with CA 8.0 SP5 (8.0.5) and 9.0 (9.0.0) prevents successful group synchronizations when using the all domains, groups, and users argument in the synchronization command line.  For these versions of CA, an expanded batch file is needed with a line for each Employee Group to be synchronized.  To create this batch file, run the following SQL script, changing the path, first (CustomerID), and second (ServerID) arguments per Steps 1 through 9 above.

-- Begin Script, copy from below this line --

/* Declare a variable to contain the CustomerID value, to be obtained from the CA Configuration db's tblCustomer table: */
DECLARE @CustID nvarchar (5) SET @CustID = '2'  -- Default to CustomerID 2 - change to the appropriate CustomerID per Step 8 above.

/* Declare a variable to contain the ServerID value, to be obtained from the CA Configuration db's tblServer table: */
DECLARE @ServID nvarchar (5) SET @ServID = '1'  -- Default to ServerID 1 - change per Step 9 above.

/* Declare a variable to contain the synchronization scope.  Valid values are:
- “a” – Synchronize all
- “ad” – Synchronize Domain  (4th param is the specific domain ID. If not specified, then it sync all domains and server)
- “ag” – Synchronize all groups.
- “ae” – Synchronize all employees
- “u” – synchronize specific employee profile (4th param used to specific employee id)
- “g” – Synchronize specific group  (4rth param used to specify group id)
*/
DECLARE @SyncScp nvarchar (5) SET @SyncScp = 'g'  -- Default to Group as All and All Groups do not work in 8.0.5 and 9.0.0 to synchronize groups

/* Declare a variable to contain the Target Group ID for the Cursor: */
DECLARE @TGID nvarchar (10)

/* Declare the Cursor: */
DECLARE BatchFile CURSOR FOR
-- Obtain a listing of all active GroupID values and place them into the Cursor:
SELECT TargetGroupID FROM tblTargetGroup WHERE ADSync = 1
-- Open a looping mechanism to create all needed command lines for groups to synchronize:
OPEN BatchFile
FETCH NEXT FROM BatchFile INTO @TGID
-- While items remain in the Cursor, repeat the commands under the BEGIN statement:
WHILE @@FETCH_STATUS = 0
BEGIN
-- Create the command line to synchronize each GroupID - replace the drive and path to the "ADSynchroniser.exe" file per Steps 6 and 7 above
PRINT '"C:\Program Files\Enterprise Vault Business Accelerator\ADSynchroniser.exe" ' +@CustID + ' ' +@ServID + ' ' +@SyncScp + ' ' + @TGID
-- Get the next available GroupID
FETCH NEXT FROM BatchFile INTO @TGID
END
CLOSE BatchFile
DEALLOCATE BatchFile
-- Print command line to synchronize all domains, groups, and users - replace the drive and path to the "ADSynchroniser.exe" file per Steps 6 and 7 above:
PRINT '"C:\Program Files\Enterprise Vault Business Accelerator\ADSynchroniser.exe" ' +@CustID + ' ' +@ServID + ' a'
 

-- End Script, copy to, but not including, this line and paste into a Query window focused on the CA Customer database --

 

This issue has been resolved in Compliance Accelerator 10.0 SP4.  For more information, see DOC6264 in the Related Documents section below.

 


Supplemental Materials

SourceETrack
Value2871400
Description

Active directory synchronization can in very large environments take a long time to complete and may cause user access to slow


SourceETrack
Value2871471
Description

Users are not getting automatically deactivated in more than groups of 10 in a 24 hour period when deleted in Active Directory and synchronized to CA




Article URL http://www.symantec.com/docs/TECH195382


Terms of use for this information are found in Legal Notices