Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

12.1.3 Replciation fails

Created: 07 Nov 2013 • Updated: 07 Nov 2013 | 45 comments

Hi Guys,

I am trying to replicate between two 12.1.2 servers and i am unable to replicate it fails, what should i do?

Error in replication-o.log

THREAD 106 WARNING: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Got untrusted Certificate Chain
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:804)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:515)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1090)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
at com.sygate.scm.common.communicate.Communicator.getPostInputStreamWithProxy(Communicator.java:509)
at com.sygate.scm.common.communicate.Communicator.postRequestWithProxy(Communicator.java:471)
at com.sygate.scm.common.communicate.Communicator.postRequest(Communicator.java:464)
at com.sygate.scm.server.consolemanager.requesthandler.ReplicationHandler.getRemoteSite(ReplicationHandler.java:344)
at com.sygate.scm.server.consolemanager.requesthandler.ReplicationHandler.handleRequest(ReplicationHandler.java:177)
at com.sygate.scm.server.consolemanager.RequestHandler.handleRequest(RequestHandler.java:369)
at com.sygate.scm.server.consolemanager.RequestHandler.<init>(RequestHandler.java:107)
at com.sygate.scm.server.servlet.ConsoleServlet.doPost(ConsoleServlet.java:87)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at com.sygate.scm.server.servlet.ConsoleFilter.doFilter(ConsoleFilter.java:69)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.security.cert.CertificateException: Got untrusted Certificate Chain
at com.sygate.scm.server.util.ServerCustomSSLSocketFactory$1.checkServerTrusted(ServerCustomSSLSocketFactory.java:112)
at sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:813)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1323)
... 39 more
com.sygate.scm.common.communicate.CommunicationException: Unexpected server error. ErrorCode: 0x10010000
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 
2013-10-31 12:55:30.203 THREAD 104 WARNING: ReplicationHandler>> getRemoteSite: Got a mismatched certificate from remote server {10.132.100.55}
2013-10-31 12:55:30.203 THREAD 104 INFO: ReplicationHandler>> Cert of remote server {10.132.100.55} is: 308202CE30820237A00302010202107B6A4AB3F518AF85377A643DA12AD5DF300D06092A864886F70D0101050500307D317B300906035504080C024341300A06035504061303555341300A060355040B0C03455347301406035504070C0D4D6F756E7461696E2056696577301B060355040A0C1453796D616E74656320436F72706F726174696F6E302306035504030C1C5345504D2D44524859442D30312E6D61686162616E6B2E636F2E696E301E170D3133303131373035323335375A170D3233303231343035323335375A307D317B300906035504080C024341300A06035504061303555341300A060355040B0C03455347301406035504070C0D4D6F756E7461696E2056696577301B060355040A0C1453796D616E74656320436F72706F726174696F6E302306035504030C1C5345504D2D44524859442D30312E6D61686162616E6B2E636F2E696E30819F300D06092A864886F70D010101050003818D0030818902818100E55EAB3958CA65EE75A941CC3696B38BB2131A34BF5499E7FD10339FCCD5B17290BBD60597EAD7C57EB9E0157D72881DB4F57ECF9BC303F851C82CAED69D0AE10E6DE124C968F7FED067688ABBEF760559DB4B4FEBFE6D039D0C9DAC703E575C5E0390BDCBDD1BA31116172CCA979E3E43E2E143F8703A8D57DFBA16133A9AF70203010001A34F304D304B0603551D1104443042821C5345504D2D44524859442D30312E6D61686162616E6B2E636F2E696E820D5345504D2D44524859442D303187040A84149B820D31302E3133322E32302E313535300D06092A864886F70D0101050500038181003662EE3D76E4110245565FA1B3F6189DCEBE74EDDA6435476C3CC03CB9AE059BB4FBF436441072370D35975C5C683B6BBB7BA90CDE842BD8B24C805BBDD26D5735B75ACE11F8BA2FFCED64417E318EE0E5147AEE81AD4985FE19D15D9BF2EA28F28BD56F760BF2FCB74CEC8DDC646069ADA9639833099D3FD32FC184D5B418FD
2013-10-31 12:55:30.203 THREAD 104 WARNING: ReplicationHandler>> handleRequest: returning...
2013-10-31 12:55:30.203 THREAD 104 WARNING: ReplicationHandler>> handleRequest: Done!
2013-10-31 12:55:30.203 THREAD 104 FINE: ------------ Thread stopped --------------
2013-10-31 12:55:33.359 THREAD 104 FINE: ------------ Thread started --------------
2013-10-31 12:55:33.359 THREAD 104 WARNING: ReplicationHandler>> handleRequest: Begin...
2013-10-31 12:55:33.359 THREAD 104 WARNING: ReplicationHandler>> handleRequest: action preRegister
2013-10-31 12:55:33.359 THREAD 104 WARNING: ReplicationHandler>> handleRequest: RpcData size=0
2013-10-31 12:55:33.359 THREAD 104 FINE: Borrow connection from pool.
2013-10-31 12:55:33.359 THREAD 104 FINE: Return connection to pool.
2013-10-31 12:55:33.359 THREAD 104 WARNING: ReplicationHandler>> getRemoteSite: Begin...
2013-10-31 12:55:34.593 THREAD 104 WARNING: Object type returned from Login Response  = SchemaContainer Expected object for this version = SchemaContainer 
2013-10-31 12:55:34.609 THREAD 104 FINE: Borrow connection from pool.
2013-10-31 12:55:34.609 THREAD 104 FINE: Return connection to pool.
2013-10-31 12:55:34.609 THREAD 104 WARNING: ReplicationHandler>> getRemoteSite: local site doesn't have remote site's SemSite object.Remote server is:10.132.100.55
2013-10-31 12:55:34.609 THREAD 104 WARNING: ReplicationHandler>> handleRequest: returning...
2013-10-31 12:55:34.609 THREAD 104 WARNING: ReplicationHandler>> handleRequest: Done!
2013-10-31 12:55:34.609 THREAD 104 FINE: ------------ Thread stopped --------------
Operating Systems:

Comments 45 CommentsJump to latest comment

pete_4u2002's picture

looks like certificate issue " Got untrusted Certificate Chain" ,

did you delete the replication earlier?

Jeshrel's picture

I deleted non, it automatically stops and even if i run management server configuration wizard it says database schema does not match. I also tried changing sem5.log to sem.log.old and ran the command from command prompt still same issue.

The only way for me to reinitiate replciation is to re-install SEPM adn then start again.

what should i do?

SMLatCST's picture

This might be a silly question, but have you verified both servers are running 12.1RU2?  It's just that you mention RU3 in the thread subject heading and that you're seeing "database schema does not match errors".

Rafeeq's picture

Database schema error is when both SEPMSs are not on same version. Kindly double check.

Is this the first time replication setup? or you were able to replicate successfully earlier.

Jeshrel's picture

No. it failed.

My primary server is intact and functional.

pete_4u2002's picture

SEPM have to be same version , are you sure both of them are on 12.1 Ru2?

pete_4u2002's picture

does remote site have other SEPM's details on both SEPM?

Jeshrel's picture

Yes, the primary SEPM has created a 3 folder by the name remote site with replication fodler in it.

The secondary server i am not able to open SEPm console after replication.

Jeshrel's picture

Found this error in scm-server log

2013-10-31 12:51:28.234 THREAD 12 SEVERE: ================== StartClientTransport ===================
2013-10-31 12:51:28.234 THREAD 12 SEVERE: [12] Initializing Tomcat ClientTransport Key for Secars...
2013-10-31 12:51:28.250 THREAD 12 FINEST: ProcessManager>> Unregistered the Process java.lang.ProcessImpl@b948fe
2013-10-31 12:51:28.265 THREAD 12 FINEST: ProcessManager>> Unregistered the Process java.lang.ProcessImpl@672132
2013-10-31 12:51:28.265 THREAD 12 FINE: Tomcat ClientTransport Key generated
2013-10-31 12:51:28.281 THREAD 12 SEVERE: startClientTransport>>resetAgentToOfflineState=true
2013-10-31 12:51:28.281 THREAD 12 SEVERE: startClientTransport>>start set agent offline
2013-10-31 12:51:28.312 THREAD 12 SEVERE: startClientTransport>>start set agent offline
2013-10-31 12:51:28.703 THREAD 12 INFO: LicenseManager>>computeDeployedClientCount>>Feature type = SEP specificClients:0 commonclients:0
2013-10-31 12:51:28.890 THREAD 12 FINE: Task scheduled
2013-10-31 12:51:28.890 THREAD 12 SEVERE: Schedule is started!
2013-10-31 12:51:28.921 THREAD 12 INFO: LiveUpdateTask>>checkLiveUpdateDisabledStatus>>Update SemServerState sucessfully, LiveUpdateDisabled= false
2013-10-31 12:51:28.984 THREAD 12 SEVERE: SEPM Service started
2013-10-31 12:51:30.187 THREAD 32 INFO: Initial UpdateGroups information
2013-10-31 12:51:31.093 THREAD 32 INFO: AgentOnlineStatusTask.updateInvalidAgentCache url = : http://localhost:8014/secars/secars.dll?action=31&reset=1
2013-10-31 12:51:33.937 THREAD 31 FINE: RequestHandlerCache >> No domains to refresh
2013-10-31 12:51:33.937 THREAD 31 FINE: RequestHandlerCacheRefreshTask >> No admins to refresh
2013-10-31 12:51:35.703 THREAD 58 INFO: <Throttler.isThrottled> Initialized Throttler. Low mark = 20 High mark = 30 Leak = 20
2013-10-31 12:51:35.703 THREAD 58 INFO: <Throttler.isThrottled> Initialized Throttler. Low mark = 40 High mark = 50 Leak = 20
2013-10-31 12:51:35.781 THREAD 58 SEVERE: 
com.sygate.scm.server.agentmanager.InvalidDomainIdRegistrationException: Client ******* is trying to register with invalid Domain ID ****************************************.
at com.sygate.scm.server.agentmanager.AgentRequestHandler.registerClient(AgentRequestHandler.java:1731)
at com.sygate.scm.server.agentmanager.AgentRequestHandler.agentRegister(AgentRequestHandler.java:919)
at com.sygate.scm.server.agentmanager.AgentRequestHandler.handleRegistrationRequest(AgentRequestHandler.java:875)
at com.sygate.scm.server.agentmanager.AgentRequestHandler.handleRequest(AgentRequestHandler.java:280)
at com.sygate.scm.server.servlet.AgentServlet.doPost(AgentServlet.java:57)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Chetan Savade's picture

Hi,

Thank you for posting in Symantec community.

Have you restored old certificates to existing replication partners, if yes refer the following article:

Certificate mismatch after restoring old certificates to existing replication partners

http://www.symantec.com/docs/TECH186962

Refer this article as well: Getting error: Replication failed. The certificate for replication partner Site is not trusted.

http://www.symantec.com/docs/TECH139540

Solution:

1) In the Symantec Endpoint Protection Manager Console, under "Admin" > "Servers", select the replication server.

2) Under "Tasks", select "Check Certificate".

3) Click "Yes" on the dialog box that pops up asking you to trust the certificate.

4) Start a manual replication.

Chetan Savade
Sr.Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

Rafeeq's picture

are able to login to your primary sepm? as per the logs its showing unexpected server error.

make sure that free space is 5times the SEPM DB. Enable advance loggin and post the replication logs

Jeshrel's picture

Yes it has, i have attached the logs please check.

The first attempt after enabling finest was on 31 Oct..

pete_4u2002's picture

are you still seeing replication failure?

can you perform offsite replication ?

Jeshrel's picture

Hi,

I started the replciation again,  its run for more than a hour now. The database on the replicating server(secondary server) is still 260 MB wat can i do more to fix the issue?

pete_4u2002's picture

is it still running?

is replication over internet ?

Jeshrel's picture

It is still running is over LAN?

Both are in different locations in india and is using a Leased line.

pete_4u2002's picture

let it run , the LAN should not take much time.. however there are other network devices between these SEPM there might be delay..

Jeshrel's picture

260 for two hours known isn't it less.

pete_4u2002's picture

yeah, can you check if the data.zip is copied on the remote site under data\replication\inbox?

Jeshrel's picture

No, There is no inbox in the remote site(primary server) i am only able to find outbox in data\replication

Chetan Savade's picture

Hi,

Following error is captured according into the scm-server-1.log.

com.sygate.scm.server.agentmanager.InvalidDomainIdRegistrationException: Client SERVER is trying to register with invalid Domain ID 69F33DD4C0A8008D00DCB5EB2903571C.

Try the possible solution given in the following article:

Symantec Endpoint Protection client is trying to register with an invalid Domain ID

http://www.symantec.com/docs/TECH90852
 

Chetan Savade
Sr.Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

Jeshrel's picture

Hi Chetan,

The server that is replicating is a fresh install of 12.1.3 i didnot make any client communicate with it but still i get this error its odd.

Now, this morning i unisntall the secondary SEPm and re-isntalled and am running a fresh replciation and it just running for more than two hours but no progress.

Chetan Savade's picture

Hi,

As the initial replication fails please gather Tomcat logs from Site 1 and “Install Error” logs from New Site.

What's the DB size on primary SEPM? What's the WAN link bandwidth?

Chetan Savade
Sr.Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

Chetan Savade's picture

Hi,

It should work however if failing then gather Tomcat logs from Site 1 and “Install Error” logs from New Site & attached to this thread.

Chetan Savade
Sr.Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

Jeshrel's picture

Hi Chetan,

Sure would, just verifing you want tomcat.log or logs inside tomcat\logs ?

Chetan Savade's picture

Logs inside tomcat\logs.

Chetan Savade
Sr.Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<

pete_4u2002's picture

is the data.zip folder created on relpication SEPM data\replication\outbox

Jeshrel's picture

Oops, i do see it but its inside data\replication\outbox\5A6A73170AFAFC3500B1F63379C50E73\data.zip

what does this mean?

pete_4u2002's picture

fine, this data is going to be placed on inbox folder of other SEPM. do you see it there?

Jeshrel's picture

I see a data.zip of 1.2 GB.

The the folder name in replication\inbox is not the same of the folder on the primary replication\outbox, does it matter?

pete_4u2002's picture

the SEPM having outbox folder will not have inbox folder until this 1.2 GB data.zip is placed under inbox folder of other SEPM.

earlier thread of your say 250 MB, is data.zip still building up?

Jeshrel's picture

In the earlier thread i has said 260 MB by looking at sem5.db size not data.zip.

And one more thing data.zip the last time it got modified was 03.40 PM

pete_4u2002's picture

ok, isnt the inbox folder created yet?

can you give this method a try if the replication does not work..

How to perform offline replication between 2 remote sites when normal replication is failing due to bandwidth issues

http://www.symantec.com/docs/TECH95122

Jeshrel's picture

Hi,

I followed the article

Primary DB size is 19.6 GB

Secondry server free space is 68 GB

At the end of replication it finised with error 

"Failed to copy data to folder C:\Program Files\Symantec\Symantec Endpoint Protection Manager\data, not enough free disk space"

At the end of replciation the drive on which i am trying to install still has  9 GB of free space. why does this error occur and how to complete my replication.
 

SameerU's picture

Hi

Is there any timeout value set between firewall between the two replication servers.

Regards

Jeshrel's picture

On both the servers windows firewall is turned off

SameerU's picture

Hi

I mean to say that is there any firewall in which there is a limit set for data exchange from one IP to another

Regards

Jeshrel's picture

Hi,

During replication everythings goes fine expect for the fact that the sem5.log is increasing and its size is equilent to that of sem5.db.

Jeshrel's picture

Hi,

Thanks but isnt this supposed to be done after installation if sem5.log is increasing in size.

In my case during replication sem5.log is increasing and replciation fails with error

"Failed to copy data to folder C:\Program Files\Symantec\Symantec Endpoint Protection Manager\data, not enough free disk space"

where in the drive has still about 10 GB of free space.

Jeshrel's picture

It did strike me to follow the article on saturday when replicatio failed but there was no ODBC present.

Chetan Savade's picture

Hi,

Is there any update? Replication is still failing?

Upon checking SEP 12.1 RU3 fix notes, following fix is listed there.

Replication failed

Fix ID: 2810324

Symptom: The replication fails continuously. The data.zip file is generated and transferred, but replication is not successful.

Solution: Fixed this issue by cloning the default management server list in the Enforcer's policy.

Chetan Savade
Sr.Technical Support Engineer, Endpoint Security
Enterprise Technical Support
CCNA | CCNP | MCSE | SCTS |

Don't forget to mark your thread as 'SOLVED' with the answer that best helps you.<