BUG REPORT - TLS negotiation with [IP] failed: the remote system violated the TLS connection protocol, Invalid padding

Article:TECH207414  |  Created: 2013-06-21  |  Updated: 2013-07-08  |  Article URL http://www.symantec.com/docs/TECH207414
Article Type
Technical Solution


Issue



The PGP Universal Server Symantec Encryption Server unexpectedly closes SSL connections.

The problem is not protocol dependent and might occur for all services on the server using SSL/TLS.

 

The problem was mostly seen with TLS encrypted SMTP connections or cluster communication between encryption servers.

In this cases, mails will be queued on the sending mail server or the cluster logs show an TLS error or a Java Exception.

A new connection is rebuild afterwards.


Error



For incoming mails the following error messages can be seen in the mail logs (/var/log/ovid/proxy*) with enabled debug logging

2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[D]: Trying to match client cipherSuite {00,39} on the server
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[I]: Selected cipherSuite {00,39} on the server, ephemeral=1
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[D]: sending server hello: Session ID [16]: bd065f17 809da373 b300cae6 70dac1fc
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[I]: incoming TLS event CE(3), current state 1; transitioning to the state 2(isClientSide=0,cloned=0)
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[D]: Received empty list of client certificates
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[I]: incoming TLS event CKE(10), current state 2; transitioning to the state 3(isClientSide=0,cloned=0)
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[I]: incoming TLS event CC(14), current state 3; transitioning to the state 5(isClientSide=0,cloned=0)
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[E]: Invalid padding at the end of the record: 0d v.s. padsize 07
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[W]: Sending fatal error because MAC check failed, setting TLS state to FatalError
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: TLS[I]: Client asked to close the session. Current state is FatalError
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987:pgpproxy: problems (-11249) starting TLS with client...disconnecting
2012/06/14 13:28:47 +02:00  ERROR  pgp/messaging[17604]:       SMTP-37987: TLSnegotiation with [10.X.X.X] failed: the remote system violated the TLSconnection protocol
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987:pgpproxy: problems processing client command (error -11249)
2012/06/14 13:28:47 +02:00  DEBUG  pgp/messaging[17604]:       SMTP-37987: C<- :421 mail.server.tld PGP Universal service not available, closing transmission channel

 

The error message indicating this problem is:

TLS negotiation with [10.X.X.X] failed: the remote system violated the TLS connection protocol

Environment



PGP Universal Server / Symantec Encryption Server up to version 3.3.0 MP2


Cause



Defect in the TLS/SSL implementation on PGP Universal Server / Symantec Encryption Server which might cause some TLS/SSL connections to fail.

This results in an incorrect session key being used in the record layer causing incorrect decryption which is caught by the invalid padding detection. The connection is then terminated for security reasons.

The connection is then re-established, therefore this error is mostly not noticed.


Solution



Upgrade to Symantec Encryption Server version 3.3.0 MP2 where this issue was resolved.


Supplemental Materials

SourceETrack
Value2821956


Article URL http://www.symantec.com/docs/TECH207414


Terms of use for this information are found in Legal Notices