Hi,
which version of SEP 14 did you run?
Did you open a ticket with support?
There is a fix in SEP 14.0 RU1 MP1 which could address this:
Slow processing of .dat files in AgentInfo
Fix ID: 4099486
Symptoms: AgentInfo .dat files are slow to process.
Solution: Resolved an inefficient database call in AgentInfo processing.
There is also a change within the http.conf of the SEPM apache server.
The "ConnectionsToQueuePerChild" has been adjusted to achive a more stable SEPM.
At the end, Support will help you to determine the root cuase of the issue.
Please think also about some tuning options for the underlying server OS.
TcpTimedWaitDelay
https://technet.microsoft.com/en-us/library/cc938217.aspx
Avoiding TCP/IP Port Exhaustion
https://msdn.microsoft.com/en-us/library/aa560610(v=bts.20).aspx
This will help to reduce the amount of open TCP connections and also help to close TCP connections where the transmission was successfully, but the ACKNOWLEDGE wasn't received by the SEPM.