What is the last rule in your firewall?
Can you post the NTP log from when the time outs occur?
This article may be of some help, although it refers to 11.x it should still work for 12.1:
Symantec Endpoint Protection 11.0 Network Threat Protection (Firewall) Overview and Best Practices White Paper
http://service1.symantec.com/support/ent-security.nsf/docid/2007121714495348
Also, check this MS article:
Service overview and network port requirements for Windows
http://support.microsoft.com/kb/832017