Update 2: I've changed the 5 source servers to 3 other's that are on the same continent as our NTP Server so the offset and jitter are much better. This helped this server to be more "stable". The server now uses clock.isc.org,