How do I optimize my PLL loop for the best phase noise and/or jitter?

There are many tools one can use to optimize a PLL loop. ADIsimCLK is a good tool to use for the ADI clock parts. Optimizing phase noise and jitter is not necessarily the same thing.

If one has a spec for phase noise at a given offset frequency, then one should supply the VCO and Reference phase noise information to a tool, such as ADIsimCLK, and use this to optimize the closed-loop bandwidth to achieve the desired targets. This process is in essence adjusting the closed-loop bandwidth to trade off the reference and VCO phase noise.

If one has a jitter spec, then the closed loop bandwidth should be adjusted to achieve the lowest jitter, which may not necessarily correspond to what appears to be the lowest phase noise for all offset frequencies.

For example, while it may be possible to achieve low close-in phase noise by extending the closed-loop bandwidth, the resulting jitter may be larger than the minimum possible because the loop is tracking the reference more than is necessary for optimal jitter. One might achieve lower jitter by reducing the closed-loop bandwidth, allowing the PLL to track the VCO at lower offset frequencies, even though the resulting phase noise plot might show more peaking at the closed-loop bandwidth offset frequency.