The reference input is differential, but my reference is single-ended. Do I need to convert to differential to drive the AD9510?

Strictly speaking, it is not required that a conversion be used, but it is recommended, especially at frequencies >50 MHz. Using a transformer to convert your single-ended reference to differential will give better noise suppression on the reference. However, if cost and board space are your critical concerns, you could use a .01uF capacitor to couple the unused input to a clean analog supply or ground plane, such as AVDD or AGND.