My reference voltage changes when I connect it to the amplifier's input?

The issue may due to the inability of the voltage references to sink current. Please see Tutorial MT-087, Voltage References.