Check the XD talk website. The sights are notoriously difficult to remove using a punch and you need to use a special tool or have a gunsmith with some experience. It's not really a matter of which way you try to punch it out which is the first thought.
Many people have to send the slide back to SA to get the sights put on. I've had success using Glow-On a luminous glow in the dark paint and stick on pad (check their website).
It's actually brighter than the tritium though you have to 'charge it up' with 10 seconds from a tactical flashlight or something.