Your XD is ESP, a tougher category than SSP.
Your Glock 35 actually has a 5.3" barrel. (the 17L is 6" but not IDPA legal)
If you like 40 cal, by all means stick with it. If you handload cost won't be too bad. If you buy all your rounds factory it will be pricey.
Benefit of the G35 is that you can download your reloads, and make IDPA power factor and USPSA Minor, or use standard power loads and make USPSA Major power. This makes it very versatile. G35 also has lower bore axis.
Downsides of the G35, XD has much better trigger(my opinion) and smaller grip-circumference. XD is also heavier in stock form (generally a benefit in competition). You can add weight to the G35, but it would then be ESP.
Your mileage may vary.