A couple of things to consider....
As Grapeshot already pointed out, can you rack the slide with one hand? What if the bad guy has one of your hands/arms incapacitated? Or, what if you just want to use your weak arm to push the bad guy away from you, or push your family/other person away to safety?
What if your gun jams on loading? Have you practiced loading your gun under the pressure/adreniline of a life or death situation? Personally, I would rather deal with a jam in the relative safety of my home where I can load the gun calmly, take extra steps to ensure the round chambered correctly, and if the gun jams to take the time to correct the jam and determine why it occurred.
So my question is....do the disadvantages of open carry that you perceive to exist, of which there are no examples of happening in real life, outweigh the disadvantages of carrying a gun that not only will the motion of drawing from concealment place the bad guy on high alert, but you will also have to load before you can shoot, and hope for no equipment failures.