It's easy to get the impression that bad encounters happen often because those are the ones you're most likely to see posted in a forum. I think that most of the time it's a "nothing happened" day for the OC folks. Which is good.
I can assure you that people notice when you OC; a gun is an attention-getter. Most people probably assume that you're LE (plain clothes or off duty) and let it go.