I would say it is infrequent, at least in the Freedom State of Virginia. Most people seem to not even notice. I have walked right by police officers in busy downtown areas (Old Town, Alexandria) and no reaction.
I have been harassed a few times (a mall, a movie theater, a DMV Office, but most of the time, like I said, no reaction.
If you can open carry legally and you feel comfortable about the situation (not overly crowded, you have good equipment that will retain the weapon if someone was stupid enough to attempt to do a gun grab) I say go for it.