I feel you on not seeing many other OCer's. In fact I think I've only seen 1 other person OCing in Hampton Roads, and that was a customer at the parts counter of the dealership I worked at last year.
I OC all over Hampton Roads, but I carry in a manner unlikely to draw attention to myself. I carry a black gun, on my right hip, in a black holster. I am typically wearing a black sweater tucked behind the holster, and dark jeans. While I am technically OCing, the way I dress helps to sort of hide the gun unless you are looking for it. Granted I have only been OCing for a little over a month, but I've only had 2 people say anything. 1 was the pharmacy clerk at Target while picking up a prescription. She asked my wife why she had a cop with her, my wife just replied that I was her husband and definitely not a cop. The other was at the Verizon store. After talking to a sales associate for about 10 minutes, he asked me what line of work I was in because I was open carrying. I told him I just sold parts, and just carried it for personal protection. He then proceeded to ask me about my permit, to which I informed him that I did not have a permit, and that one is not required to open carry a firearm. It was interesting to see the response on his face, as he was a little taken aback by that. He said he had a CHP and did not know that you could OC without a permit. I think there are a lot of people that just have no idea that you can even do it. Then you have people like my dad and step-father, who think that I am making myself a target by open carrying. I always reply to those people, "when was the last time you read a story of someone being targeted because they were open carrying." They usually have no answer, and I always tell them that it's because it just doesn't happen as often as they like to think it would.