I've only had three issues in eight years where people freaked out over my open carry (Butte, MT).
First was a bank (on Dewey, near the post office). Had open carried plenty of times over the course of two years there. Ran $300,000 through their bank all at once and traded large US Treasuries. One day, $55,000 check in hand, I'm stopped at the door by their security and told very rudely that I wasn't coming in carrying "that." He was so forceful, he was spitting saliva. I kept a smile on my face, as I always do, but the security guy and the bank manager were livid - bodily blocking the doorway and leaning threateningly. I closed my account through the drive-in window.
Second was another bank, Wells Fargo. Had been in multiple times, always polite. Approached by someone not the manager and told I couldn't carry - that the bank was a federal building. The manager who was listening started shaking her head immediately and said, "actually we're not." Good for her because claiming to be a federal employee when you aren't is a federal crime punishable by up to 3 years in prison. Still, I was asked to leave and I did. I now do all my banking there through the drive-thru, though I have been inside open-carrying once when they asked me to come in and they did not ask me to remove it.
Third was the real freak-out. At Safeway, a checker I hadn't seen before began eyeballing me and quivering. I always smile and act just like any other customer. When we were checking through, she asked, "Are you going to shoot someone?" To which I replied, still with a smile, "My greatest fear is ever having to pull it." She went on for the few minutes checking our groceries about how she hated guns, they caused death, and fear, and blah blah blah. She was making quite a scene. I told her I was licensed to carry, but she even said cops scared her. My wife was pissed, but I shrugged it off.
Overall, I believe the best response is to keep smiling. If they appear conversant, I tell them I'm licensed (CCP) and to get that license, I had to be cleaner than a cop. That usually abates all that fear. If they appear curious, then I tell them I had to clear both state and federal Department of Justice background checks and fingerprints. Usually at that point, I get smiles in return.