Marshaul probably explained it better than I could, but the point is that you have to separate what is a "RIGHT" from what are consequences of exercise of that right. So for your question the issues are-
Our RIGHT to go peaceably about our business while openly armed, verses someone else's comfort at our exercise of that right. The last time I looked there was nothing anywhere guaranteeing a person the right to be comfortable with the exercise of someones else rights, so long as their rights are not violated.
So their next argument would usually be that they have a right to move through society without fear and that I am inducing fear. That is a bogus argument. First off they do not have a right to a "fear free" life, but even if they did, the fear is self induced, it is not caused by any threatening act on my part directed toward them.
So that leaves courtesy. DOes it show a lack of courtesy on my part to be openly armed in the presence of someone who is not comfortable with firearms? I say no it is not. They have decided to be someplace where firearms are acceptable and leagl to carry. The easiest example is smoking, and while it is not a perfect allegory, it will do for our discussion.
There are non smoking areas, just as there are areas where people may not go armed. There are smoking areas just as there are places were firearms are legal to carry. AS a smoker I would have to abide by those restrictions and so should others. But it is two way street. As a non-smoker you cannot come into the smoking area and tell everyone they must stop smoking because a non-smoker has entered the space.
If my firearm bothers you that is unfortunate but then you should stick to those parts of the world that are "gun Free" zones, just as I avoid such areas.