The visible carrying of the arms to defend oneself and other innocents serves the same purpose as uniformed police officers openly carrying arms; it serves to preserve the peace and deter aggression.
If concealed carry deterred crime because "a criminal wouldn't know who was armed and who wasn't and therefore wouldn't take the chance that his victim might have the means to resist.." then the police should only be in plainclothes so that again, a criminal wouldn't have a clue that his intended victim was an officer instead of just an innocent sheep to be fleeced.