I think starting with "I have a license for it" is the wrong move - it acknowledges the weapon and your nervousness in the situation. Best to start with "is there a problem" followed by (when he says "I saw your pistol, which is concealed") "am I being detained." If the officer says "no" say "have a nice day," turn around, and keep walking.
Most likely, the response will be "yeah, I saw your weapon, which is concealed, I need to <blah>." Depending on what is requested, you have a number of options. In the case of the officer who cuffed you and removed your weapon - I would have stated very clearly "I do not consent to this seizure of my sidearm nor any unlawful search based on this seizure." Do not stop the officer if (s)he proceeds to try to take your weapon, but as they are seizing I would, again, state "though I am cooperating with your request, my cooperation does not and should not be viewed as my consent. I feel coerced in this situation to have my sidearm unlawfully seized, and my cooperation should not be viewed as implied consent."
Ideally, it wouldn't come to that - a reasonable officer would respond "I saw your weapon, and would like to see your concealed pistols license as well as your identification." At this point, you inform the officer where your CPL is, and ask if they would feel more comfortable retrieving it, or if they would prefer you reach for it (and with which hand).
If, upon commencement of contact, the officer says they wish to remove it for their safety, I would still say "I do not consent to such seizure." In addition, I would probably say "the safest place for my sidearm is holstered, with the trigger covered." The key thing is that you explicitly state I DO NOT CONSENT. Moreover, stating that "my cooperation should not be viewed as implied consent" is another good phrase to remember - you should cooperate, but still firmly state your rights, especially if you are in front of their vehicle and thus within range of their cameras. Immediately after being let go, you call a lawyer and subpoena the tapes, because they *will* get erased if you don't.
In short, your rights were violated because you didn't know what they were. Hopefully, moving forward, you can better respond to such a situation.