It may not be worth arguing with the cop at the time... but it's always an advantage to know your rights and know the law. Especially when dealing with a sworn "law enforcement officer".
Code of Alabama, Section 15-5-30:
A sheriff or other officer acting as sheriff, his deputy or any constable, acting within their respective counties, any marshal, deputy marshal or policeman of any incorporated city or town within the limits of the county or any highway patrolman or state trooper may stop any person abroad in a public place whom he reasonably suspects is committing, has committed or is about to commit a felony or other public offense and may demand of him his name, address and an explanation of his actions.
When a sheriff or other officer acting as sheriff, his deputy or any constable, acting within their respective counties, any marshal, deputy marshal or policeman of any incorporated city or town within the limits of the county or any highway patrolman or state trooper has stopped a person for questioning pursuant to this article and reasonably suspects that he is in danger of life or limb, he may search such person for a dangerous weapon. If such officer finds such a weapon or any other thing, the possession of which may constitute a crime, he may take and keep it until the completion of the questioning, at which time he shall either return it, if lawfully possessed, or arrest such person.
In other words, the cop cannot even force you to stay and talk with him unless he "reasonably suspects" you're committing a crime. And if you do stop, he has no authority to check for a weapon unless he "reasonably suspects" he is in danger of life or limb. And if he finds a weapon, he has no authority to take it unless your possession of it constitutes a crime! (which, while carrying concealed, a simple look at your pistol permit would clear up)
Anyways, I had a cop do the same to me, and it made me pretty uncomfortable... I have no idea of this person's ability to handle my weapon safely. I think we would all be much safer if this weapon that is unfamiliar to him just stayed safely tucked away in its holster. So I went home and did some research and learned that he has no right to take it in the first place. Food for thought...