I have had it happen one time too many, and few times they felt it necessary to remove the gun from the holster. Good way to get shot, most of the time I can read a persons intent. But an itchy off duty LEO might not be so forgiving, or even a on duty that witnesses the exposure of a concealed firearm to another.
Not sure why this is not being covered in CC classes. Well assuming it is not being taught since CCers are doing it.