It sounds like you handled that situation very well. To be honest I probably would have been a little freaked out, especially after that @#$% started cursing at you.
Based on his actions, it sounds as though he was harassing you( the ignorant/rude citizen).
He may have inconvenienced you a little bit, but you can sleep softly knowing that he is probably throwing a @#$%-fit about it as we speak and it has ruined his entire day. :P