Did he honestly ask you if you knew you had a gun on your hip??
I guess LEOs have nervous conversational breakdowns like the rest of us.
Believe it or not, I have had that exact same line given to me, but not from an LEO, it was a co-worker in Alaska.
"Yep, and you don't, so whats up with that?"
I think his brain locked up on him, because his jaw worked for a few moments but no words came out. I just smiled big at him and chucked him on the shoulder and walked on.