Whether a reply is smart-alecky or not is in the delivery, not the words. You can say the exact same words with a friendly smile or playfully and it comes across entirely differently.
My standard reply to the cop question is playfully exaggerated surprise and obviously mock indignation while saying with a smile, "Oh, my! Do you always insult strangers? I would never stoop so low as to be a cop." The playful tone and smile change the whole ballgame. Then, "No, I'm just a regular guy." This has only ever failed once.
Also, unless it is an obvious attack, keep your response lighthearted, friendly, warm. For example, I was in a bank recently, OCing. A new teller. As I arrived at the window, she asked, "Was that a gun on your belt?" I replied very friendly, "Yes." Her expression looked a little doubtful, and she said, "That's dangerous." To which I replied with a playful grin, "Only to the bad guys." This started to win her over. By the end of the transaction she was relaxed and friendly herself. A big part of that was that I kept all the conversation of the transaction playful and friendly. Basically, I know from retail experience that clerks get really bored dealing with drones for customers. Almost all of them appreciate the warm, friendly, or playful customer. If you want to win over a clerk on the spot, just ask or say something that shows you are interested in them. Few customers ever do this--they just pay for the stuff, mutter a robotic thank-you and head out the door.
If there's a lot of customers, ask, "How do you keep from going crazy with all these customers?"
If there are few or no customers, ask, "How do you keep from being bored to death when its this slow?"
Any comment, question, or compliment that isn't creepy will almost always work. Then just have a twelve second, two-comment conversation while you're being rung up. The key is to be interested in them as their own person. And, don't be surprised if that clerk gives you a cheerful hello the next time you walk in the door.
Back to the cop question. If you don't like my response, you can always tweak it to suit you. The important part is lighthearted, playful, friendly. "No, that job requires fighting skills. I'm a big softy. The very thought of being a cop makes me shiver." Put your own twist on it. And, follow it up with a question to keep the conversation going. For example, "why do you ask?"
Now, of course, the gun is the object of the discussion, so sooner or later the cop-question person or the curious is going to mention the gun. At which point, if the person is not hostile, I'll often deliberately look at their belt and ask in friendly way where their gun is. Of coure, they're not carrying one, but that's not the point. The point is to steer the conversation a little bit. And, this question just assumes that everybody is supposed to be carrying a gun. Now, you would be surprised at how many men actually do think they should be carrying a gun. You can tell by the slightly sheepish excuses you get when you ask where their gun is. The reason for asking is to establish that they should be carrying one, which also opens the door to conversation that lets you promote carry. If they don't want to OC (most don't), then promote how easy it is to get a CC permit in your state and how important it is to be able to defend yourself. Just mention the recent movie theater, Batman massacre or the headline robbery du jour.