THIS. If you even acknowledge him, you are complying. You have ZERO reason to comply, or even answer him. You don't owe him anything. Just some idiot in walmart saying, "hey wait" should be ignored. The entire interaction with him was VOLUNTARY on your part. You volunteered to stop and listen to his questions. Then you got to the point of asking him, "am I being detained"? Once he answered, "no", you should have said, "see ya later" and started walking. It might sound rude, but it's more rude to be illegally searched by some thug. Next time, don't even respond. Or if you have to, try this.
Plain clothes nobody with badge: "Hey wait a minute".
You: "No I don't want to, see you later". (walk away) If he's whipped out and showing his badge, you could say, "thanks for the offer to contract, but no thanks, see ya fella".
Anything that has to do with leaving without complying is best. If you interact at all to begin with, remember it is VOLUNTARY, until they decide to "detain" (arrest) you. NOTHING says you have to talk to them.