Some more background: I was at a good friend's house, only maybe 10 people there, total, we'd all grilled steaks earlier and were watching the game, no heavy drinking (but in case apparently heavy enough to do something stupid. Guy who pulled the knife is apparently a good friend of the fella who's place we were at; I just didn't know him. Further, I thought he was kidding, and still think he was half kidding. He wasn't fuming mad (which is why I was confused), I think he was just "Billy Bad A$$" and decided he needed to look good for either his friends or some ladies...The comment he made later was again made in a smart tone, not a threatening tone (again, just think he was asserting his badassedness by saying I wasn't the only one with a gun, he made no move to go get it and he wasn't leaving), but either way I said "time to leave." I'm not going to involve the police.
As to brandishing, I understand the point being made that in VA even a justified illegal action can be charged and then you have to, well, justify it :? Don't think that'll ever come into play here.
As was said in my OP, I would have deleted it if I could...I typed it because I was anxious for some reassurance but then realized it didn't really matter, what's done is done, but thanks for the feedback.