Sounds to me like they were checking out the neighborhood, maybe trying to get to see the inside of some houses so they could see if people had anything worth stealing. I would be VERY concerned about someone who didn't back off when I asked them to while OC.
I would definitely talk with a few neighbors and let them know about it. I would also let the local LEA know about it, depending on what kind of relationship you have with them. The police might know if they have been in other neighborhoods and if there were any complaints or criminal activity associated with their presence.
I think you handled it well. At least they know that one person in that neighborhood is armed!