In the three potential attacks I've experienced, the presence of my OC gun - and once my grip on that gun after lifting the retention strap - caused the would be bad guys to run like hell. I didn't have to tell them I had a gun... they could see it. I only actually had to shoot once... http://www.thepriceofliberty.org/?page_id=846 That attacker ignored the gun in my hand, pointed straight at him. So you just never know.
The potential for a continued attack is greater, obviously, if they cannot see it... but in that case there should be no problem with clearly warning them that you DO. And then you just have to hope they take the hint.