The only other considerations are:
(1) private property - The Tea Party as the venue has said hand guns legally carried are OK
(2) Restricted facilities i.e. court rooms, jails etc as enumerated in § 18.2-308
(3) There is a restriction on the number of rounds (20) that you may have in your magazine in certain cities/counties if you do not have a CHP, but that is not likely to cause a problem for handguns. Check this out as it was likely why you heard what you did.
Bottom line - yes you can CC or OC throughout the state including the Convention Center which is owned by the City of Richmond. The venue as a private player may restrict if they so decide - The Tea Party is NOT restricting - so carry on.