Basically to carry a handgun openly or concealed in the state of Georgia (outside of your home, your car, or your business), you must have a Georgia Firearm License issued under code 16-11-129 and the concealed handgun must be carried in any type of holster or any other securing device. There are exceptions, read below to find out what those are.
(Quoted from http://georgiapacking.org/law.php)
On the matter of long arms, the law seems to be silent and the maxim of 'that which is not prohibited, is allowed' applies. You may be stopped, but you can not be cited for any illegality on the basis of carry/possession of a long arm.
IANAL and all other internet warnings apply.