For me, I usually do.
Unless if I just loaded a magazine or two then I dont worry about it.
But after handling ammo at the range and shooting, or loading up a lot of magazines I surely do.
It may not seem like a lot of lead one or two times a week (or whatever your shooting schedule may be), but think about how many times in a year, then how many years over a lifetime of 100% pure lead being absorbed in the skin.
If the FDA takes lead out of paint, and its not that often we touch paint) then why not be safe after handling ammo.
Just my 2 cents.