My opinion? You would look silly. :P When I think of an ankle holster and shorts my mind comes up with sock suspenders!
I've worn my belt mounted holster with shorts and it is an okay way to carry, especially with a snubby. You look like a bike cop or park ranger.