There is no law prohibiting you from wearing part of a uniform. The law prohibits you from wearing items which specifically identify you as military when you are not entitled to them, and generally this WILL be overlooked unless there is apparent intent to deceive. This would be an additional charge piled on following an arrest for, say, fraud.