I am squarely with Hawkyns on the subject of clothing. IMHO it comes down to whether you have clothing or a costume.
That said, I recognize that for most their "garb" is costume, and they are just trying to make their costume better.
So you have a day job? What do you wear when you are doing yard work? Jeans and a t-shirt? Get over it!
Put on your slops and the rest o' yer kit! Don't worry about the neighbors - most likely you're better armed than they are anyway
Wear your kit while working on your car, while barbequeing, etc.
There, that will get it all nice and grungy like. So, how do you go about keeping it clean?
Rule #1: no modern washing machines or detergents. Hand wash in good old fashioned pure soap. Fels-Naptha can be found almost everywhere still, and there's fancier stuff around too. What you absolutely want to avoid are the UV whiteners prevalent in most detergent. Hand washing and line drying will also help your kit age appropriately.
Rule #2: see Rule #1.
Just my two-cents worth, and spoken as an overly clean pyrate.