You're a well dressed guy (from your avatar), surely they'd see the suit and think twice? I've been worrying about being taken as a skinhead considering I'm visibly tattooed, but I'm really sorry to hear you're in such a tough spot. I hope you find a solution that doesn't compromise who you want to be.
I don't see it as a practical problem, because no one will touch you.
Nazis will think you adimire them, and the others will not buy the thing sly guy/nazis or, if they will, they will be also be intimidated and will stay away from you.
Maybe tre problem could be psychological for you, because you don't want to be considered a nazi, but if someone judge the book by the cover... you know... better loosing than keeping such a person.
The term skinhead has different meanings in different places. I was one meself still got the boots and braces and wear a MA1 jacket . I am mostly certainly not a nazi or a hammer skin . I think in the Us and parts of central Europe it has these connottations of the extreme right and ive been a bit careful how I dress when I Visit Germany . There are plenty of sly guys in Berlin its passe ,
I've been sly 15-20 yrs, very light skinned (white) and very heavily tattooed (sleeved up & neck) have lived and worked in what's considered a "tough" neighborhood (South Bronx NY) and have never been called a "skinhead" or any other thing due to my "sly'ness"... And I'm by no means a big guy... In fact, I see tons of guys doing the "sly" thing more and more just as a hair (or lack of) style.
Unless there's a "skinhead" population in that exact area, then maybe they may mistake you as one. They normally hang out with their own type and don't socialize with other types of groups, so just by your normal actions people would (should) know you're not associated with them.. I think people can tell.
Just an opinion..