I've been bald for 30+ years, so my head is pretty much hardened to the cold. But with shaving it all off, the back of my head is still sensitive to the cold on the worst winter days.
I hate wearing hats and only wear them for the coldest days. This past summer I kept 45 SPF sunblock with me all the time and hardly wore a hat at all with the sun.