If you're going from buzzed to clean-scalp shaved, there really isn't much of a difference in appearance. Also, there's a psychological factor. People probably looked at your buzzcut, thought, "Oh, he's shaving his head now," and then just incorporated that mental picture. Your new look isn't that much different. It's better to have good attention than none at all, but not being noticed is better if noticing is going to produce a negative reaction.
If you are like a lot of us, including me, you will feel a lot happier scraping the dome smooth. It just feels right to complete the job you started, and the feel of a smooth scalp is great!
