It sounds like you're just in the experimental stage. Everyone develops their own shaving routine, and that's part of the fun of wearing a baldie style.
I'm a daily headshaver, but over the last 8 years, I've tried many different routines. I have grey hair too, and I don't like any visible stubble. It sounds like you're using an electric shaver, so it won't work well after about 2 days growth.
Once you get used to wearing a bald head, I would suggest trying a razor wet shave. There is nothing like the feel of a freshly shaved smooth head. The guys here suggest keeping your head shaved for 30 days to get used to it. Again, welcome to the bald Brotherhood.