Depending on how coarse your hair, and how many passes, I also think it is possible to dull a Feather enough with a head shave to affect the face shave. Next time try it in reverse. Start with a fresh Feather and do face first then head and see if you get different results.
I find that it's hard to maintain proper angle on the back of my head, which likely leads to dulling a blade faster, since for me it means I'm using the blade much more perpendicular on the back of my head than I am everywhere else.
I just got my first SE in a PIF, and I'm anxious to see if it helps my angle issue on the back of my head. I have a theory that since the SEs are much more sensitive to angles, it may be easier to tell whether or not you are maintaining that than it is with DE. That might be why Warhawk seems to be so fond of his Mongoose and Cobra...