I had that until I found the right blade/prep combination for my scalp.
To stop it I would use a variation on the little piece of TP ... pretty much a folded up square of TP soaked in Listerine. Stops the bleeding every time, and doesn't evaporate quickly like rubbing alcohol.