Nick, I think you handled it well. If it was a one time thing, and if it doesn't bother you, I would let it go, but I would limit contact with this guy. If something like this happens again, by the time I'm finished, this guy would be a former employee (if that were me in your shoes)