The thing I might recommend is to find a competent couples' counselor and make an appointment to go in and talk this out in front of someone who has nothing invested in the problem.