Wow, this all sounds so hard. FWIW, I think you're doing everything right, as right as husband can be anyway, you're still human.
To explain the source of my perspective, and a story, in case it helps:
I'm my husband's second wife. Once upon a time, I, and many others, looked up to his first...