The thing with new relationships is that one tends to want to covet them and keep them precious, because we feel all swoony and hyper-sexed over them. That is awesome when you are mono, and don't have other responsibilities to others, but when you are undertaking a polyamorous relationship it can lead to disaster. I am wondering if this is how he is feeling.
My husband, PN, does this with new relationships. His "on/off" button is not the best, but he is getting it slowly. Currently in a new relationship, he is finding it hard to be present with me and connected. When he gets like that, I begin to feel disconnected and suspicious of his intentions. Luckily for me, he would never let his emotions and NRE allow him to be bowled over, until he just throws up his arms with a "What the hell!" and have sex with someone who he is having strong feelings like that with.
I have noticed that he gets irritable and accusatory, like I am dampening his good time just by my very existence. That is part of it. I get in the way of his wallowing in his lovey-dovey feelings. He really he wishes I would just bugger off and let him covet and cling to his feelings, mumbling "My precious" under his breath. I wonder if this is what your man is doing. Your asking to be involved is wrecking his zen, not that you shouldn't be. You are his wake-up call! I definitely support you on this one.
I have found that letting your partner do what they need to do is the best bet. That way they can feel what they feel, and then come out the other side and make decisions. The more I try and get PN's attention, the more it distracts him from making decisions. Of course I trust him entirely. I have experienced him in these situations a few times and so know enough to rely on that trust.
The clincher here is your h's mistress's infidelity. She isn't a woman of integrity, it seems. To me that is a huge deal. When PN was in love with a woman that I struggled with, I eventually threatened him with a veto. Maybe not the best idea, but it got the message across. I told him that he was too involved, and it was taking away from his family that he was so distracted by this woman that couldn't care less about his feelings, who was going to continue to sleep around and hurt him. In my opinion, he needed to tell her how he felt about her, and if that didn't get a response back of her loving him, that he should give her up. He had over-invested. Poor dear, he does tend to over-invest in those that are not as invested. He gave her up, because he also trusts my judgment and relies on it when he is negotiating a relationship.
Her response to his undying love for her was, "Oh, too bad I don't get to fuck you anymore." Pretty clear what her investment was, and her intentions. She laughed at him and looked confused when he told her he loved her.
I find it interesting that this woman is not interested in knowing you after you phone sex, not once, but twice, and that she declined the third time. I think she sounds like she is used to getting her needs met for a good time, not a long time, and likes the thrill of sexual relations, but is not interested or doesn't think that connection and depth are possible.
I would wonder if she is drawing him in to satisfy her need to fuck around, and then when the thrill is worn off, will go and look for someone else. Some people derive their feelings of self-worth from that. I find that very sad and lonely, as I have been in that place, and realize now it's selfish and false. It's all temporary. When I got to a place of wanting more depth I was abandoned by my confused pursuers. "What? I just wanted to masturbate in your body and then move on!" Unfortunately, she will find a world of men that would be quite happy to feed her false self worth.
The thing is, your husband is better than them. He is now walking amongst their ranks, and is willing to do so in the hopes of being loved by this woman. It sounds like she has turned on her charm in order to lure him in, and is giving him all the right cues to do so. She might even think she does love him and all the other flash in the pans she has in her life. But I wonder if she does. That's just me, though. When love is pure and full of honesty, then, to me, it is real. Otherwise it's a wolf dressed up in sheep's clothing, and a nightmare waiting to happen.
Not much, but I guess it's a start.