She's poly and has a husband, as well as A. It's not cool for her to control his relationship with you. If she isn't comfortable being where you and A are, that's fine, she can stay home.
If I were you, I would be gentle, but firm, and explain that if he needs permission from her to date you...