Non-Monogamy is for Mature People
I don't mean to sound harsh, but it seems to me that your husband is a very childish man.
You say you are both full-on open poly, but it doesn't seem that way to me. Polyamory is not for the immature. It is for those who are mature enough to handle it. In a "traditional" relationship, a person may fear that an ex can/could be a threat, but in a poly relationship, you would think that inclusion of the ex would/should be possible.
Poly is about many LOVES. Jealousy is a fear of loss. And many men fear other men. There are a lot of men, and possibly women, who fear that they are not as good as someone's ex. And many people can handle the physical aspect of non-monogamy, but fewer people are prepared for, or can deal with, the emotional aspects.
One thing I find that I am constantly having to remind people is that once you open the flood gates you can't close them, no matter how hard you try. You know as well as I do that you will always love your ex. And he will always love you. The question I have is how much your husband truly loves you. The way your husband has acted has hurt you, your ex and your current relationship with your current husband. His actions have not only been immature, but they have been hurtful.
I would question the love your current husband has for you as well as the health of your relationship with your current husband.
I am just speaking from my own perception, but it doesn't seem as the two of you have a polyamorous relationship at all at this point, possibly never did.