He gets to do anything he wants, with anyone he wants, and you have no input and can't do the same.
He now picks what sounds like a partner that has inherent risks for you (STDs and emotional).
It sounds like he a lot less poly and much more just a cheater, who got you to go along, possibly...