Wow, this almost sounds like my ex marriage, though we were not poly (I did not even know poly was a thing yet).
It was a contributor to the end of the marriage and I was totally monogamous with him.
In his own case, he had both a private porn issue coupled with an oddly prudish attitude about actual sex (he wanted to have a lot of sex, just not with another human being). He would not have even discussed sex in a therapist's office and when we got marital therapy, he hoped that it would "fix" me and that would save the marriage. The therapy actually sped up the divorce...
So, it is a very good thing that you two are actually talking about this in therapy, it really seems like a start...