Thanks for providing more details. I guess when you dated and married him, you were in love and thought the lack of sex wouldn't bother you. But now you've finally admitted to yourself and your husband that it really does seem like a missing piece.
I honestly don't fully understand how any partner who isn't interested in sex (with anyone or) with a partner who has a more average in interest in sex could in good conscience object to that person seeking sex elsewhere. It just doesn't seem like a loving act. People with average or higher than average libidos feel a lack of sex as an actual physical pain, not to mention there is an emotional component that sex provides.
I have a high libido now, but when I was knee deep in raising 3 small children, my drive was at an all-time low. I felt bad for my husband as far as sex went. But on the other hand, he went out to a desk job all day, and wasn't breastfeeding. I mean, I thought he had it easier than me, since I was a SAHM. He got to be around adults all day and have lunches out, and normal conversations, while I was just fully touched out by bedtime; I'd been on high alert all day, go go go, and needed sleep and nothing else. When he admitted years later that he'd secretly stopped at a local club for a lap dance now and then in that period, I wasn't even mad. It was one less thing I needed to "take care of." Anyway...
It's easy to say to the partner with the sex drive, "Well, honey, just masturbate," but sex with a partner is just so different than going solo. It's fine to masturbate to
augment regular sex with a partner, nothing wrong with a little self love, but if you have a partner, whom you do desire, and they are out of reach that way, it can feel so lonely and isolating, and make you go a little crazy.
That said, I agree that getting to the point of him consenting for you to seek intimacy elsewhere is going to take time, effort, reassurance and patience on both sides, if you want to stay together. Besides the two books recommended, we have a whole slew of more reading resources, and a podcast, here: