Just so you don't feel so judged and criticized and ganged up on, I will add briefly my story.
I too have always been poly. I didn't have a word for it other than.... not sin, since I am not Christian. Insead, evil or bad or nasty slut were things I told myself I was. And my ex h told me I was too.
I married him young because that is what one does. And he was better than all the other guys I had dated. I am also bisexual, but settled into a mono hetero marriage.
2 decades passed, 3 kids were born and into their preteens/early teens, and my h, who was always mistrustful of my roaming eye, and love of women, suddenly had the idea, Oh Mags is bi! We could have a sexual 3some! I agreed rather reluctantly, since I was fine with just his acceptance of my nature. But I went for it.
We found our "girl," she seemed into the idea. But lo and behold, she wasn't into me, really, just him. They fell deeply in love.
I was devastated. His NRE and neglect of me and our 3 kids was pretty unbearable. We didn't go on romantic dates anymore, and he was driving hours away to wine and dine the new gf, leaving me home with 3 kids and all our pets for long weekends, no help for me, no more home maintenance on his weekends.
So, I vetoed. And I now know that was wrong of me. I was trying to close the barn doors, but it was too late. I "allowed" them to be "friends," but they were lovers, despite having stopped having sex. They still kept in frequent electronic touch and shared a bed when mutual friends had parties, weddings, etc. I stayed home from those since I knew she'd be there.
So, it took 10 years, lots of counseling, hopeful stretches of romantic dates and frequent sex, but I finally realized he was not providing for my needs. He was deeply resentful of my veto, stopped saying I love you, stopped calling me honey, stopped PDAs, stopped kissing me. We had great sex, but no kissing!
He is passive aggressive. Finally I called it quits.
So! You are doing a great job looking at your very difficult situation and weighing all your options. No options will be pain free. You just need to find the least sucky choice and go with it. Short term pain for a certain period of time is better than decades of mistrust, resentment, fakery, and putting up a false front.
There is no need to make a snap decision. I think the most humane thing, even tho far from perfect, is to tamp down your romantic yearnings, give in to the needs of your baby, and focus on that for now. Sleep deprivation lowers the sex drive anyway! But babies are a joy and you will have a new precious loved one. Girlfriends like Penny will fade into the distance for a while.
But divorce and shared custody may well be in your future.

I hear you on the 85% good, 15% bad idea. In my case it was 60% good, 40% bad. But I am so glad we parted ways. Now my life is pretty much 100% good, with a poly partner (we live together) and our other partners we see as well. My ex h and I parted amicably and continue to co-parent more or less successfully, as a team. And my kids now have his gf (they moved in together 6 months after our separation) and my gf as other kind adults in their lives.