How close are you to his primary partner? Are you a frequent visitor to their place? Her friend? Is she supportive and encouraging of your relationship with her primary? Does she currently make time and space for you guys to get together? Does she have other relationships of her own even? I ask this because one thing is for sure: he is going to have a lot less free time in the months ahead! Now, some poly groupings do really well by keeping relationships very separate and distinct. No socialising together, just one on one time with each dyad. I feel like these groupings tend to be the ones where free time is plentiful for the wo/man in the middle. When time as a resource is more limited, it helps if all the partners are happy to share time together.
I think your worries are completely natural and understandable. You know yourself how much this is going to shake up his world, even if he can't fully comprehend it himself!

You need to start talking to him about your fears now, but don't panic - you have 9+ months to sort all this stuff out. You don't need to make a snap decision now (and indeed, you probably shouldn't). I'd start by asking what he and his other partner have already discussed between the two of them about this period in their life and how it relates to their polyamory. What are her expectations of his level of support during her pregnancy for instance? If she's anticipating needing him around more, is he prepared to drop other commitments he has (time for his hobbies, time hanging out with friends) to make that extra time, or is it going to come from you and him time? If he says he hasn't thought that far ahead, gently encourage him to start thinking about these things NOW and talking with his other partner NOW so that he knows where she stands in this.
Treat it simply as the fact-finding mission that it is - don't jump to conclusions that this will automatically mean you get less of him. I mean, you might need to be more flexible about when and where you see him, for sure, babies are unpredictable and he is going to be inexperienced at managing his responsibilities at first. He's also going to be swept up in NRE for his kid, and you know yourself how powerful and wonderful those emotions can be! However, many poly groupings manage to have kids and maintain their loving relationships - it just takes extra planning and the support of everyone involved.
Be brave and just start asking the hard questions now. When me and Nina start trying to have kids, we have decided that we probably will need to take a step back from some of our relationships - certainly the more casual ones, and we definitely won't be adding new partners until after sprog is a bit older. But if one (or both) of us were in a more serious relationship when the time came around, we'd be trying to make space to accommodate that because we don't think it's fair to just drop those. We'd be approaching and talking with those people before we even started trying to conceive (as your partner has done) to ensure that we took their thoughts and feelings (and fears) into account. The only way I see it working for us personally is if the other person is happy to become a little more integrated into 'family-life' in some sense. I would hope that Nina will always pick partners I can relate to and like - basically, people that I wouldn't mind coming over for dinner and who were happy to have some of their date times be fitted into the routine of family life. I don't want to be stuck at home every other night putting the kids to bed on my own while she is out wining and dining at expensive restaurants, and neither does she. We want to spend time with these (hypothetical) kids together! So if that means that her date comes by for family dinner, hangs out with me while she gives kiddo a bath and reads the bedtime stories, and then they go out to a movie together later, then that seems like a workable solution. It's not all on him to make time and space for your relationship - you can start thinking now about what changes you would be willing to make in your interactions with him to help make this work for everyone too.