It won't be easy, but you need to at least tell him.
The advice I would give you, is that you should talk to him about how you're feeling. That doesn't mean you should expect him to not see other people. But talking about it can help a lot, and as somebody who was cut out of someone's life...