Lori
You should not feel badly IF the decision to open up the marriage was truly made by both of you. the fact is that very few men will care about forming a relationship that becomes sexual with a married or partnered woman, but fewer women will want to do that. Men usually do have a harder time finding a partner.
That is why it is important that you two have discussed this and he understands totally what he agreed to. otherwise he will probably start to get resentful and you will have problems, especially if you were the initiator of this and he reluctantly went along, which happens a lot.
I would keep talking to him and make sure he is becoming Ok with your situation and not just tolerating it.