Shot in the dark here... If you wife told you she sees that you're attracted to the new guy, is it possible she told him that too? Maybe he's uncomfortable being in a threesome with you because he's not bi and afraid of what you might do.
As far as the jealousy is concerned, has your...