My initial thought was, you don't have a mono/poly relationship. You have an active poly/lazy poly relationship :) In other words, you date girls....and he dates the girls you date :)
If you do end up dating the same women, which is cool, I think it'll be pretty tricky (and asking a LOT of...