I have no experience in these things, but could it be that the sneakiness about it felt like cheating to you, even though it wasn't, and it stirred up emotions in you? I mean, it's not really a boundary that y'all have that he needs to tell you every time he is going to do something with her...