Jealousy could be an issue for some people. It was for me when I was younger. I think it may depend on how stable your relationship is and how comfortable both of you are with it.
My wife had a few lovers before we met and I always fantasied about her having sex with other men.
The situation...