In my experience there is a small but noticeable difference.  As with every women each has their own distinct taste and like you I to have found the same to be true with each man.  So the main difference is brought by the new partner.  His "flavor" mixed with my wife's makes each creampie a little different.  By no means am I an expert on he subject and have only enjoyed my wife's pussy after another man has filled her up about nine times.  Thee has been a couple of guys that I sucked off and swallowed their cum as well as eating their cum from the wife's pussy so I agree with 2wheel is their taste is diluted by the wife's own juices. 
I would add that I also find the act of sucking cock and feeling another man cum in my mouth is better then the taste.  I get off giving him pleasure, being rewarded with his load, and how empowered it makes me feel to give another man an orgasm.