I suppose I can understand where you are coming from provided it gives you pleasure to see your wife/partner taking some decent sized cock but otherwise I'd think it more of a mind fuck for you. I also think a small cocked guy might try that much harder to be a good provider for her and to love...
I meet with a couple who are in their mid 70's and they won't be stopping anytime soon. I'm 70 and the three of us are respectful of each other and just enjoy each other's company. We've been meeting for 8+ years now and enjoy it immensely.