While the fantasy sounds amazing, conceiving a new human being shouldn't be dictated by lust or by the horniness of a moment.
However, not all couples can conceive and sometimes need help. If that's the case and you're comfortable with knowing the donor, go for it (protect each other legally...