A meeting with his brothers wife led to sexual infidelity