Why won’t the man attend the party?

W: Hello, John. It’s good to see you. When did you get home?
M: I arrived at the airport this afternoon. My wife picked me up.
W: There’s a party at the Hale’s tomorrow night. Would you and Nancy like to come along with us?
M: Thank you just the same. Nancy and I are planning a quiet weekend with the children. I have to be back in Washington in a few days.
W: Of course. I understand.