Continuity mistake: Christine walks up to the bench, Harry is holding the invitation. The shot changes and he is no longer holding it. The shot changes and he is holding it again.
Continuity mistake: When Harry and Christine are talking, Christine is building a house of cards. The top card keeps changing between a four and a five.
Answer: It's possible when Harry called the military, he implied that the book contained sensitive information, seeing as how many government and military officials were part of the client list.
I thought it was because he developed feelings for the madam and couldn't bring himself to turn her in.