Hannay is talking to the milkman inside the door. Outside the door you can see across the street, the two guys walking down the sidewalk. A moment later, Hannay goes toward the door and now there's the milk wagon on the street right in front of the door. When Hannay pointed out the two guys across the street to the milkman, the milk wagon wasn't there and the milkman is inside with Hannay so he couldn't have pulled it up.
Director Alfred Hitchcock has a cameo role in this film - he is seen tossing some litter as Richard and Annabella run from the music hall.