Audio problem: Just before Aunt May does her hero speech she tells Peter that he 'Did a very brave thing, telling me, and I'm proud'. There is a pause between each clause of the sentence and the 'Telling me' line is obviously dubbed, the shot is from behind but the sound of the line is different.
Audio problem: When Aunt May gives Peter his birthday money, before she says, "Can you believe that it's two years next month since he was taken?", we see her from the side, and the motion of her jaw shows that her lips are moving before she speaks. When she says, "Were I to face the one responsible.", there's another shot of her from the side, and as she continues, ".for what happened", her jaw doesn't move at all. In the same shot, when she says, "Oh, I don't know what I'd do", once again the motion of her jaw show that her lips are moving before she speaks. (00:13:20)
Answer: While it's never mentioned in the film, the most likely explanation is that Harry, as a rich kid, grew tired of "slumming it" and decided to move back into his mansion.
Harry and Peter's relationship quickly turned sour after Peter refused to discuss Spider-Man as Harry felt Peter knew more than he let on. Peter is Spider-Man's "personal photographer," and Harry only wants to avenge the death of his father. It's not unlikely to assume Harry left to live alone because he was angry Peter wouldn't help him find Spider-Man.