Mr. Ditkovich: If promises were crackers, my daughter would be fat.
May Parker: We need a hero, couragous sacrificing people, setting examples for all of us. Everybody loves a hero, people line up for 'em, cheer for them, scream their names, and years later tell how they stood in the rain for hours just to get a glimpse of the one who told them to HOLD ON a second longer. I believe theres a hero in all of us, that keeps us honest, gives us strength, makes us noble. And finally gets us to die with pride. Even though sometimes we have to be steady and give up the thing we want most, even our dreams.
Peter Parker: Pizza time!
J. Jonah Jameson: Guy named Otto Octavius winds up with eight limbs. For mechanical arms welded right onto his body. What are the odds?
Dr. Otto Octavius: If you want to get a woman to fall in love with you, feed her poetry.
Peter Parker: Poetry?
Dr. Otto Octavius: Never fails.
Peter Parker: Mr. Jameson, please, aren't there any of these shots you can use? I really need the money.
J. Jonah Jameson: Awww. Miss Brant?
Miss Brant: Yeah?
J. Jonah Jameson: Get me a violin.
Dr. Otto Octavius: I will not die a monster.
Green Goblin: Avenge me!
Peter Parker: Am I not supposed to have what I want? What I need? What am I supposed to do?
Dr. Otto Octavius: I finally got lucky in love when I met Rosie here. She was discussing T.S. Eliot, and I was discussing... I still don't know what she was talking about!
Miss Brant: Sir, your wife's on the line, she said she lost her checkbook.
J. Jonah Jameson: Thanks for the good news!
J. Jonah Jameson: I'll give you $150.00 for all of them!
Spider-Man: $300.00
J. Jonah Jameson: That's outrageous! Done.
Answer: While it's never mentioned in the film, the most likely explanation is that Harry, as a rich kid, grew tired of "slumming it" and decided to move back into his mansion.
Harry and Peter's relationship quickly turned sour after Peter refused to discuss Spider-Man as Harry felt Peter knew more than he let on. Peter is Spider-Man's "personal photographer," and Harry only wants to avenge the death of his father. It's not unlikely to assume Harry left to live alone because he was angry Peter wouldn't help him find Spider-Man.