The mansion Allie is staying in over the summer is Boone Hall Plantation in South Carolina, near Charleston. It was built in the 1930's, so factually the house couldn't have been in the family as long as the characters make it seem. A lot of the filming was also done in Keowa, an island near Folly Beach.