Aunt Bee's Invisible Beau - S5-E27
Revealing mistake: When Clara and Aunt Bee are talking in the kitchen, in the closeups facing Aunt Bee we see the counter, the cabinet, and the wall, but what we don't see are the curtains, the window, or the windowsill, which all should be on that wall. The fake window wasn't placed on the set wall for Bee's closeup.
Answer: Those "boxes" are valences that used to be very common, before air conditioning. They allow for windows to be open during rain storms. They permit air circulation, without letting the rain in.