Continuity mistake: When Glenn wakes up we see the door behind him is flush with the concrete, but when he goes inside you see stairs he had to descend to get to the door. There is also no scene in the entire prison episodes where an outer orange door is preceded by steps, they are all flush. Also If you look at the door behind him you see two folding chairs leaning up against the door. When he turns to open the door the chairs are gone.
Internment - S4-E5
Continuity mistake: In the previous episode, Rick and Carol are shown hunting through houses for household drugs to fight the flu. They even make a mention that most of what they find expired a year ago, but to bring it back to Hershel anyway, and let him decide about it. But in this episode, Rick returns, talks to Hershel, and no mention is made about him bringing medicine. Considering how important it was to fight the flu with any means possible, Rick would not have let Carol drive off with the potentially life-saving drugs.
Answer: The walkers could have knocked them down more than once. There may have been another wave that knocked down the fence earlier and they're going to put it up again soon.
Captain Defenestrator