Deliberate mistake: The Olympic symbol (of 5 inter-connected rings) is upside-down on the tracksuits the torch carriers are wearing near the end of the episode. It has 3 rings at the top, not 2.
Deliberate mistake: After Peggy has knocked King Philip off his horse at the end you can the sun is setting behind her, when it goes to a reaction shot of Bobby and Luann in the stands, it is a blue sky and sunny behind them. The following shot of Peggy it goes back to the sun setting. The mistake could be said to be deliberate as the shot was used earlier in the scene and probably used again to save time and money than re-animating the exact same shot.
Answer: If you are talking about the "Thou shalt not do this, thou shalt not do that" song, I'm not sure if a song like that actually exists. Green Day, the guest voices on the episode, might have just been improvising to keep their appearance from being too obvious.