Deliberate mistake: When the penguin goes into the cemetery, he bumps a tombstone and it moves. Almost certainly a tribute to legendarily poor director Ed Wood, who Tim Burton later directed a biopic of - this exact thing happened in one of his films. (00:39:15)
Deliberate mistake: Gonzo and Rizzo are selling apples in the beginning, including the Red Delicious cultivar. That wasn't invented until the 1880s and the story is set sometime in the first half of the 1800s. Most likely intentional as they are the narrators.