Deliberate mistake: 1-1 (Pilot): Coming back to Earth in 1998, Stone is amazed to see the Cincinnati Reds and the NY Yankees on TV in an interleague game. This wasn't yet possible in '98, as an American League East team could then play only a National League East team (and the Reds are NL Central). But the producers put it in anyway as their own little "fantasy baseball game," because these were their favorite teams.
Continuity mistake: 1-4 "Repentance": The memorabilia dealer shows Stone slides of escaped Nazi war criminals. There's a page of text projected on the screen, which Stone asks the man to translate. While they talk, the image on the screen changes to a picture between takes, even though we can see that the projector's carousel tray hasn't moved.
Factual error: 1-2 "Heat": The Los Angeles police car pulls up to John's loft, which has the address number 923 on the wall. The building across the street is numbered 929. In California (in fact, in the vast majority of the US) address numbers are odd on one side of the street and even on the other. 923 and 929 wouldn't be on opposite sides.