Factual error: At an 1868 church service, Mark sings "How Great Thou Art." While the tune of this hymn is 19th Century, the English lyrics he's singing weren't composed until the 1920s.
The Case of the Wayward Wife - S3-E13
Factual error: May buys a book for 4.95. Give the clerk a $10 bill but the register shows change of $5.15.
The Hoodlum - S4-E1
Factual error: Diamond's car phone, quite a rarity in the 1950s, attracts a lot of attention. But the car should have a huge antenna attached to make the gadget work (it doesn't), and Diamond would not be getting direct calls - they'd have to go through a mobile operator first.