Revealing mistake: After a shootout with Baby Face Nelson at a hotel, Melvin Purvis converses with J. Edgar Hoover over the telephone. As he says, "Our type cannot get the job done." Watch the background. You can see a modern era train move along the tracks in the darkness. This is inconsistent with the film's setting of 1933/1934.
Revealing mistake: In the scene where Dillinger breaks out of jail in Indiana, there is a shot of the getaway car and its license plate, and you can see that the license plate says "Historic Vehicle" on it.