Factual error: The Western Electric Model 639A microphone prominently featured in front of J Edgar Hoover in the 1934 Washington DC Senate appropriation hearing scene, wasn't invented till late 1939.
Other mistake: In the scene where Dillinger and his gang raid a bank and take the bank manager and a teller hostage, they come out shooting with machine guns and jump into and on the running boards of the getaway cars, all through this encounter the bank manager shows no concern for his own safety, no reaction whatsoever to flying bullets and loud gunfire, very strange.
Continuity mistake: When John gives Billie her mink coat, the amount of wrap paper coming out of the corners of the box differs between the front and back angles.
Chosen answer: Baby Face Nelson doesn't pick up Dillinger. He picks up the characters played by Stephen Dorf and another associate. Dillinger and Hamilton escape through the woods and steal a car from an older man they come across in the forest. After the shootout, Dillinger and Nelson are never in the car together.