Other mistake: In the final scene outside the MIB headquarters, Kay opens the neuralizer and we see it open, but there is no sound. Later when Kay tells Jay that he hasn't been training a partner but a replacement, we hear the sound of the neuralizer opening and Kay tells Jay how to operate it.
Other mistake: There is a mistake in the French subtitle translation. In the diner when the cook asks Edgar the Bug, "where's Ivan?" Edgar responds, "[I] gave him a break". The translator writing the subtitles must have thought he was saying "GIVE him a break" because the subtitle reads, "laisse-le tranquille" which means, "leave him alone".
Other mistake: Soon after Jay pushes the little red button on the Ford POS, Kay inserts an eight track cassette. If you look at the eight track player, you can read "Select". The car is upside down at this point, but the word "Select" on the player, as well as the title of the cassette, "Elvis 'Promised Land'" are both right-side up. This means that, if the car were right-side up, the text would be upside down, and, in turn, the entire cassette player would be upside down. It is clear that this mistake was made by not paying attention to the orientation of the car as the scene was shot. It is also clear that the shot is done with the camera right-side up since Kay is in the driver's seat, which, right-side up, would be on the left, but since the car is upside down, he's on the right, which is seen when he inserts the tape.