Continuity mistake: At the end, when Kevin Spacey is leaving the police headquarters and that cop is sitting on his desk and is looking at the wall.he sees....something [don't want to give it away], he drops his cup and it breaks, but it breaks in different pieces each frame. (01:35:20)
Continuity mistake: Near the end, when Agent Kujan realizes who Soze is, he drops his coffee cup and it breaks in slo-mo. If you look closely, you can see that the bottom of the cup does not have the red writing on it. (I will not give away what the red writing is, as it may be a spoiler).
Continuity mistake: In the scene where Dean Keaton is outside the police station talking to Edie Finneran, the cigarette that he's smoking is in his mouth, then the shot changes and it's magically sprung from his mouth into his hand. (00:25:15)
Continuity mistake: When the suspects are in the garage after Saul, Saul drops a gun and Keaton kicks it over to McManus. However in the next shot McManus picks up a different type of gun.
Continuity mistake: When we first see the Hungarian guy lying in the hospital he is burned over most of his face, however his eyebrow is still there. In a later scene, his eyebrow is gone but his eyelashes are still there.
Continuity mistake: When McManus and Keaton are clearing out the boat, the time of day outside (as shown by the daylight) changes from shot to shot. It should be dark during the entire scene.
Plot hole: The premise is that the back-story is a fabrication, including the five being placed together in the police line-up and in custody in order to get them to work together. But in the flashback of Keaton in the restaurant, it is Dave Kujan who takes Keaton in. It is inconceivable that Kujan would be complicit in the staging of the line-up (and doesn't fit with the plot). And if Verbal is inventing it, then Kujan is going to know it didn't happen. It doesn't make sense and undermines the plot.
Revealing mistake: In the flashback, when the boy has his throat slit, there's no blood or wound.
Other mistake: The burned guy who's supposed to be Hungarian keeps making bad language mistakes, although his pronunciation is great.
Answer: Keaton doesn't believe in Keyser Soze, and is so convinced that Kobayashi is the one pulling the strings and manipulating them. They figure that if they take HIM out, they are home free.