Revealing mistake: In the scene where Warden Norton is loading his pistol, as he dumps out the bullets and loads them the primers are dimpled. Unfired bullets would not have the dimples from the gun's hammer (even if they were home made reloads, the primer would have been replaced).
Continuity mistake: When Hank Williams is singing "Lovesick Blues" the MGM label is from 1968. This is meant to be 1963.
Other mistake: When the opera records are being played, the camera stops near a loudspeaker in the main yard. There are two terminals for the wiring of the speaker, but no wire is to be seen; however, music is heard coming from that speaker.
Revealing mistake: When Brooks is carving his name, "dust" seemingly like plaster dust vs wood dust is shown falling on the table top, his shoes etc. When they cut to the view of Brooks looking at his work and closing his knife, there is no "dust" on his hand or knife. That much carving with that much dust falling on the table would have left his knife covered with dust. It was clean.
Continuity mistake: The warden pulls out a box of Remington .38 Special cartridges to load his revolver prior to his suicide. The warden empties the cartridges on to his desk and loads the revolver with what are Winchester/Western cartridges.
Chosen answer: He probably only knows the name. He doesn't want to know anymore, so he can pin the blame on Andy should anything go wrong. He no doubt hasn't even considered that Andy might have ID so he can access the accounts after his escape.
Grumpy Scot
Also, Andy indicated that the deposits were night deposits, which were in envelopes. So, the warden may not have known the "Randall Stevens" name.