Other mistake: When the cons are drinking beer on top of the roof, the man that finished his bottle gestures for another. The guy takes out an already-opened bottle of the bucket horizontally, the beer should've poured out.
Other mistake: When the warden is questioning the night guard in Andy's cell the morning-after the escape, he rips the clipboard out of his hand and hands it to Hadley. The pencil is stuck to the paper but is not under the clip. The pencil would have flown off.
Chosen answer: In a nutshell, tax evasion. He's running the prison like a company doing building work with, as its described by the builder in the movie, a pool of slave labour to allow him undercut other contractors. He is skimming profit for his own pockets, along with taking bribes, etc. that Andy's creative book keeping is hiding from the IRS.