Continuity mistake: Archangel's alumni status (or lack thereof) with the X-Men varies throughout the series. Later episodes showed him to be an original member of the team just like in the comics, yet when he first appeared in Season 1, none of the other original members in the show (Cyclops, Jean Grey and Professor X) seem to know who he is. The time traveling mutant Shard also states in Season 4 that he is destined to eventually join the X-Men.
Deliberate mistake: There's at least two episodes (Season 1's "Captive Hearts" and Season 4's "Sanctuary, Part 1") in which a character shows video footage of the X-Men and various supporting characters in action, but the footage is merely clips from previous episodes.
Audio problem: Cable, Erik the Red and Captain America all have the exact same voice.
Answer: Chris Potter, the original Gambit voice actor, did indeed quit the role in the 4th season. His last episode was The Phalanx Covenant Part 1 so it seems reasonable that Fox would limit the use of the character even though it was recast. It doesn't appear that Potter left the role due to animosity, he stated in an interview that he wished to play Gambit in the first live action X-Men film.
BaconIsMyBFF