Deliberate mistake: There are several historical dates wrong in this episode. For a start, Prince George didn't become Prince Regent until 1811, and yet this is set before the Battle of Trafalgar in 1805 (as Black Adder gives Wellington some advice on it). Similarly, Wellington didn't get the title "Duke of Wellington" until 1814. Again inconsistent with being pre-Trafalgar.
Deliberate mistake: There are a lot of jokes with the name of the book "The Prince and the Pauper." However the book was published in 1881, 51 years after the Prince Regent died.
Answer: His first name is "Sod Off". Blackadder asks him in series 3 where this came from, and he replies "Well, when I was a kid, living in the gutter, I'd go up to the other kids and say 'Hello, I'm Baldrick'. And they'd say, 'Yes, we know. Sod off, Baldrick.'"
Shay