Deliberate mistake: There are several historical dates wrong in this episode. For a start, Prince George didn't become Prince Regent until 1811, and yet this is set before the Battle of Trafalgar in 1805 (as Black Adder gives Wellington some advice on it). Similarly, Wellington didn't get the title "Duke of Wellington" until 1814. Again inconsistent with being pre-Trafalgar.
Deliberate mistake: There are a lot of jokes with the name of the book "The Prince and the Pauper." However the book was published in 1881, 51 years after the Prince Regent died.
Answer: The closest we ever get to knowing his first name is in series 3, when he claims it's "Sod Off". Blackadder asks him in series 3 where this came from, and he replies "Well, when I was a kid, living in the gutter, I'd go up to the other kids and say 'Hello, I'm Baldrick'. And they'd say, 'Yes, we know. Sod off, Baldrick.'" The S in this series is likely a reference to that, but it's not addressed, and given this Baldrick is clearly different from the series 3 Baldrick, we can't draw a direct link.
Shay