Factual error: The Prince Regent's servants address him several times as 'Your Majesty'. Since his father, the King, is still alive he is actually 'Your Royal Highness'. No royal servant would make this mistake.
Factual error: Several of the soldiers at the Tower wear their hair tied back in queues. These were abolished in 1808 and the series is set in 1814. Since they were loathed as well as against regulations, it's highly unlikely that any soldier would have worn them after they were abolished.
Character mistake: The quote on the wall in Bethlem: Mens sana in corpora sano. 'Corpora' would be feminine, whereas 'sano' is masculine. So it should be Mens sana in corporE sano.