Factual error: The Prince Regent's servants address him several times as 'Your Majesty'. Since his father, the King, is still alive he is actually 'Your Royal Highness'. No royal servant would make this mistake.
Episode #1.7 - S1-E7
Factual error: Lorna uses the word 'okay'. The first recorded usage of this word in America was not until 1839 and in Britain not until the 20th century. The series is set in 1814 and generally uses language contemporary to its setting.
Episode #1.8 - S1-E8
Factual error: Sir Stuart Strange, on learning that Wilton and Pettifer have not been contacted conjectures, "they must have left early for the weekend" This was 1814 and the earliest recorded use of the word, and concept of weekend was in 1879.
Factual error: Several of the soldiers at the Tower wear their hair tied back in queues. These were abolished in 1808 and the series is set in 1814. Since they were loathed as well as against regulations, it's highly unlikely that any soldier would have worn them after they were abolished.