Factual error: There is a scene where Chitty is pacing the train, which is an 'Austerity' tank engine. This engine was made during WW 2 but the movie takes place in the early 1900s.
Continuity mistake: In the candy-tasting scene with Lord Scrumptious, his head chef blows a whistle and holds it at chest level. In the next shot, Caractucus descends the stairs behind the chef, and now the whistle is down by the chef's side.
Continuity mistake: When Truly accompanies Mr. Potts and the children to the beach, her hair is first in a bun, and then later is worn down around her shoulders when they reach the beach. Also, the tightness or her curls vary from scene to scene while at the beach.
Answer: It seems those times were occasions when she was on her way to see her father, Lord Scrumptious, at his candy factory, or wherever he may be (like at the Potts' wargaming with Grandpa).
Scott215