Trivia: The Anabelle doll from The Conjuring, another film of director James Wan, is visible while Mera is taking Arthur to her submarine ship. Blurry but there.
Continuity mistake: Right when the guards enter the home and the walls explode, Arthur swaps between having short curly hair to it being very long and straight.
Answer: You said it yourself some might have stumbled upon it, others could have used the map, which is why it was hidden in the first place.
How could they use the map when Arthur was the first to find it...quite frankly to stumble on it is impossible, you would have to randomly travel to the trench, go through the entire kingdom, see a tornado of death and decide to go in, get spat out and not eaten, get to shore, check behind a waterfall and follow it down...bit much?