Stags boss Peter Shirtliff said "Walsall's first goal came from their first shot on target, which is happening to us all the time, and the second goal was due to a goalkeeping error.
"After that I was disappointed with our character but we have to pick ourselves up and it's no good moping about it.
"But we are too easy to score against and that's our problem.
"Their goalkeeper made a very good save from Alan Sheehan's free kick at 1-0 and things like that change a game. Walsall's goalkeeper played very well and I thought at 1-0 he was the difference between the two sides."
Walsall manager Richard Money said: "We were forced to change the shape slightly because we were being outnumbered in the middle of the pitch.
"The players showed they are very adaptable and listen to what I say at half-time and full credit to them."