Substitute Louis Dodds glanced in an injury-time winner to leave the Stags without a win in seven matches and leave them on the fringes of the relegation battle.
Dodds, on loan from Leicester, raced in at the near post to meet Gary Brown's cross unchallenged and hand Dale a dramatic winner with his first goal for the club.
Mansfield at least ended their long wait for a goal when veteran defender Jon Olav Hjelde snatched an equaliser deep into first half injury-time, Mansfield's first goal in 337 minutes.
The Stags, ravaged by injury in recent weeks, were dealt a blow in the pre-match warm-up when captain Gareth Jelleyman aggravated an Achilles tendon injury and limped off the pitch - handing 18-year-old youth team captain Ashley Kitchen a surprise last-minute debut.
Mansfield were able to welcome back Stephen Dawson and Hjelde from injury, and also replaced out-of-form young keeper Jason White with the vastly experienced Carl Muggleton for his first start since December.
Dale, beaten just twice in 14 games but having ended their play-off dreams in an Easter Monday defeat, started brightly and Glenn Poole's early cross almost bounced in before Chris Dagnall's goal-bound drive was blocked by Alex John-Baptiste.
But the visitors did find an early breakthrough in the ninth minute when Glenn Murray raced clear onto Brown's hopeful punt up-field, sprung the offside trap and cleverly lobbed the advancing Muggleton from 20 yards.
The Stags almost levelled just seven minutes later, Barry Conlon heading Matt Hamshaw's floated free-kick against the post with keeper Matthew Gilks beaten.
Conlon then nodded another Hamshaw cross straight into Gilks' arms, before the keeper bravely raced out to save at the feet of the on-rushing Dawson.
In the 44th minute Hamshaw again created a chance for the home side with an incisive pass, but Michael Boulding hesitated and keeper Gilks saved his final shot with his legs.
However, two minutes into injury-time the Stags finally found an equaliser, Conlon heading down a Hamshaw corner for Hjelde to poke in from close range - ending Mansfield's 337-minute goal drought.
Eight minutes into the second half a long throw from debutant Kitchen created mayhem in the Dale area and both Conlon and Callum Lloyd saw shots blocked as the Stags, now looking more confident, chased a winner.
In the 72nd minute Bryan Hodge's clipped drive was well held by Gilks and that save proved vital when Dodds struck in injury-time to snatch victory.