The Millers swept into a fourth minute lead when a corner swung in from the left by Stephen Brogan was headed down by Ian Sharps and Marcus Bean was hovering at the edge of the box to smash in a low shot that crept under keeper Carl Muggleton.
And when the home side struck on the counter attack, Derek Holmes was only a fraction wide with a towering header.
Young striker Marc Newsham made his first start of the season and snapped up his first goal of the campaign in spectacular style.
Veteran defender Paul Hurst curled in a cross from the left and Holmes headed into the path of Newsham, who produced a textbook overhead kick which crashed past Muggleton.
Mansfield responded with their best spell of the game and saw two chances go close from striker Michael Boulding.
In the 34th minute, Mansfield's Rotherham born winger Matt Hamshaw picked out Boulding with a well-aimed free-kick but the header was off target and, when Hamshaw swung in another cross from the right, Boulding did well but this time keeper Andy Warrington made the save.
Mansfield keeper Muggleton, who had been doubtful for the game after an injury in midweek, was replaced for the second half by Jason White.
Another chance on the break came for the visitors when a clearance by White was volleyed by Hamshaw into the path of Boulding, but Warrington made the save when he angled a low shot.
But Andy Todd might have added a third for Rotherham when he made a quick break from midfield but White saved well.
And White produced another last-ditch stop to deny Newsham a second when the livewire frontman nipped in smartly and almost clipped the ball past White but the keeper got a vital touch.
The third goal came after 67 minutes when Newsham saw his header hit the bar and Brogan raced in to nod in the rebound home for his first of the season.
Boulding claimed his sixth of the season with a 76th minute shot after he pounced onto a loose ball in the area, and substitute Nathan Arnold gave the visitors late hope when he crashed in a second eight minutes from the end.