Now Mansfield need to win their last game of the season at Dagenham - whose fightback at Darlington looks to have condemned the Stags to non-league football - and hope that Chester lose their remaining two matches.
There looked to be no danger when Yates punted the ball upfield from the byline and 40 yards out in the 71st minute.
But young keeper Jason White completely misjudged the ball in a swirling wind and it dropped under the bar and into the roof of the net to settle a fiercely contested local derby - and more importantly leave the Stags' 77-year stay in the Football League almost at an end.
The goal - which clinched Rotherham's third successive win - also sparked angry scenes at the final whistle.
Fans chanted against controversial owner Keith Haslam and then some tried to force their way into the boardroom and confront him, before being turned away by police.
The Stags, looking for back-to-back wins for the first time this season, had been dealt a massive blow just before kick-off when influential defender Alex John-Baptiste was ruled out through injury.
But Rotherham, whose players again agreed to defer a percentage of their wages, were boosted when keeper Andy Warrington shrugged off injury to play.
And the Millers started the derby, in front of a packed Field Mill and a red-hot atmosphere, the better. Keeper White had to kick away a Tom Cahill byline cross and then, as the Stags struggled to clear, Jake Buxton blocked a Mark Hudson drive.
At the other end Micky Boulding went close to adding to his 25-goal tally with a weaving run and shot.
Then Jefferson Louis flashed a cross-shot across the face of goal which the striker was unable to touch in.
The Stags carved out the best opening of the early stages in the 13th minute, a neat exchange of passes between Louis and Boulding leaving the leading scorer clear, but his finish was pushed away by the diving Andy Warrington.
Rotherham rallied and only a terrific block by White denied Cahill for a second time after the striker wriggled clear in front of goal. From the follow-up a fine last-ditch tackle from Stephen Dawson stopped Danny Harrison's goalbound drive.
Then White was beaten from 20 yards, but Peter Holmes' curling shot shaved the outside of the post from Ryan Taylor's knock down.
After a helter-skelter opening the match settled into a scrappy, midfield affair until Boulding struck a tame 20-yard drive wide eight minutes before half-time and moments later Keith Briggs blazed over from 25 yards.
A fine diving save by Warrington kept out a Nathan Arnold strike in the 40th minute as Mansfield regained control of a fiercely contested derby.
Then a terrific sliding tackle from Graham Coughlan blocked Boulding after another link-up between the striker and his partner Louis.
The Stags maintained their attacking momentum at the start of the second half, Louis' bulldozing run ending with a blocked shot that Warrington comfortably collected. The keeper then dived low to hold yet another Boulding shot.
On a rare Rotherham counter-attack, a poor Mullins clearance allowed Holmes a free shot, but he curled tamely into the arms of White.
However, from the hour mark the visitors took control and Dale Tonge blazed over from 30 yards.
But as news came through that relegation rivals Dagenham had turned a two-goal deficit at Darlington into a 3-2 lead, substitute Yates grabbed a freak 71st-minute winner for Rotherham.
A hopeful punt into the area from 40 yards out on the byline somehow fooled White and dropped under the bar and into the net.
Within 60 seconds the Stags should have been level, but Boulding, sliding in, poked Arnold's cross over from just six yards.
Five minutes from time White redeemed himself with a flying save to deny Mark Hudson a killer second goal.
Then, in stoppage time, Rotherham full-back Tonge over-ran the ball in front of an empty net.