Mansfield had stormed into a four-goal lead after 32 minutes, but were then forced to hang on as County struck two goals in six minutes before half-time and the Stags lost Lee Bell 15 minutes from time to a controversial red card.
Stockport, unbeaten in the League and having thrown away leads in two drawn matches following an opening day win, almost got off to the perfect start with a first minute goal.
Jason Taylor found space on the edge of the area and his low drilled shot was just palmed away from the foot of the post by the outstretched palms of the diving Carl Muggleton.
The importance of the save from the veteran keeper, recalled for his first game of the season after youngster Jason White's errors cost the Stags points in previous matches, was shown just two minutes later when the Stags took the lead.
Former Stockport star Matt Hamshaw crossed from the right and Michael Boulding knocked his cross down for the unmarked John McAliskey to hammer in his first Stags goal from close range.
McAliskey, signed in the summer from Huddersfield, was then joined on the scoresheet by a fellow former Terrier who made the same switch to Field Mill in August.
For in the ninth minute experienced defender Martin McIntosh curled an unstoppable 25-yard free kick into the top corner, over the defensive wall and past motionless keeper Conrad Logan as the Stags made a blistering start.
Despite the goals, Stockport remained dangerous in front of goal and almost hit back instantly when Taylor again went close from distance, his 20-yard drive grazing the woodwork.
Mansfield almost made it 3-0 in the 19th minute, full back John Mullins flashing a 20-yard blockbuster narrowly wide from an audacious back-heel by Boulding.
But in the 25th minute the third goal came, McIntosh heading back a Hamshaw corner for Jake Buxton to sweep in.
In a remarkable game the Stags then won two penalties in two minutes. First keeper Logan brought down Stephen Dawson, was only booked for the infringement, and got up to save Boulding's spot kick on the half hour by diving low to his left.
However, two minutes later Taylor tripped Mullins and Boulding got his revenge, blasting his penalty into the opposite corner to make it 4-0.
Resillient County refused to give up and just three minutes later launched their fightback when Adam Proudlock shrugged off a defender and finished neatly at the near post.
Then the striker capitalised when McIntosh went off to receive stitches to a head wound, finding space inside the six-yard area in the 41st minute to turn in Anthony Pilkington's cross and complete a memorable first half.
McIntosh never reappeared, forcing the Stags, without a defender among the substitutes, into a major reshuffle.
And within three minutes of the restart County almost capitalised, Paul Tierney's tremendous 18-yard free kick headed off the line by Buxton.
In an equally thrilling second half Muggleton dived full length to hold substitute Gary Dicker's header and Dominic Blizzard blazed over for Stockport, while a magnificent 50-yard run by Hamshaw ended with Logan denying Dawson from point-blank range.
The drama continued 15 minutes from time when Mansfield midfielder Lee Bell, filling in at right back in the second half, was harshly dismissed for a fierce tackle.
Stockport striker Anthony Elding twice missed from six yards in a thrilling finish as the Stags hung on for their first win of the season.