Finnigan raced on to a clever lay-off from substitute Kayode Odejayi and rammed in a 25-yard drive to stretch Cheltenham's winning run to three games and end Mansfield's own two-match winning sequence.
Mansfield thought they had earned a point when Colin Larkin's 73rd minute penalty, the striker's first goal for four months, cancelled out Martin Devaney's 61st minute curling drive.
But Cheltenham, who adapted better to the gale force wind, had the final say just moments after keeper Shane Higgs had kept them in the game with a diving save to deny Adam Rundle.
Mansfield caretaker manager Carlton Palmer had sprung a surprise before kick-off when he gave a debut to diminutive wing man Goma Lambu. But the former Millwall player had little chance to impress in a turgid opening 20 minutes dominated by the weather.
The home side, backed by the very strong wind, finally created a chance in the 25th minute when good work by Lambu and Richie Barker released Larkin, but he shot tamely at Higgs.
Tom Curtis twice and Adam Murray also went close and in first half injury time Barker, looking for his third goal in a week, headed Rundle's deep cross narrowly wide.
Two minutes into the second half Mansfield keeper Kevin Pilkington was finally forced into action, diving low to his right to palm away a Grant McCann swerving shot.
Cheltenham, now with the gusty wind at their backs, went close again just 30 seconds later when Jamie Victory glanced a McCann corner into the side netting.
In the 55th minute Pilkington acrobatically beat away Brian Wilson's far post finish before the almost incessant Cheltenham pressure finally paid dividends.
There looked to be no danger when Devaney picked up the ball outside of the area on 61 minutes, however, he unleashed a curling effort around an oncoming defender and in off the post to stun the home side.
In the 67th minute McCann scrambled a Barker shot off the line, but Mansfield did level six minutes later when substitute Derek Asamoah was sent tumbling in the area by Victory and Larkin converted the penalty.