The striker opened the scoring in the first half before Antonio Cassano doubled the hosts' lead shortly before the break, but goals from Alessandro Matri and Matias Silvestre in the second half looked to have secured another dramatic comeback for the Rossoneri.
However, Parolo drilled an effort from all of 35 yards into the bottom corner in stoppage time to snatch the three points for the Gialloblu and compound Milan's recent poor run of form away from home.
The hosts made three changes from last week, with goalkeeper Antonio Mirante returning after a one-match suspension while Mario Balotelli was one of four changes for Milan, back in the starting line-up after recovering from a thigh injury.
Parma took the lead with the first action of note in the 11th minute as Parolo found the net, after Ignazio Abate had blocked Massimo Gobbi’s effort into his path following good work from Jonathan Biabany.
The visitors' first chance came in the 33rd minute as Andrea Poli fired narrowly wide of the left-hand post after being played into the area by Valter Birsa.
Cassano then doubled the advantage in first-half stoppage time, receiving a pass from Parolo inside the box and firing into the bottom left-hand corner, with Biabany again troubling Milan left-back Kevin Constant in the build-up.
Five minutes after the break, Gabriel reacted smartly to keep the visitors in the game, getting down low to his left to deny Amauri after he had combined with Cassano.
Walter Gargano then went even closer to a third as Parma continued to dominate, rattling the crossbar from 12 yards after Cassano had teed him up.
They were made to pay for that miss as Milan found two goals in as many minutes against the run of play.
Substitute Matri, on in place of Balotelli, angled a drive across Mirante on the run, the striker's first goal for the club since making his debut as a 19-year-old in his first spell more than a decade ago.
Suddenly Milan were full of life and they found an equaliser two minutes later.
Another second-half introduction, Kaka, was denied by Mirante and from the resulting corner, Cristian Zapata struck the bar with a header before Silvestre nodded home the rebound from six yards.
The sides began to cancel each other out as the half drew to a close, with Nigel de Jong perhaps fortunate to remain on the pitch after a scything challenge before Gabriel made a fine late stop to deny Nicola Sansone.
With the game seemingly destined for a draw, Parolo then stepped up to lash his set-piece beyond Gabriel from distance after Zapata tripped Sansone in stoppage time as Milan's wait for a first league away win of the season continues.