Brown was adjudged to have recklessly tackled the former Liverpool midfielder and was given his marching orders.
Steven N'Zonzi, having set up the opener, doubled Stoke's lead nine minutes from the end as Mark Hughes' men made the most of their numerical advantage to record a first Premier League win since the end of August.
The result moves Stoke three points clear of the relegation places and drops Sunderland to bottom.
Hughes made one change to his Stoke starting line-up, with Glenn Whelan coming in for Wilson Palacios, while Sunderland manager Gus Poyet named an unchanged team from the one that garnered a 1-0 win over Manchester City prior to the international break.
It was Sunderland who were first to find their rhythm. Adam Johnson and Emanuele Giaccherini combined to set up Jack Colback, whose shot was blocked, before Stoke goalkeeper Asmir Begovic brilliantly denied Steven Fletcher from a one-on-one opportunity.
Despite the clever passing of Adam and some neat touches from Peter Crouch, Stoke struggled to create a meaningful chance until breaking the deadlock after half an hour.
N'Zonzi raced on to Geoff Cameron's long punt forward, held it up and stroked the ball back to the onrushing Adam, who stroked into the bottom corner from just inside the box for his third goal of the season.
Things got considerably worse for Sunderland on 35 minutes when Brown appeared to have made a fair sliding challenge on Adam, but referee Kevin Friend adjudged that the 34-year-old defender had dangerously fouled the Scot in the process and sent him off.
Stoke enjoyed much of the possession at the start of the second half.
However, both Jonathan Walters and Crouch failed to connect with an inviting Cameron cross, before Adam burst through the midfield but failed to keep his shot down.
Johnson found space on the edge of the opposition box on 70 minutes, but the winger, who scored a fine strike for Manchester City against Stoke in the 2011-12 season, dragged his shot wide.
Begovic had to be alert to race from his line and, with an outstretched leg, clear the ball ahead of the advances of Fletcher. The Bosnia-Herzegovina international caught the Scotland striker in the process, but Sunderland’s appeals for a penalty were waved away.
Sunderland were enjoying a good spell of pressure, but were hit on the counter-attack with nine minutes remaining. Adam fed Couch, who slid the ball between two defenders for N'Zonzi to clip it homeand become Stoke's 10th scorer in the Premier League this season.