Chelsea made the most of the man advantage and Werner and Chilwell scored in the last 10 minutes to send them top of the Premier League.
Chalobah broke the deadlock after nine minutes when he reacted to Ruben Loftus-Cheek‘s flick on from a corner.
The hosts thought they doubled their lead on two occassions but both strikes were disallowed.
Romelu Lukaku scored after 36 minutes but his strike was ruled out for offside while just before the break, VAR ruled out a Werner goal after Cesar Azpilicueta was adjudged to have fouled Kyle Walker-Peters beforehand.
Southampton performed better following the break and were awarded a penalty after Chilwell brought down Tino Livramento.
Ward-Prowse equalised from the spot on the hour mark but was sent off after 75 minutes.
The midfielder was originally given a yellow card for a challenge on Jorginho but was shown a red card after a VAR check.
With six minutes to go, Chelsea went ahead again after Werner tapped home into an empty net following Azpilicueta’s cross.
It was Werner’s first Premier League goal of the season. Chilwell grabbed a third with a minute remaining to seal the win for Chelsea.