Former military man turned television star Ant Middleton faces an Ofcom investigation.
According to The Sun, the remark sparked 16 complaints to Ofcom, which has the power to open a formal investigation.
Ent Daily has contacted reps for Ofcom and Ant for comment.
During the opening episode of the new series, Ant shouted at the celebrity contestants as they ran across tough terrain.
He barked at Anthea to hurry up and referred to her as "grandma".
"Come on Grandma"
Ant said: "Come on Grandma, get a move on!"
Many took to Twitter to share their outrage at the comment.
One tweeted: "#CelebritySAS Typical misogyny, calling Anthea grandma. [Bleep]."
Another argued: "No need to call Anthea Turner 'grandma' just because she is older. She did the backwards dive. Judge her on that. #CelebritySAS."
A third chimed in with: "Absolutely no need to call Anthea Turner grandma, totally disrespectful #CelebritySAS @antmiddleton."
A fourth posted: "#CelebritySAS slightly offended by the grandma comment to Anthea turner."
Television presenter Anna Richardson even joined in by tweeting: "Good to see a bit of good ‘ole fashioned ageism & sexism their lads with the ‘grandma’ references."
"Bit of good 'ole fashioned ageism"
Anthea didn't seem too bothered by the cheeky remark, and gave her verdict to Hello! magazine.
She said: "Well, I think that's very kind of them but I absolutely don't hold it against him whatsoever.
"I do completely understand why they do that."
She added: "Oh it will be lovely to be a grandma one of these days."
She also confirmed to Metro that his comments weren't going to "rile" her up.
Anthea explained: "Even though Ant was enjoying calling me Grandma there were times when I was thinking, I’m not old enough to be your grandma, you’re not going to rile me. I’m old enough to give birth to you but not old enough to be your grandma."
Despite Anthea being the oldest contestant to ever compete on the gruelling reality show, she performed remarkably well at the majority of the tasks.
However, she became the second celebrity to quit (after Katie Price) after failing to keep up during a 5km run.
Strapped down with weights, she struggled to stay with the rest of the contestants.
Celebrity SAS: Who Dares Wins continues on Channel 4 Mondays at 9pm.
Do you think Ant Middleton went too far with his jibe? Share your thoughts with us on our Facebook page @EntertainmentDailyFix.