Tom – who accompanied the Duke and Duchess of Sussex on last year's royal tour of Africa and featured in their African Adventure documentary – is a close confidante of Harry and Meghan.
Tom claimed the couple were told by the palace that there were plans for a "slimmed-down monarchy" and they "weren't really a part of it".
He told ITV News: "It had been made clear to them in their absence there was going to be a slimmed-down monarchy and they weren't really a part of it."
When they came back from Canada, the couple reportedly wanted to discuss their plans with the royal family and were told to put their suggestions "on paper".
Tom said they didn't for fear of leaks. However, when some details were leaked, they decided to go public with their plans without informing Harry's family.
As a result, Tom claimed that it was not true that the palace was "blindsided" by their announcement that they were stepping down as senior members of the royal family.
However, the Queen was said to be angered by its timing.
"The couple's view was they came back and wanted to talk to the family about their plans," Tom claimed.
I guess Harry and Meghan have done Prince Charles a favour. He intends to slim down the working Royal family, and it seems they jumped before they were pushed...
And this way they get to own the narrative. Charlie must be FEWMIN!
— Ms Person (@MsPerson) January 9, 2020
Tom's comments came after ITV's royal correspondent Chris Ship revealed the New Year portrait featuring the Queen, Prince Charles, Prince William and Prince George may have been the "final straw" for Harry and Meghan.
It had been made clear to them in their absence there was going to be a slimmed-down monarchy and they weren't really a part of it.
He told Good Morning Britain: "The portrait that Buckingham Palace released to mark the start of the decade – it was the Queen and the three future kings."
He added: "I'm not saying they were cutting Harry and Meghan out because it was a brutal fact that Harry was never part of that line of succession."
View this post on Instagram
📸 To mark the start of a new decade, a portrait has been released of Her Majesty The Queen and Their Royal Highnesses The Prince of Wales, The Duke of Cambridge and Prince George. The portrait was taken by Ranald Mackechnie in the Throne Room at Buckingham Palace. Photograph © Ranald Mackechnie
However, he added: "But start of a decade it felt like the royal family was very much focusing on the future – the Queen and her three heirs."
Meghan and Harry are now said to have flown back to Canada, where they left baby Archie with his nanny.
Head to our Facebook page @EntertainmentDailyFix and let us know what you think of our story.