It comes as she awaits trial for assault by battery on boyfriend Lewis Burton.
Caroline is due in court at the start of next month and it's said that, despite putting on a brave face, the star is "struggling".
A source told New! that Caroline isn't in a good place.
They said: "Caroline is home and friends fear she’s not in a good place. It’s a major struggle for her and she’s feeling the pressure of the public judging her. As much as she’s trying to put on a brave face, she’s privately, and understandably, feeling very stressed."
Although Love Island bosses have said the door is open for the star to return to the show, the "massive turnaround in her fortunes" has taken its toll.
She’s fretting to her closest friends that she lies awake at night thinking her career could be over and everyone is revelling in her downfall.
"She is really struggling with the massive turnaround in her fortunes. The toll it’s taken on her has been immense. She’s fretting to her closest friends that she lies awake at night thinking her career could be over and everyone is revelling in her downfall," they added.
The fact that Caroline and Lewis are banned from contacting each other as per her bail conditions is also affecting the star.
"The worst thing is she doesn’t even have the security of her relationship,” the insider added.
Last week, Lewis took to Instagram to insist the pair were still very much an item, despite reports they had broken up.
He shared a picture of them together on Instagram Stories and said: "We ain’t split," with a love heart emoji.
He added: "Can’t wait to see you."
Head to our Facebook page @EntertainmentDailyFix and let us know what you think of our story.