Inside Pierce Brosnan's Heartbreaking Loss of His Wife & Daughter to Cancer: 'I Was in a Helpless State'

Pierce Brosnan is all too familiar with the pain of personal loss and grief.

While promoting his new AMC show The Son, the 63-year-old actor recently opened up about the heartbreak of his past — twin tragedies that struck 22 years apart: Brosnan lost his first wife, Cassandra, and their daughter, Charlotte to ovarian cancer.

“There is an incredible cruelty in it all, losing a person you shared everything with,” he previously told PEOPLE in 1992, four months after Cassandra’s death. “This is the first time in my life I’ve ever experienced bereavement, and it’s overwhelming.”

Brosnan met the Australian-born Cassandra, whom he called Cassie, in the late ’70s and they married in December 1980. The actress had two kids from a previous marriage, Charlotte and Christopher, and the couple had their only son, Sean in 1983. After Charlotte and Christopher’s father, Dermot Harris, died in 1986, Brosnan adopted the children and they took his last name.

The family of five often traveled together as Brosnan’s career took off and his wife would encourage him to take on bigger and broader roles, but that changed in 1987 when they returned to London from India to find out Cassandra, who was in her late 30s, had an aggressive form of ovarian cancer — the disease that had taken her mother.

“From day 1, we really had a fight on our hands,” said Brosnan. “This wasn’t a shadow or a small tumor — this had invaded Cassie’s being.” But his wife, he added, “took her destiny in her own hands with incredible courage and grace.” She wisely questioned every prescribed treatment and decided which ones to take, which to reject. “You must,” Brosnan said. “As frightened as you are, you have to second-guess. I was the quiet party, but I was always there for her.”

Cassandra, who starred in the 1981 James Bond movie For Your Eyes Only, went through four years of treatment, including eight surgeries and a year and a half of chemo. Two years into her battle, the couple moved to Malibu and Cassandra dedicated her time between surgeries and chemo to decorate the house and make it a home for their three children. The house, Brosnan said, “gave Cassie so much joy.”

The couple continued to fight the disease until December of 1991, when Brosnan said they realized it was the end.

“I was in a helpless state of … confusion and anger,” Brosnan said about a moment when they lay in bed together near the end. “She was comforting me. She said, ‘Please, darling, don’t worry. It’s just a life winding down.’ What can you do? Up until then there was always something, some new treatment. But then the options got fewer and fewer. At the end, Cassie didn’t want to be resuscitated [by] any machines.”

Cassandra died on Dec. 28, 1991 at the age of 43, with Brosnan and Christopher by her side. Charlotte, who was 20 and in London studying acting, was on the phone. Hours after her death, Brosnan went home to tell a then-8-year-old Sean about his mom.

Though it was the hardest time in Brosnan’s life, the actor continued to be there for his children and did everything he could to give them a joyful life — all while still feeling Cassandra’s effect on him. “She has made me the man I am, the actor I am, the father I am,” the actor said at the time. “She’s forever embedded in every fiber of my being. She’s there with me every day. I was so blessed to have met someone like that.”

In 2013, almost 22 years after Cassandra’s death, Brosnan was struck by tragedy again when their daughter Charlotte died of the same disease at the age of 41. The late actress had just gotten married to her longtime love Alex just weeks before her death. The couple had two children, Isabella, who was 14 at the time and Lucas, who was 8.

“Charlotte fought her cancer with grace and humanity, courage and dignity. Our hearts are heavy with the loss of our beautiful dear girl,” Brosnan told PEOPLE at the time.

Charlotte’s death was another hard blow to Brosnan, who relied on Christopher and Sean and his new family — current wife Keely Shaye Smith and sons Dylan and Paris — to help him cope with his grief.

The family leaned on one another once again as they grieved the loss of their unofficial “custodian of laughter,” as Charlotte was nicknamed. Charlotte was “a bubbly, almost kind of goofy, gorgeous girl,” Brosnan’s longtime friend Nancy Ellison told PEOPLE after her death. “Pierce wrote to me after she died that the most intense memory that he had was of always being able to make Charlotte laugh. He wanted to be able to make her laugh again.”

Charlotte’s children were roughly the same age that she and Christopher were when they lost her mother, Cassandra — and family friends said she had already made a huge impact on her children, much like Cassandra had on her children.

“To their mummy, Bella and Lucas were absolutely the pinnacle,” said her close friend Clare Beckwith. “They have always got that for the rest of their life, that their mum just worshipped them. People would ask her why she didn’t move into acting, and she would always say, ‘No, this is my family.’ She wanted to give her family as much love as she could. She was totally devoted to them.”