Although he “feels bad that Selena isn’t doing well, that has nothing to do with his gloomy mood a few days ago,” a Bieber source tells PEOPLE

Justin Bieber still has love for his ex Selena Gomez — even though he is a married man.

A Miami music source tells PEOPLE of the “What Do You Mean?” singer, who secretly tied the knot with Hailey Baldwin, 21, in September, “He is not over Selena.”

“This was his first great love and while he was young and traveling around the world as a superstar, he learned a great deal from her,” the source explains of Bieber, 24, and Gomez, 26.

PAP NATION/SPLASH

The source says that Gomez’s health struggle is taking a toll on Bieber. Gomez, who was hospitalized twice in the last few weeks for a low white blood cell count, recently entered dialectical behavior therapy after suffering a panic attack during her second hospitalization.

“He was extremely upset over what has happened to Selena,” the music source explains. “He feels like she is part of his life and he wants her to be happy and healthy.”

SNORLAX / MEGA

Bieber was seen crying in a car with Baldwin on Thursday as they left their pastor’s home. He also was spotted looking somber as he hugged friends at church in Los Angeles.

A source close to Bieber tells PEOPLE, though, that Bieber’s public displays of emotion were unrelated to Gomez.

“Hailey makes him super happy,” the Bieber source said. “He feels bad that Selena isn’t doing well, but that has nothing to do with his gloomy mood a few days ago.”

Selena Gomez and Justin Bieber. VALERIE MACON/AFP/GETTY

Earlier in October, the Bieber source told PEOPLE, “Justin will always care about Selena.”

The Bieber source added, “It’s hard for him to hear that she isn’t doing well. They have both had their separate struggles over the years and Justin was always very sympathetic to Selena’s. He wants the best for Selena. He wants her to be happy and healthy.”

BILLY FARRELL/BFA/REX/SHUTTERSTOCK; JAMES DEVANEY/GC IMAGES

Another source told PEOPLE in October that Bieber’s marriage is one of many causes of stress for Gomez.

“Yes, having Justin’s relationship everywhere isn’t easy, but she’s also been dealing with other stressful situations and being back in the hospital was really difficult and brought back tough memories,” the second source said. “It’s not fair to say this caused it or that caused it because depression and anxiety is so layered. She knows it’s something she will be facing for the rest of her life, but she’s committed to tackling her issues head on.”

UMV/STAR MAX/GC IMAGES

As for whether the singer’s lingering affection for Gomez could create issues in his marriage to the model, the source close to Bieber says, “It could be a problem down the road.”