Three out of five parents who went back to work after their baby died said no-one talked to them about their loss, a stillbirth charity says.
A survey of more than 2,500 bereaved parents also found that most employers did not offer any support.
One father said he was sacked after taking seven days off when his baby died.
Another father, Dan Wood, who lost his two-day-old daughter, says deaths should be acknowledged and not ignored.
"It's difficult to start a conversation about it but I would rather talk about her than not," he says.
"Our baby was part of our lives for a long time."
Sands, the stillbirth and neonatal death charity, is calling on colleagues and employers to find the words to talk about the loss of a baby.
Dan, from Warwickshire, returned to work in a large team after Layla, his first baby with wife Amanda, died in December 2014.
'Talking helped us grieve'
Recognising how uncomfortable some people might be talking to them about what happened, the couple wrote down their experience on paper and handed it out to colleagues and friends.
They also told people to ask them about it if they wanted.
"Some felt comfortable, some didn't, but overwhelmingly most mentioned it.
"And talking really helped us both with the grieving process," Dan says.
However, not every bereaved parent has such a positive experience.
One mum who completed the Sands survey commented: "At first my colleagues never said a word about my baby, although quite a few of them, including my manager, attended the funeral, which was very kind and very supportive."
Gradually people did broach the subject, but she said: "I wish someone had said to them it's OK to talk to me about the baby."
Another mum from south west England said people at work moved on quickly "excitedly telling me about their baby news", so she didn't bring up the subject of her baby's death.
- Baby loss: 'People sharing stories is the biggest comfort'
- We must do better on baby deaths and injuries – Hunt
Responses to the survey found that only one in five parents had been offered or given any support by their employer on their return to work and around half said employers did not discuss entitlements to pay and leave following the death of their baby.
Two out of five parents said they were not offered any additional time away from work.
If a baby is stillborn after 24 weeks pregnancy, or is born alive at any time but dies shortly after birth, parents are entitled to maternity, paternity and shared parental leave, as well as full parental rights and benefits.
The government is proposing two weeks of additional pay and leave for bereaved parents from 2020.
In the UK every day, 15 babies die before, during or soon after birth.
'I'm so sorry'
Dr Clea Harmer, chief executive of Sands, said: "Sadly, the death of a baby is not a rare event but too often, bereaved parents are faced by a wall of silence because people around them, family, friends, and colleagues, are lost for words.
"Death is never an easy subject and when a baby dies it is even harder to talk about.
"But finding the right words at the right time can really help to support bereaved parents and families when they need it most.
"Not everyone will be ready to talk about their bereavement – but simply saying, 'I'm so sorry,' for example, can really help."
Dr Harmer added: "For grieving parents, returning to work may be a difficult step but the workplace can be a vital source of support."