Male infertility care can be insensitive and one-sided, says charity Fertility Network UK which has surveyed men seeking help to become fathers.
Men can feel excluded, with female partners being the main focus of attention in clinics, it found.
The charity says men's needs are too often ignored, which must change.
Gareth Down, 31, set up a support group for men with fertility problems because he says he had nowhere to turn for help when he needed it.
Men's feelings
At the age of 20, in a blunt conversation with his GP, he was told tests showed he had no sperm.
After eight years, nine cycles of treatment and four miscarriages, he and his wife Nat finally had a baby 17 months ago.
He said a men-only space for sharing experiences was important, and would have made him feel much better about his personal situation.
"I just wanted to rant sometimes, but I didn't feel my thoughts were valid.
"Women are the ones having to go through it all and I felt I didn't have a right to talk or complain.
"But now I can say in confidence what other men feel.
"It just makes you feel less alone."
Gareth says the long, agonising wait for a baby "nearly broke us many times over".
He says male fertility should be treated like any other medical condition.
"It's still quite a taboo subject – made even worse when you're made to feel like you're wasting NHS time and resources."
Overlooked
Fertility Network UK says very little is known about how men cope with infertility – men are often reluctant to share their experiences.
Forty-one men responded to its online questionnaire, designed with researchers from Leeds Beckett University.
On average, men who responded had been trying for a baby for five years and most had suffered directly from male infertility.
Nearly all the men said fertility problems had affected their wellbeing, with many saying they felt worthless or "less of a man".
Many felt excluded or marginalised.
One man said: "The whole experience has been focused towards my wife… even consultants' letters about my genitalia are addressed to my wife. There seems to be no equality."
Another said: "I now know what it feels like to be identified on official documentation… as an appendage to one's spouse."
Some said they were treated insensitively: "A very rude GP… then on the analysis result, rather than explaining the result, called me at work to ask if I had had a vasectomy."
A lack of emotional support for men going through treatment for infertility was another common complaint.
"There are always women in the media admitting that they have had infertility issues but we do rarely hear about the man. I know it is a big issue for men and the risk of being called 'less of a man' but I feel infertility is something that many men suffer in silence, fearful of being called hurtful names. It is a silent problem that many men suffer," said one of the men who responded to the anonymous survey.
Susan Seenan, chief executive of Fertility Network UK, urged men to speak out and said clinics should pay extra attention to the care needs of the men they see and treat.
"Men are half of the fertility equation; when they cannot create the family they long for without medical help they suffer and struggle physically and mentally just as women do."
