Companies love to tout their generous maternity leaves and flexible work arrangements as evidence they are supporting their female employees. They promote perks such as company-sponsored child care and elder care benefits to show they’re committed to helping women balance their lives – and therefore, to helping women stick around the workplace and move up in their careers.

But are such programs successful at better retaining and advancing women? The answer, according to a new research report, appears to be “no.” In some cases, these programs are slowing women’s ascent to the executive ranks.

In a study released Thursday, the consulting firm Mercer surveyed 164 organizations about their benefits and human resources practices, such as whether they offer paternity leave and whether they consider diversity goals when paying executive bonuses. Then, the study compared those practices against actual outcomes for female employees.

Each of the organizations not only answered a questionnaire about what it offers, but also provided Mercer access to its internal employee data. This allowed the consulting firm to see how many men and women were at each level of the company and then project – based on hiring, promotion and exit patterns – how many women would progress into the executive ranks.

When looking specifically at flexible work arrangements and maternity leave benefits, the research found that organizations offering them had more female employees. Yet there was another correlation: Those companies also appeared to promote women into their executive ranks at a slower pace.

How can that be? Aren’t flexible work arrangements and long, paid maternity leaves manna for working women, keeping them in the workforce through their child-rearing years?

It turns out that the problem is not with these critically important benefits themselves, but that companies can be lulled into complacency, thinking that offering them alone is good enough.

“It’s very easy to check the box,” says Pat Milligan, president of Mercer’s North America region.

When companies institute such benefits but don’t follow them up with the proactive management that helps women get back on track for promotion, the initiatives can actually have a negative effect.

“We’re not saying they’re not good programs,” Milligan says, but they can lead to a mentality where companies think they’ve done their job just by implementing them, conflating H.R. perks with real action. “If I’m not actively managing and really coaching these women, they can have very unintended consequences.”

Unconscious bias against, and stereotypes of, working mothers could also play a role. All too often, there’s a stigma attached to taking advantage of flexible work arrangements or long maternity leaves. They’re formally offered but informally frowned upon.

The study found that when these benefits were ranked by an organization as one of its top five programs for developing women – meaning the culture values them – the negative effect goes away. And when they were used by both men and women in similar numbers, they could even be linked to improved projections for women joining the top ranks.

“You can have a great employee handbook, and all these great programs that say you can do this, but if your manager makes it clear that your butt needs to be in the seat, then what’s the sense of having them?” Milligan says.

“We’re just honestly saying it takes a lot more work to use this stuff for the good than simply to offer it.”