Around 37% of all borrowers with 30-year conforming fixed-rate mortgages — who collectively hold about $1.2 trillion of home loans — have mortgage rates of 6% or higher, according to investment bank Credit Suisse. Many could reduce their rates by a full percentage point if they refinanced at current rates, about 5%. More than half could lower their rates nearly three-quarters of a percentage point, according to Credit Suisse.

But new refinance applications in January stood near their lowest levels in the past year. Weekly data compiled by the Mortgage Bankers Association also show that refinance activity has been muted, considering that rates are so low.

“Traditionally, these borrowers would be aggressively refinancing,” said Mahesh Swaminathan, senior mortgage strategist at Credit Suisse.

One indicator of the economic impact of refinancing: Loans that refinanced in 2009 will result in $3.4 billion in savings for consumers this year, according to a report by First American CoreLogic, a research firm based in Santa Ana, Calif. That will return an additional $17.2 billion in savings to borrowers over the next five years. That’s money consumers can potentially use to help spur economic recovery.

About a quarter of all mortgage holders are “underwater” — they owe more on the house than it’s worth — which normally makes it impossible to get refinancing: Banks want collateral to back the value of home loans they make. The Obama administration recently extended a program intended to help underwater homeowners refinance, but few people have tapped it so far. The program has faced logistical hurdles, delays and confusion from brokers and lenders.

Some people are so far underwater, refinancing ends up being out of the question. John Albright, a retired Navy officer in Manassas, Va., hasn’t been able to refinance because the value of his home has plunged. He figures its market value is now around $275,000, but he and his wife still owe more than $500,000 on their mortgage.

Their refinance application was turned down last year because they lacked equity in the home. He says his lender told him he could refinance only if he could come up with about $200,000 to pay down his mortgage. So they are stuck with an interest rate of about 6.5% at a time when his wife’s income has declined. “We’re going from paycheck to paycheck, but what can you do?” Mr. Albright says.

Some mortgage bankers say higher fees by lenders have undermined the effort to encourage refinancing. Fees that Fannie and Freddie began imposing in 2008, as loan delinquencies began to rise, have made it unattractive for some borrowers to refinance. For example, a borrower with 20% down and a 695 credit score seeking to refinance must pay fees equal to 1% of the loan amount. Those fees rise for borrowers with weaker credit scores, higher loan-to-value ratios, or other risk factors.

Overcorrecting for the abuses of financial institutions “has defeated the Fed’s purchase program,” said Alan Boyce, a mortgage-securities-market veteran. Those loan fees, he said, are partly “responsible for why there’s been no refi boom.”

The higher fees and tight credit standards show the tensions facing Fannie and Freddie. As the government-controlled companies try to raise revenue to offset their losses, those efforts can conflict with their basic public-policy mission: to help stabilize the housing market.