In her 43-page ruling on the case, Judge Anna Diggs Taylor in Detroit said that the surveillance programme violated protections on free speech and privacy.
"Plaintiffs have prevailed, and the public interest is clear, in this matter. It is the upholding of our constitution," Judge Taylor wrote.
Stinging rebuke
White House spokesman Tony Snow said: "We couldn't disagree more with this ruling."
The programme is "firmly grounded in law and regularly reviewed to make sure steps are taken to protect civil liberties," he said in a statement.
He said the Justice Department had launched an appeal and a hearing was set for 7 September. The judge's ruling is on hold while the appeals process is under way.
"We're going to do everything we can do in the courts to allow this programme to continue," Attorney General Alberto Gonzales told a news conference.
The judge's ruling is a stinging rebuke to President George W Bush over what he has called a vital tool in the war on terror, says the BBC's James Westhead, in Washington.
Mr Bush authorised the Terrorist Surveillance Programme, as the secret interception scheme is known, after the 9/11terror attacks.
But after the programme was uncovered by the media a year ago, the American Civil Liberties Union (ACLU) filed a lawsuit arguing that the secret interception of US phone calls was unconstitutional.
The ACLU welcomed the judge's ruling. "It's a sound rejection and rebuttal to the Bush administration argument on the war on terror," said executive director Anthony Romero.
The ruling is a further set back for the president's self-proclaimed wartime powers, our correspondent says.
He has already been rebuked by the US Supreme Court over his plans to try suspects being held in Guantanamo Bay. The Supreme Court stated that the president did not have a blank cheque.