The lawsuit, filed by Kathrine McKee, revolved around a letter that an attorney for Cosby sent New York's Daily News in 2014 as a wave of women was coming forward to accuse the comedian of a string of sexual assaults dating back to the 1960s.
The statute of limitations on the alleged crimes had long expired, leading some accusers to pursue civil lawsuits, such as McKee's. The lawsuits and accusations by dozens of women shattered the family-friendly reputation Cosby built in a career highlighted by his role in the 1980s television hit The Cosby Show.
Cosby, 80, has denied wrongdoing, saying any encounters with his accusers were consensual. He is awaiting an April retrial in Pennsylvania on charges he sexually assaulted a former basketball coach at his alma mater, Temple University.
McKee argued in her suit that the attorney's letter to the newspaper called her a liar by saying the article was "defamatory, characterizing her claims as "wild" and suggesting she had a criminal record.
But the 1st U.S. Circuit Court of Appeals in Boston stood by a lower-court ruling that the entertainer could not be sued over the letter.
The decision found that McKee had made herself a public figure by wading into the controversy with Cosby, rejecting her claim that her dispute with the entertainer was a private one.
As a public figure, McKee would have to prove that Cosby acted with malice in his response.
"The web of sexual assault allegations implicating Cosby, and internationally renowned comedian commonly referred to as 'America's Dad,' constitutes a public controversy," U.S. Circuit Judge Sandra Lynch wrote for the three-judge panel.
William Salo, McKee's attorney, said he disagreed with the decision and may appeal.
"They're saying just because a famous person rapes you, you become a public figure if you complain about it," he said.
Alan Greenberg, a lawyer for Cosby, welcomed the "well-reasoned decision confirming that there was no defamation."
McKee sued Cosby in 2015, a year after the Nevada resident told the newspaper he raped her in a Detroit hotel room in 1974.