Revenue Minister Gordon O'Connor said Tuesday his department will consider delisting the Banyan Tree charitable foundation after the Canada Revenue Agency called it a sham.

O'Connor said the government could remove the organization from the CRA's list of Canadian registered charities, meaning donors can't claim tax credits for their contributions.

A CBC News investigation found more than 20 charities that were promised tens of millions of dollars by Banyan Tree didn't get their money this year. As well, thousands of people who donated to the organization have been told by the government to repay more than $63 million in tax credits.

Under Banyan Tree's complicated structure, donations were made through cash contributions and loans provided by a company owned by Banyan Tree's owner. Donors would then receive a tax credit worth more than their donation.

O'Connor said donors must be more cautious.

"You invest $100 in a charity and you get a tax return of $300 or $400 dollars — there's something wrong!" he said.

"If something looks too good, it's too good. I mean that's just your basic rule your mother gave you."

Greg Madden borrowed heavily to make donations to The Banyan Tree Foundation, giving roughly $14,000 of his own money over a two-year period, plus about $50,000 he got from a loan.

Each time, the Canada Revenue Agency processed Madden's returns and he received a tax break that was bigger than his own personal donation.

But now, just like roughly 3,400 other Canadians, Madden has been asked to pay back all of his Banyan Tree tax credits.

"My family members had been involved in this previously and that Revenue Canada had approved their tax returns made me believe that it was real," he said.

"And to find out it was a sham. It's a shock you know, you're taken aback and wonder where are the checks and balances to stop this from happening to people."

Letters of reassessment from the CRA were sent to donors as early as last year.

Opposition politicians questioned why the organization hasn't already been delisted.

"Why is Banyan still listed as a registered charity? Why was the public not informed if a review was ongoing?" said Liberal MP Susan Kadis. "Clearly the left hand does not know what the right hand is doing here."

NDP MP David Christopherson said the government has a responsibility to inform the public.

"Put down that you've either got a problem with this, it's under review, pull it off the page, do something! But don't just leave it there in a way and in a fashion that says to the ordinary person 'here's a bona fide charity, go for it!'" he said.