The two brothers of the man accused of holding three women captive for about a decade say they have no sympathy for him. One called him a "monster" who he hopes "rots in jail."

Onil and Pedro Castro told CNN that they want the freed women to know how sorry they are for their ordeal.

"I'm just grateful they are home and out of that horrible house, and I'd just tell them I'm sorry for what Ariel done," said Pedro Castro, 50.

The brothers were initially taken into custody but released after investigators said there was no evidence against them. Brother Ariel Castro has been charged with rape and kidnapping and is being held on $8 million bond.

Amanda Berry, Gina DeJesus and Michelle Knight have all been reunited with their families.

Pedro Castro says he was shocked to learn DeJesus was a victim, because they'd known her father for a long time, and Ariel even went to a vigil for her when she went missing.

"You go around like it was nothing? You even went to the vigil. You had posters. You give his momma a hug and you got his daughter captive?" Pedro recalled saying to Ariel.

Pedro Castro said he had been in the house but never noticed anything amiss. He said part of house was blocked off by curtains, and a radio or TV was always on.

Onil Castro, 50, said he wanted Ariel to "suffer in that jail."

"I don't care if they even feed him," Onil Castro said. "The monster's a goner."

The brothers said they are worried that people will continue to associate them with their brother's alleged crimes, even though police say they didn't know anything about it. They have received death threats since being released from custody.

"This has torn my heart apart," Onil Castro said. "This has killed me. I'm a walking corpse right now."

"I hope the world listens to us," Pedro Castro said. "You already got your monster, please give us our freedom."

An imposing, 10-foot privacy fence will soon guard the Ariel Castro house, with windows and doors boarded shut to keep people out of the place that police say was once meant only to keep people in.

The run-down house has become a two-storey piece of evidence in the abduction and imprisonment case of three women, but neighbors who remain shaken by the horrors alleged inside want it torn down and erased from the landscape of Seymour Avenue.  

Not simple issue

The house and what becomes of it will be a daily talking point for the Seymour community, as city officials deal with the irony of keeping the dreaded site of the women's imprisonment safe while neighbours almost uniformly want it torn down.

The issue isn't a simple one.

First and foremost, it's evidence against Castro, who investigators say kept the women in chains in a basement before eventually allowing them to live under close control upstairs. The 6-year-old daughter of one victim, Amanda Berry, was also freed; DNA tests showed Castro was her father, a dark twist on years of captivity during which Castro is also alleged to have induced multiple miscarriages in one of the women by repeatedly punching her belly.

The nondescript white house with a red-and-white tile roof sits on a street of other boarded-up houses, victims of the foreclosure crisis which hit the city hard. The house has thousands of dollars in unpaid tax liens, which would have to be sorted out as the city attempts to control the property. County records show it was built in 1890 and updated in 1950. Forty-two years later, Castro bought it for $12,000.

Preserving evidence

Workers over the weekend began boarding up windows and doors and erecting a metal fence around the house.  

The plywood and fence have a two-fold purpose, said Councilman Brian Cummins: preserving the scene as evidence and protecting it from the threats already circulating on the streets to burn it down in a stroke of vigilante justice.

It's a decision for neighbours and also for the women, said Cummins, whose ward encompasses the property and who is in close contact with police and city officials about the situation.

"The issue is how do we respect the wishes of the survivors in this case and it's too premature to know what their wishes would be," Cummins said Saturday.

Both prosecutors and the defence will want Castro's home still standing until the trial ends, said Michael Benza, criminal law professor at Cleveland's Case Western Reserve University.

"The prosecutors are going to want to preserve it so they can take jurors into it to view, and the defence would want it preserved so at least they could do their own investigation," Benza said.

It's unlikely the house would be needed once the trial ends; typically only evidence like weapons or fingerprints are preserved for appeals, he said.