Fahey, 59, told the court he has not had enough time to tear down the buildings and he doesn't have a crane to do it safely but that he has removed about 70 doors and windows. He also said he's not an electrician and doesn't know how to disconnect the power.
People living in three of the structures were told to move out as the judge ordered, he said, but "they come back when I'm asleep. What can I do? They have nowhere to go."
Fahey was convicted of a dozen misdemeanor building code infractions last month. His sentencing was rescheduled to July 22, and could face several years in prison unless he demolishes his buildings. If he complies, he should get off with no more than a fine and community service, said his attorney Jerry Lennon.
Fahey began building the village when his children were born -- 10 in all -- and needed more room.
Among the 30-year phone company technician's structures is a replica of a 16th century Viking house that he built for one of his daughters and a mobile home remodeled to look like an antique railroad car.
Then came the centerpiece -- a 70-foot tower with stained-glass windows and stunning views of the San Gabriel Mountains.
Fahey built it all over 25 years while ignoring orders from Los Angeles County to get permits. In addition to his conviction last month, he was ordered to tear down Phonehenge West.
He has said he will file an appeal as soon as he's sentenced. If the appeal fails, county officials have the authority to tear down Phonehenge themselves.
"They're going to find out that Faheys don't fold," the self-taught builder growled as he took visitors on a recent tour of the property.
Fahey said his constitutional right to do what he wants on his property is being trampled, calling that the real issue in his fight with the county -- not his colorful personality or his penchant for building odd-looking buildings.
Fahey built Phonehenge with just about anything he could find. He used steel beams foraged from an old car wash and an ocean pier. He traded with utility yard supervisors for poles they didn't need, picked up discarded doors and windows from movie studio sets, and bartered with home builders who overbought supplies.
With his razor-sharp wit and nonstop patter -- "You've heard me, I can talk all day," he quips -- he's sometimes been compared to Santa Claus for his bulky build, flowing white hair, full beard and jolly demeanor.
It's a comparison Fahey detests, although it might be accurate if Santa was a burly biker who rode a Harley, dressed in denim coveralls, had arms the size of tree trunks and sometimes wore a black T-shirt with the words, "Acton -- Where Men Are Men and So Are the Women."
Fahey said he is not the only person in Acton -- a rural, sparsely populated area 50 miles north of Los Angeles where the gold mines petered out in the 1800s -- who is being forced out by increasingly stringent enforcement of building codes, but he is the most vocal.
The district attorney's office has asked county officials not to comment on the case until Fahey is sentenced, said Tony Bell, a spokesman for county Supervisor Mike Antonovich, who represents the area. Deputy District Attorney Patrick David Campbell, who prosecuted Fahey, did not return calls for comment.
During the trial, Campbell told jurors the Phonehenge buildings are unsafe, and Fahey didn't get permits because he considered himself above the law.
"I tried to comply initially," Fahey said as he watered his rosebushes. Nearby, chickens, turkeys and peacocks clucked loudly and chased each other on the 90-degree morning.
Fahey said he got a permit in 1984, when he began his first building project, an addition to his 1930s ranch house. He immediately began to argue with county officials over their demand for blueprints.
That was difficult to do, he said, for a self-taught builder whose creations came from a mixture of spontaneous inspiration and whatever materials he could scrounge. Still, when he began work on a barn in the 1990s, Fahey said, he got the blueprints the county wanted, only to have officials lose them.
"You know what they told me? `Too bad,"' he said. "So I built this, and I kept working on stuff, and then 12 years went by."
His tower, originally planned to be 120 feet tall, was going up when county officials arrived before dawn one day and ordered him to stop or go to jail. It launched his five-year legal odyssey that has resulted in more than 50 court appearances.
Lennon, his former attorney, said county officials explained that they left Fahey alone for 12 years before someone complained while he was building the tower.
Fahey doesn't believe the bureaucrats, and neighbors up and down his rural road swear they didn't complain.
"I absolutely love it, it's really creative," said his closest neighbor, Jennifer Swenson, whose family has lived next door for 22 years.
The tower is Fahey's pride and joy. He and the structure steadily gained notoriety as supporters compared it to folk artist Simon Rodia's esoteric Watts Towers in Los Angeles.
Tourists began dropping by; he spoke at the local Rotary Club; and Phonehenge was featured in news stories and on websites. Its very own Facebook page has collected nearly 30,000 followers.
The tower was even the site of a recent Glamour magazine photo shoot.
Some supporters have said they are willing to lie down in front of the bulldozers if Fahey loses his appeal, but he is having none of that.
"No, hell no," he said. "I don't want anybody getting hurt on my property."
If he does lose his appeal, he said he won't wait for the county to send a crew out to tear down Phonehenge, Instead, he vowed to dismantle it himself and start rebuilding it someplace else, perhaps in rural Nevada or Colorado, where some of his children now live.
"It sure won't be in California," he said with a hearty laugh. "I can tell you that."