He has also collected locks of hair from Confederate Gens. Robert E. Lee and Jeb Stuart, Union Gen. Ulysses Grant's ceremonial sword, a Rebel soldier's bloodstained Bible, bullet-pocked battle flags and an arsenal of weapons on display at his nonprofit museum in White Settlement.
"He has the world's finest Civil War collection. It's truly amazing," says Don Frazier, a history professor and Civil War expert at McMurry University in Abilene. "It's as good as the visitors center at Gettysburg."
Richey, 55, "absolutely hated" history when he was growing up in Wichita Falls, but a visit to the Smithsonian 25 years ago opened a window to the past that he jumped through.
Richey and his wife, Judy, soon started buying antiques to accessorize their home. But when he went to a Civil War show in Virginia to find a musket and sword to decorate his office, he was a "goner."
"I saw all this cool stuff, and I couldn't believe it. It just whacked me upside the head," said Richey, who owns an oil and gas company he started in 1978.
"Fanatical" is how his wife laughingly describes it, but that fervor has resulted in an extraordinary multimillion-dollar collection that the couple shares at the Texas Civil War Museum, a 15,500-square-foot facility that opened in 2006.
A 3,500-square-foot addition will provide space for his cannons and storage for Judy's collection of 300 Victorian-era dresses.
"I was convinced that in order to be with my husband I had to play museum so I got my collection to complement his collection," said Judy Richey, who met Ray at a church camp when she was 16.
The museum, which includes exhibits on loan from the United Daughters of the Confederacy, Texas Division, bills itself as the largest collection of Civil War artifacts west of the Mississippi. Richey's accumulation stands out in another way.
"Most collectors are hoarders," Frazier said. "It tends to be very personal, and they are very reticent about sharing it. Ray puts his out there in the public for educational purposes."
Dennis Lowe, director of Civil War auctions for Dallas-based Heritage Auction Galleries, said Richey "knows what he's doing and he has the wherewithal to do it. It's very unusual. Lots of collectors have more money than they can spend but they don't do their research.
"It's a very treacherous world to be in the high-end collection world. Ray pays top dollar, but he gets top dollar. Everything in that museum is fantastic."
Richey's office, a short walk across a parking lot from the museum, is cluttered with artifacts.
Casual and unassuming, Richey lights up when he unlocks the door to a storage room loaded to the ceiling with boxes of uniforms worn by everything from privates to generals. A plastic jar full of Minie balls sits on the floor a few feet away from a medical corps backpack packed with original bandages.
One shelf holds a dozen revolvers and about 20 swords, including one carried by Gen. Philip Sheridan. There are enough long guns to outfit a small troop.
"Some of these guns might have fired a thousand times. Your imagination goes wild," he said.
But these aren't anonymous weapons. Provenance is everything. The value, he says, is "that this guy carried that gun." Leaning against the walls are some of Richey's 52 Civil War flags, including his latest prized addition -- Jeb Stuart's personal flag that was sewn by the cavalry commander's wife and then partially burned when it fell into a campfire.
He's also got Stuart's sword.
"That's a biggie for me because he carried it every day. It was the sword he loved, given to him by his aide. When you have stuff from this caliber of men, it's pretty special."
There are an estimated 70,000 to 80,000 collectors of Civil War artifacts, and competition is keen.
"It's more difficult now. It's not a treasure hunt anymore," Richey said. "People know what things are worth. It has evolved from Grandpa's cartridge box going for $100 to it selling for $10,000 at auction."
Lowe said Civil War collecting is the most active segment in the country.
"Civil War stuff is all over the place. It keeps turning up," he said. "When you start hitting six figures, it's amazing what comes out of the woodwork."
Richey wants to add more artillery to the armory, but the couple plan to eventually donate their collections to the museum. "I've made a lot of money in the oil business, but except for what supports my family, every penny is here," Ray Richey said.
"I don't buy for investments; I buy it because I love it. This is the history of the greatest nation in history."