The Pryors and Taylors came to Sumner County from Campbell County, Virginia. Massa Taylor married John Pryor in 1812. Earlier her sister, Spicey, married a William Pryor. Several families traveled from Virginia to Tennessee with Elijah Garrett. Some of the families settled in Overton County. John Pryor was in White County prior to 1830. At the time of the 1830 census, John Pryor and his brother-in-law David Taylor are found living near each other in Sumner County.

John and "Massey" Pryor had seven children. Allen L., their oldest, stated that he was born in White County in 1816. Edward b.1825 was killed in the Battle of Monterey during the Mexican War and his name is inscribed on the war monument in Gallatin cemetery. Their brother Alfred was a blacksmith and died of Typhoid in 1850. The other children, Elizabeth, b.1822, Jonathan b.1824, George b.1826, Alfred b.1828, and Samuel b.1835 are found on the 1850 census of Sumner County, as well as their mother.

Allen L. Pryor began working for himself at the spinning machine trade in about 1838 in Sumner County. His father owned land near the old railway tunnel and Allen bought land near the Rockbridge area. In 1845 he married Elizabeth Talley, the daughter of his neighbor William Anderson Talley and Polly Dowdy, formerly of Cumberland County, Virginia. Allen and Elizabeth Pryor had eleven children, many who married into local families. Their sons, William and "Curg", moved from Sumner County to Texas at the turn of the century. Another son, John Pryor, was a school teacher in Sumner County.

After the death of his wife Elizabeth, Allen L. Pryor remarried to Margaret McWhirter who was born in 1836 in Hardeman County, TN to Isaac and Adeline McWhirter. "Maggie" McWhirter Pryor died in October 1898 and within the month Allen Pryor had written his will. However, Allen Pryor lived out his natural life (he died in 1904) on the family farm with his daughter Bettie and her husband Thomas Washington Gregory. Allen Pryor deeded the farm to Bettie and Thomas Gregory. There is a family burial plot on the land in Sumner County where Allen Pryor is buried. In later years, his descendants in Sumner County have put up a marker for Allen Pyror and Elizabeth Talley Pryor, but many of the other old markers in the plot are not readable.

Thomas Washington Gregory was from Cairo in Wilson County, TN and appears to be related to one of the Gregorys in Sumner County; his uncle William C. Gregory b.1821 and married to Lucinda P. McAnnis. Although from Wilson County, there are many marriages for his Gregory and Donnell relations recorded in Sumner County. Thomas and Bettie Gregory had four children. After Bettie's death in 1909, Thomas remarried to "Cousin" Willie Ann Pryor, the daughter of Thomas Jefferson Pryor and Pamelia Ann Talley (an aunt of Bettie Pryor Gregory). Thomas Pryor was the son of William Pryor and Margaret Curry Pryor Hunter.