Chapel of the Cross, Chapel Hill, North Carolina on Franklin Street, adjacent to the university campus

Mary Ruffin Smith took charge of her brothers' mixed-race children by Harriet. On Sundays she brought them to church at Chapel Hill's Chapel of the Cross, located in the heart of the village. The mulatto slaves sat in the balcony while their white relatives worshipped downstairs. Cornelia, the oldest, was confirmed at the age of twelve at the same time as the daughter of university president David Swain. Neither Sidney nor Francis Smith married and both died before their sister. At her death in 1885, Mary Ruffin Smith left the family plantation to the university, which established a scholarship fund named in honor of her brother Francis.