The 1868-70 house at Upper Shirley was a secondary residence on the Shirley plantation, seat of the Carter family. Built by Hill Carter for his son William Fitzhugh Carter at a time when few houses were erected in Virginia because of the deprivations of the Civil War, the original portion was constructed by A. H. Marks and Brothers of Petersburg. The walls were laid with bricks taken from a large 18th-century dependency that formed part of the original architectural complex at Shirley. Under the ownership of the Edmund Saunders family, the house was enlarged in 1890-91 to nearly twice its original size and became one of the first homes in rural Charles City County to incorporate modern turn-of-the-century conveniences. Upper Shirley enjoys a romantic setting on a bend of the James River.