Potter is the only daughter of Philip B.K. Potter (1927-1975), an American diplomat who served in the OSS, and his wife, the former Madeleine Mulqueen Daly (1921-1985). She is also a great-great-granddaughter of Episcopal bishop Alonzo Potter and a great-grand-niece of Episcopal bishop Henry Codman Potter.
She was married to Patrick Fitzgerald, an Irish-born American actor, whom she wed in 1990.
Potter's only child, Madeleine Daly (born June 4,1995), appeared as her character's niece in the 2005 movie The White Countess.
