LeRoy Philip Graf was born in Fremont, Ohio, to John Charles and Rose Hammel Graf on March 17, 1915. He completed his BA degree at Oberlin College in 1936 and continued on to earn both his MA (1937) and PhD (1942) in history at Harvard College.

After teaching briefly at Tufts and Ohio State University, Graf joined the faculty of the University of Tennessee's history department in 1945. He quickly proved to be a talented and dedicated teacher. He became a full professor in 1950 and was named department head in 1965, a position he held until 1980. Even after his official retirement in 1985, Graf continued to mentor students on a volunteer basis. Graf also wrote prolifically and edited seven volumes of The Papers of Andrew Johnson with his friend and colleague Ralph Haskins.

Graf was also active in the Knoxville community thoughout his life. Dedicated to the causes of civil rights and interracial and religious cooperation, he was active in the efforts to desegregate Knoxville's restaurants in the 1950s and 1960s, the Knoxville chapter of the National Conference of Christians and Jews, and the Beck Cultural Center. Graf was also a patron of the arts, and indeed performed in a number of plays, dances, and musical events.

LeRoy Graf died on May 25, 1993. He was survived by his wife, Ruth Adena Peal Graf, and his three children: Christina Graf, Melissa Graf-Evans, and Jeremy Graf.