Brown Ayres was born on May 25, 1856 to Samuel Warren and Elizabeth Little (Cook) Ayres in Memphis, Tennessee. He earned his BS (1878) and MS (1888) degrees in Engineering from the Stevens Institute of Technology in New Jersey. In 1880, he became a professor of physics and electrical engineering at Tulane University in Louisiana. He married Katie Allen Anderson on July 5, 1881 and the couple had eight children: Martha Garland (Ayres) Newman (1883-1982), Samuel Warren (1886-1915), John Anderson (1890-1969), Elizabeth Cook (Ayres) Link Ayres May (1893-1980), Mary Douglas (Ayres) Ewell (1895-1999), Katherine Stewart (Ayres) Watson (1897-1990), Ruth Virginia (1899-1987), and Morgan Brown (1902-1981). Ayres rose rapidly through the ranks at Tulane, where he remained until accepting the presidency of the University of Tennessee in 1905.

As president, Ayres made a number of notable improvements to the University. He increased the number of faculty and students, made the College of Liberal Arts equal in stature to the Colleges of Agriculture and Engineering, enlarged the College of Agriculture, elevated the departments of Law, Medicine, and Dentistry to Colleges, constructed a separate library, stiffened academic requirements (resulting in recognition by the American Association of Universities), reorganized the Board of Trustees, established scholarships to attract students from outside the Knoxville area, merged the Medical and Dental Colleges with the University, and made the Summer School of the South a regular part of the University program. Ayres died on January 28, 1919 in Knoxville, Tennessee.