This collection is divided into three series: Series I: James Agee and Family Papers, circa 1920-1960; Series II: The James Agee Trust Papers, circa 1920-2003, Series III: Oversize materials, 1930s-2000s.

The first series, James Agee and Family Papers, is divided into six sub-series: Sub-Series A: Correspondence; Sub-Series B: Financial and Legal Documents; Sub-Series C: Clippings; Sub-Series D: Pictures; Sub-Series E: Writings; and Sub-Series F: Other. In the Correspondence sub-series (I. A.), the original order has been maintained. Correspondents without a last known name are in a folder marked "No last name", and some may be organized by several last names (e.g., James' sister Emma is under A for Ms. Agee and L for Mrs. Ling).

The Writings sub-series (I. E.) contains examples of Agee's prose, including autobiographical material, drafts of Let Us Now Praise Famous Men, and movie reviews; Agee's poetry, such as Permit Me Voyage material and Sunday: Outskirts of Knoxville, Tenn. drafts; and the writings of others, including Witness by Whitaker Chambers. These writings by others were likely presented to Agee or one of his wives.

The bulk of the material related to the James Agee Trust is currently restricted. The only files available to researchers are located in Sub-Series A: James Agee Files, 1920s-2000s. This sub-series contains photocopies of Agee's letters and works, pamphlets from St. Andrew's School in 2001-02, as well as copies of the University of Tennessee Library Development Review with articles on Agee.