Outcomes Based Evaluation

Definition: Outcomes Based Evaluation is an approach to measuring the effects of a project or an institution's services and activities on the target audience that these programs seek to benefit or serve. IMLS recommends outcome based evaluations as a means of tailoring each project's design and objectives to achieve their specific goals. Through the use of outcomes- based statements, methods of evaluation, inputs and outputs, projects such as PBO can both monitor and adjust its progress towards completing its program purpose and organizational mission.

Purpose & Mission

• Project Purpose: Publishers' Bindings Online (PBO) is the result of The University of Alabama Libraries (UA) in cooperation with the University of Wisconsin Libraries (UW) working together through the use of a National Leadership Grant to create an online database of book bindings from 1815-1930 in order to increase public awareness about the role of “medium-rare” books as “common” objects, to expand an understanding of how the mass production and publishing of books from 1815-1930 shaped America’s relationship to and understanding of the book as object, to provide a digital model for other repositories, and to offer access to students, educators, and scholars about the historical, cultural, artistic, aesthetic, and literary relevance of the “medium-rare” books.



• Organizational Mission: PBO seeks to address a lack of awareness and understanding about the significance of “medium-rare” books by making digital images of the selected bindings accessible to a wider public through the creation of an Online Database. PBO will serve as a model for digital collaboration in the creation of databases. Also PBO seeks to create a standard template for describing the elements involved in book bindings and binding designs. Furthermore, PBO intends to facilitate these goals through the inclusion of a variety of value added materials such as a concise bibliography, a gallery of relevant biographical/historical essays, a comprehensive glossary of related terms, a controlled vocabulary search, sample lesson plans, and web resources to aid in further research.

Audience & Targets for Change

• Target Audience : PBO's academic audience include K-12 educators and students, as well as scholars, educators, and students of advanced academic disciplines/fields such as American Studies, Applied Arts, Archives, Art History, Bibliography, Book Arts, Cultural Studies, Gender Studies, History, Industrial Art and Design, Library Science, Literature, Special Collections, Textiles, and Women's Studies.
  
  Additionally, PBO also provides a resource for practitioners/professionals in the following fields: Antiquarian Book Collectors and Dealers, Book Binders, Book Designers, Center for the Book Members, Conservators, Industrial Arts, Librarians and Descriptive Bibliographers, Museum Educators and Curators, and Special Collections Professionals and Curators.
  
  PBO’s target audience is widely varied due to the length of the historical period PBO addresses, the range of supplementary materials available online at the PBO site, and the stated intention that National Leadership Grant recipients are to provide a model for other repositories in similar fields. PBO’s varied audience has been and continues to be an asset which aids in furthering the project’s goal of promoting a greater awareness of the significance “medium-rare” books play in the development of history and culture.



• Targets for Change : PBO will provide an adaptive template using The Qualified Dublin Core Metadata Initiative for physical, visual, and iconographic descriptions of binding materials and design elements pertinent to the project; create a leadership model for collaboration on digital projects between geographically separate archives or special collections; and increase public exposure to the “medium-rare” book.

Outcome Statements

1. Scholars and students will develop a greater appreciation of the role of book production in American History and Culture from 1815-1930.
   
   
2. K-12 Educators will have access to a variety of individually adaptable educational resources such as PBO's research and teaching tools.
   
   
3. K-12 students will gain an awareness of the notion of “medium-rare book” and the historical period 1815-1930 from biographical/historical information provided in the Galleries/Essays section.
   
   
4. Visitors to the PBO site will increase their awareness of mechanized book production from 1815-1930.
   
   
5. Visitors to the PBO site will gain a convenient means to access to over 5,000 titles previously available to the public only through on campus visits to these respective special collections.
   
   
6. Visitors to the PBO site will increase their awareness of 19th and early 20th century book production and the artistic movements reflected in the bindings themselves.
   
   
7. Visitors to the PBO site will increase their understanding of publishers' bindings as both historical and art objects.
   
   
8. Cultural Heritage Professionals (Archivists, Curators and Librarians) will have access to a model for the metadata creation and database development.
   
   
9. Archivists and Librarians will be provided with a model for creating a shared database through digital collaboration from disparate collections and/or remote locations.
   
   
10. Collections Management will be provided with a model for developing policies for the categorization and designation of medium-rare books.
    
    
11. Scholars will gain a greater understanding of the role of artistic trends in the mass production and marketing of books from 1815-1930, the first era of mechanized production.
    
    
12. Book Arts Scholars and Students will have access to digital example of publishers' bindings from the first era of mechanized book production in America.

Inputs & Outputs

Inputs:

1. Include up to 5,000 items that represent 19th century trade bindings.

Projected intake:

• 2500 items from The University of Alabama Libraries (UA)
• 1660 items from UW-Madison General Library System (UW)

2. Provide up to 10,000 images of 19th century trade bindings including spines, covers, and end papers.

3. Create a controlled vocabulary describing the representative trade bindings selected.

4. Develop a comprehensive online glossary which includes images of visual images and techniques as well as biographical/historical information on binders, designers, and printers.

5. . Design a user-friendly, easy to use educational resource accessible via the World Wide Web that will meet the diverse needs of PBO's target audience.

Outputs:

1. Total Number of Items in Database: 4864

• 2501 W. S. Hoole Special Collections Library (UA)
• 2363 items from UW-Madison (UW)

2. Total Number of Entries in Bibliography: 193

• 145 Print Resources: Books/Articles
• 48 Online Resources: Articles/ Web sites

3. Glossary: 456 terms listed

4. Digital Images: 10,570 from 4864 individual books

5. Value Added Components:

• Glossary
• Controlled Vocabulary
• Essays/Galleries on Historical/Cultural/Literary themes
• Essays/Galleries on Specific Binding Designers
• Essays/Galleries on book-as-object – techniques and physicality

6. Research Tools:

• Bibliography of Print and Online Resources
• Controlled Vocabulary Search
• UW Madison's Scandinavian-American Publisher's Holdings, 1840-1920 (117 page pdf document)
• Publisher's Map (Java) with lists of all publishers in project (total of 2132 publishers)

7. Teaching Tools:

• Tutorials for Searching Database
• Galleries (26 to date)

• Abraham Lincoln, Civil War President
• Black and White: Paul Laurence Dunbar and Race in Post-Civil War Literature
• Booker T. Washington: Author, Educator, Advocate
• Confederate Imprints: Publishing in the Civil War South
• From Domestic Goddesses to Suffragists: The Story of Women Told on Bookbindings, 1820-1920
• Founded in Handcraft: The Roots of Publishers' Bindings
• Gallery of Book-Cloth Grain Patterns
• Genesis & Apocalypse of the "Old South"Myth:
  Two Virginia Writers at the Turn of the Century; Part I: Thomas Nelson Page's Literature of the "Lost Cause"
• Genesis & Apocalypse of the "Old South" Myth: Two Virginia Writers at the Turn of the Century; Part II: Ellen Glasgow' s Feminist Approach to the "Old South"
• Heroes of the “Lost Cause”
• Indians, the Frontier, and the West in American Bookbindings
• Lafcadio Hearn: 19th Century World Citizen
• Louisa May Alcott: Little Women, Big Pen
• “Moonlight and Magnolias:” History and Literature in the Big Easy
• “The Most Written About War in History:” The Civil War in Fact and Fiction
• Progress & Invention: Representations of the Industrial Revolution in America as Reflected in Bookbindings
• Publishers' Bindings & Their Designers: Making Their Mark
• Publishers' Bindings from the Max Kade Institute
• Publishers' Bindings from the Wade Hall Collection
• Silver & Gold: The Art of Metal Stamping
• Son of a Comet, Star of the West: The Life and Literature of Mark Twain
• True Tales of Bondage and Freedom: Nineteenth Century Slaves Narratives
  

8. Lesson Plans & Teachers' Resources Listed by Subject:

• American West

• Myths of the Wild West
• Guidelines for Book Reports Handout
• Wild West Book List

• Civil War

• Civil War Literature Lesson Plan
• Guidelines for Book Reports Handout
• Civil War Book List

• Industrial Revolution
• Literary History
• Uncle Tom's Cabin

• Uncle Tom's Cabin K-4 Lesson Plans
• Uncle Tom's Cabin 5-8 Lesson Plans
• Uncle Tom's Cabin 9-12 Lesson Plans
• Uncle Tom's Cabin 9-12 Handout

• Louisa May Alcott

• Louisa May Alcott K-5 Lesson Plan
• Louisa May Alcott 6-8 Lesson Plan
• Louisa May Alcott 9-12 Lesson Plan

• Mark Twain

• Mark Twain K-5 Lesson Plan
• Mark Twain 6-12 Lesson Plan
• Mark Twain Handout

• Paul Laurence Dunbar

• Paul Laurence Dunbar 6-12 Lesson Plan
• Dialect Poems of Dunbar Handout

• Slave Narratives

• Slave Narratives Lesson Plan
• Slave Narrative Book Report Handout

• Women's History

• Women's History K-5 Lesson Plan
• Women's History 6-12 Lesson Plan
• Women Writers of the 19th Century Lesson Plan

Outcome Based Evaluations Methods

I. Online Survey:

• Data Source :

Online site survey available at bindings.lib.ua.edu

• Applied to :

All visitors to the PBO site are asked to participate in survey.

• Data Interval :

Collation of the data gathered from the online survey will be made after July 2006 and the results will be posted online at the PBO web site.

• Indicator :

The precise # and % needed for the Indicator can not be determined until all the data from the online survey has been collected. Due to the vast nature of PBO's target audience a precise number or percent can not be forecasted.

• Goal :

The data collected from the online survey will aide in making the PBO web site increasingly more accessible and useful for our target audience.

Additionally, the survey will remain posted online for the duration of the PBO project in order to continue refining the web site. In future the online survey might be modified to reflect how the changes made to the PBO site increase its effectiveness.

• Written Comments from Online Survey:

II. Control Groups:

• Data Source:

A survey given to three separate control groups that involved an introduction to the PBO site, PBO site searches, level of PBO user-friendliness, and general feedback/comments about PBO site.

• Applied To: Control Groups and Testing Locations

  Group I: History Educators K-12, Tuscaloosa City Schools
  Testing Location: Bryant High School , Tuscaloosa , AL

  Group II: Master of Fine Arts in Book Arts Students (UA)
  Testing Location: Gorgas Library, UA Campus

  Group III: Undergraduate Advanced Creative Writing Students (UA)
  Testing Location: Hoole Special Collections Library, UA Campus

• Data Interval & Participant Numbers:
  
  • Group I: 17 March 2005; 25 Participants
  • Group II: 21 March 2005; 15 Participants
  • Group III: April 2005; 20 Participants

• Indicator:

Participants in Three Control Group and 75% of 60 who responded favorably/positively to PBO site study and said they were able to navigate the site with relative ease.

• Goals:

To test the PBO's Web site for navigability, user friendliness, aesthetic presentation, and usefulness of information.

To achieve a 60%, or higher, positive response to the Total Control Group Participants experience during their search of the PBO site.

• Written Comments: Participant Responses to the Survey
  
  • Group I:
    • “Teacher resources and Research tools seem very useful to student/teacher projects. Homepage would be appealing to my 5th grade students.”
    • “Controlled vocabulary for subject headings was a very helpful way to search for topics, but some headings were confusing.”
  • Group II:
    • “Site map - do you have one, couldn't find it, would really help!”
    • “PBO is a great resource for book research, not only in relation to bindings. Can search by keyword, subject of bindings; great description of bindings!"
  • Group III:
    • “Beautiful book covers; must have taken a long time to organize!”
    • “I could use this to talk about how a book was marketed when it came out.”
    • “Very attractive, straightforward site. Nothing to dislike.”

Interpretations:

Publishers' Bindings Online has produced some interesting outcomes and impacts. Many of PBO's impacts reflect goals established in its Organizational Mission and Targets for Change. The series of impacts listed below is not definitive. As PBO continues to grow and develop its impact will expand.

Impacts:

Collaborative Projects: PBO's initial challenge was to organize and coordinate methods of communication between two disparate repositories and collections. The communication between the two project sites was conducted mainly through phone calls and emails. The successful completion of PBO will provide a model for digital collaboration by other collections separated by geographic distance.

Controlled Vocabulary: PBO's controlled vocabulary allows visitors to the PBO site to search the database for names of artistic movements, design elements, specific designers, glossary terms, and thematic topics. This feature aids in tailoring each search to the precise needs of the site user.

Galleries: The Galleries/Essays provide additional information and insight into specific eras, geographical locations, single authors, titles, publishers and designers. These selections are designed to be used with the teaching tools and lesson plans, or as a springboard for further research.

Glossary: The PBO site's glossary contains over 450 entries relating to bindings and publishers' bindings. The site user may search for terms either alphabetically or specifically. The glossary component is particularly useful to purveyors of "medium-rare" books, Book Arts Scholars and Students, Archivists, Museums, and those who may have an interest in publishers' bindings but are unfamiliar with binding techniques or terminology.

K-12 Educators : PBO's lesson plans, worksheets, and teaching tools are written in such a way as to be easily adapted to High Schools, Middle Schools, or Elementary Schools. Most lesson plans contain three variations that are age appropriate, and they are also compatible with the Galleries/Essays section of the PBO site.

K-12 Students : Through the digitization of over 5,000 books, PBO is able to provide a visual resource about publishers' bindings from 1815-1930 for students. Previously, these books and images of these books had been inaccessible to the classroom, but through PBO and Digital Media students will benefit from exposure to the cultural and historical "common" object of the book. Students will also glean an awareness of publishers' bindings, historical methods of book production/distribution, and certain authors form the Galleries/Essay section of the PBO site.

Presentations, Publications & Podcast:

I. Presentations: (from earliest to most recent)

• June 21, 2006: Jessica Lacher-Feldman presented a paper at the RBMS Preconference, Austin, TX, Developing a Collaborative Model for Researching 19th-Century Books and Presenting Them to a Larger Audience: Issues and Prospects
• January 10, 2004: Jessica Lacher-Feldman presented a paper at the American Historical Association, Exploring Book History in the Digital World
• March 9, 2004: Jessica Lacher-Feldman spoke to the University Women's Club at Alabama
• March 2004: Jessica Lacher-Feldman presented the PBO project to the UA Libraries Forum
• November 2004: PBO Online presented to Tuscaloosa St. Andrew's Society
• January 6-10, 2005: Jessica Lacher-Feldman and Kristy Dixon gave a presentation at the American Historical Association annual meeting entitled 19 th c. Publisher's Bindings: A Reflection of American History and Culture
• April 2005: UW-Madison and UA presented at the American Association for History and Computing
• April 2005: Publishers' Bindings Online: Reflecting American History and Culture via the WWW , presented at the American Association for History and Computing, Roosevelt University , Schaumberg , IL
• July 2005: A tent show at the University of Iowa 's Center for Book Conference, The Changing Book: Transitions in Design, Production, and Preservation
• October 4, 2005: Jessica Lacher-Feldman presented the PBO project and the development of a Digital Project as a New Model for Academic Librarianship for the IMLS Fellows, School of Library and Library Studies, The University of Alabama
• November 2005: Jessica Lacher-Feldman, Jennifer Mathews (UA graduate student), and Erika Pribanic-Smith (UA graduate student) presented Publisher's Bindings Online: Building a Multifaceted Collaborative Digital Project at Society of Alabama Archivists Annual Meeting
• January 2006: Jessica Lacher-Feldman, Dr. Micki McElya (UA, American Studies), and Dr. Josh Rothman (UA, History) presented A Nation Bound: American Publishers' Book Bindings, Imagery, and The Relevance to and Reflection of History and Identity in America's First Century at American Historical Association in Philadelphia
• June 21, 2006: PBO focus of seminar Developing a Collaborative Model for Researching 19th c. Books and Presenting them to a Larger Audience: Issues and Prospectus at Rare Books and Manuscripts Preconference, “Libraries, Archives, and Museums in the Twenty-First Century: Intersecting Missions, Converging Futures?” in Austin , TX

II. Publications:

• December 8, 2003: Dialog , the UA's University-wide newsletter, electronic versions available at http://dialog.ua.edu/dialog20031208/bookcovers200321208.html
• Spring 2004: Alabama Archivist (Vol. 27, Issue 27)
• June 2005: CRNL 66.6
• Fall 2005 Friends News . UW?
• Spring 2005: “Article Title.” Library Horizons . www.lib.ua.edu/pubs/horizons_spring_05.pdf
• Spring 2006: “Judging Books by Their Covers: Online Resource is a Bibliophile's Dream.” Max Kade Institute Friends Newsletter (V. 15 No. 1 Spring 2006) Kevin M. Kurdylo

III. Podcast: www.lib.ua/randd/podcast/index.html

Outcomes Logic Model

IMLS provides a Logic Model Worksheet designed to aide in the creation of project focused outcomes and outcome based evaluations. PBO's Logic Model Worksheet can be accessed by selecting the link below.

Outcomes Based Evaluation Logic Model