Web archiving

Web archiving is the process of harvesting web content, organizing the content into a collection, and preserving the collection for access and use. Archiving the web allows us to combat the impermanent nature of online content, making future access and use possible. The Folger has been collecting and archiving select websites using the Archive-It subscription service since 2011. The Folger Shakespeare Library web collections can be accessed here.

Web Archiving

Web archiving is the process of “collecting portions of the World Wide Web, preserving the collections in an archival format, and then serving the archives for access and use” (IIPC). Web content is harvested through a process in which a web crawler accesses and gathers content from designated URLs through a process referred to as crawling. A web crawler is an internet “bot,” or program, that browses the web for indexing purposes. Crawlers access the desired website in a similar way to a web browser and captures all content related to the site, including any necessary information needed to render the site correctly as if it were live on the web: CSS files, etc.

The results of these crawls are captures of web content that can then be archived, described, and curated into digital collections. There are multiple digital resources involved in the capture and harvesting of even just one seed. A seed is an individual URL within a web archive collection. Following a web crawl, the information pertaining to a seed is organized into a WARC preservation file. The WARC file format is able to contain all necessary information and digital resources gathered from a seed during a crawl. It can also be expanded upon to include ancillary metadata elements. Websites archived in the WARC file format can be viewed and interacted with in a web browser using access tools such as the Internet Archive’s Wayback Machine. Advanced manipulation of web archive data can facilitate a number of research techniques: potential uses for web archive collections include textual or link analysis, among others.

Ultimately, web archiving is intended to preserve a realm of cultural history that is increasingly present, and sometimes only present online in digital format. Digital information is very sensitive. Sites are reliant upon a number of external factors in order to be accessed by users: content creators, host domains, web browsers, markup languages, etc. Subsequently, internet content can disappear for a variety of reasons frequently and often without notice. For example, the popular web resource Mr. Shakespeare and the Internet was taken offline in October of 2013. If not saved, the information it contained would have been lost to users. Fortunately, the website was archived in time by the Internet Archive and is made accessible via the Wayback Machine.

Web Archiving at the Folger Shakespeare Library

The Folger began archiving and preserving select websites using the Archive-It subscription service in October of 2011. Collections are administered by the Folger Shakespeare Library: Central Library. They can be accessed here. The mission of the Folger Shakespeare Library is as follows: “to preserve and enhance our collection; to make our collection accessible to scholars and others who can use it productively; and to advance understanding and appreciation of Shakespeare’s writings and the culture of the early modern world.”

Developed by Jim Kuhn (Head of Collection Information Services, 2006-2013) and Emily Wahl (Central Library), the Folger web collections were created to address a new update (2010) to the Folger Collection Development mandate which expresses an institutional commitment to digital collecting in Shakespeare-related areas, including born-digital ephemera.

Current Folger Shakespeare Library Web Collections

An institutional collection; Folger Shakespeare Library Websites and Social Media archives and preserves the Folger's web presence over time. The collection includes all Folger domains, blogs, and social media profiles. Seeds in this collection are crawled for new content on a quarterly basis. The collection can be accessed here.

A thematic collection; its purpose is to archive official websites for theatrical companies and drama festivals which focus on Shakespeare performance. The scope of this collection is primarily limited to the United States; however, a growing number of international resources are included as well. There are currently over 280 seeds in this collection and they are crawled for new content on a semi-annual basis. The collection can be accessed here.

An events-based collection; this collection seeks to document various celebrations, commentary, and events as depicted on the web related to the 450th anniversary of William Shakespeare’s birth. The collection can be accessed here.

Additional Folger Shakespeare Library Web Collecting Activities

The #Shax450 Tweet Archive and Visualization, created using Martin Hawksey’s TAGSExplorer tool and Google Spreadsheets, is an interactive archive and visualization of tweets that have used the hashtag #Shax450 on Twitter to celebrate the 450th anniversary of William Shakespeare’s birth in 2014. The archive can be accessed here and the data visualization can be accessed here.

Additional Resources

An Introduction to Web Archiving at the Folger | The Collation

Continuing the Celebration: Preserving Birthday-Related Digital Ephemera | The Collation

William Shakespeare: Playwright, Icon, Web Archivist? | The Archive-It Blog

Contact

Please feel free to contact the Folger Web Archives Administrator at folgerwebarchives@gmail.com if you have any questions or comments regarding the Folger Shakespeare Library web collections, or if you would like to report a problem you have encountered while interacting with these collections. If you would like to nominate a website for inclusion, you may complete this form. While all nominations are carefully reviewed, please note that we cannot guarantee the inclusion of a nominated website in the Folger web collections.