If you attended the recent GA, or read some of the many blog posts about it, you probably heard about the potential benefits of standardized web archiving APIs. This was a common theme that came up in multiple presentations and informal discussions. During a conversation over lunch mid-week one person suggested that the IIPC form a new working group to focus on web archiving APIs. Clearly some institutions were interested in this, but how many? And are they interested enough to participate in a new working group? A group of us at Harvard decided to find out. We developed a short survey and advertised it on the IIPC mailing list.
The survey was open from May 14 through June 1 and was filled out 18 times, by 17 different institutions from 8 different countries.
|Czech Republic||National Library of the Czech Republic|
|France||Bibliothèque nationale de France|
|Iceland||National and University Library of Iceland|
|New Zealand||National Library of New Zealand|
|Spain||National Library of Spain|
|United Kingdom||The British Library, The National Archives|
|United States||Stanford University Libraries, Old Dominion University, Internet Archive, LANL, California Digital Library, Harvard Library, Library of Congress, UCLA, University of North Texas|
Table 1: The institutions that responded to the survey
The survey asked “Is the topic of web archiving APIs of interest to your institution?” The answer was overwhelmingly “Yes”. All 17 institutions are interested in web archiving APIs. Personally this was the first unanimous survey question I have ever seen.
When asked “Why are web archiving APIs of interest to your institution?” the responses (see Figure 2) were varied but had common themes. Many of the reasons were from the perspective of an institution providing or maintaining web archiving programs or infrastructure, for example:
- “The sustainability of our program depends on the web archiving community as a whole better aligning itself to collaboratively maintain and augment a core set of interoperable systems…”
- “…appreciate how this would reduce our technical spend in the long term…”
- “… APIs should ease the maintenance and evolution of the complex set of tools we are using to complete the document cycle: selection, collect, access and preservation.”
Another common response was from the perspective of providing a better service for researchers, for example:
- “…a common/standard API would make it easier for researchers to work with multiple web archives with standard methodologies.”
- “To help researchers explore our collection, including within our catalogue system, to link with other web archive collections and potentially to interface with different components of our infrastructure.”
- “We often do aggregation and want to have a way to archive resources of interest with the help of scripts, in both of these cases an API would be ideal.”
The respondents were asked “If we organized a new working group within the IIPC to work on web archiving APIs would your institution be willing to participate?” All but one institution said “Yes”. The institution that said “No” said that they were interested but did not have enough staff resources currently to actively participate.
We asked “In what specific ways could your institution participate? Please select all that apply.” The results are shown in Table 2. Most of the respondents would like to help define the functional requirements, but a good amount would also like to contribute use cases and help design the technical details. Importantly, there are institutions willing to help run the meetings.
|Specific Way||% of Respondents||Count of Respondents|
|Help define the functional requirements for a web archiving API||94%||15|
|Contribute curatorial, researcher or management requirements and use cases||81%||13|
|Help design the technical details of a web archiving API||69%||11|
|Help schedule and run the working group meetings||19%||3|
Table 2: The specific ways institutions would participate in the working group
* One institution said that they would be willing to implement and test web archiving APIs where appropriate and aligned with local needs
So the answer to our original question is a clear YES! There are enough IIPC institutions that are interested and willing to participate in meaningful ways in this new working group. Stay tuned while we work through the logistics of how to start. One of the first steps will be to identify co-chairs for the group. If you are interested in this please let me know! And thanks everyone for taking the time to fill out this survey.
By Andrea Goethals, Manager of Digital Preservation and Repository Services, Harvard Library