Digital Research Data Curation: Overview of Issues, Current Activities, and Opportunities for the Cornell University Library

Cornell University Library's Data Working Group has deposited its Digital Research Data Curation: Overview of Issues, Current Activities, and Opportunities for the Cornell University Library report in the eCommons@Cornell repository.

Here's the abstract:

Advances in computational capacity and tools, coupled with the accelerating collection and accumulation of data in many disciplines, are giving rise to new modes of conducting research. Infrastructure to promote and support the curation of digital research data is not yet fully-developed in all research disciplines, scales, and contexts. Organizations of all kinds are examining and staking out their potential roles in the areas of cyberinfrastructure development, data-driven scholarship, and data curation. The purpose of the Cornell University Library's (CUL) Data Working Group (DaWG) is to exchange information about CUL activities related to data curation, to review and exchange information about developments and activities in data curation in general, and to consider and recommend strategic opportunities for CUL to engage in the area of data curation. This white paper aims to fulfill this last element of the DaWG's charge.

CIC Shared Digital Repository Project Update

A recently updated description of the Committee on Institutional Cooperation's Shared Digital Repository Project is available at Indiana University's Project: Shared Digital Repository page.

Here's an excerpt:

Description: The Shared Digital Repository (SDR) leverages the tradition of leadership in collaboration among the institutions of the Committee on Institutional Cooperation (CIC). The SDR operates under the leadership of the Repository Administrators (Indiana University and the University of Michigan), which also provide a large part of the funding. Additional governance and financial support are provided by the charter participating libraries of the CIC, and by other libraries and library consortia wishing to archive digital content.

Outcome: The SDR offers persistent and high-availability storage for digitized book and journal content, beginning with the Google content from the CIC members and later extending to other digitized content. The SDR will leverage technology investments and developments at the University of Michigan to build (through IU/UM collaboration) more generalized versions of Michigan's services and gain efficiencies from Michigan's investments. . . .

Milestones and status:

As of April 11, 2008, the SDR contains:

  • 1,122,007 volumes
  • 791,460 titles
  • approximately 393 million pages
  • 213,379 individual volumes in the public domain (19% of the total)

Timeline:

  • Early 2008: Bloomington backup storage installed
  • January-March 2008: Page turner mechanism with branding; ability to publish virtual collections (UM-specific version); assessment of global searching functionality; access mechanisms for persons with visual disabilities
  • September-December 2008: Mechanism for direct ingest of non-Google content; compliance with the required elements in the "Trustworthy Repositories Audit & Certification (TRAC): Criteria and Checklist"

Survey of Canadian and International Data Management Initiatives Released

The Canadian Association of Research Libraries (CARL) has released Survey of Canadian and International Data Management Initiatives.

Here's an excerpt from the "Introduction":

Research libraries have a role to play in this emerging data-intensive environment. A 2007 CARL survey found that most CARL members are interested in managing research data, but few have a formal data archiving policy. CARL has formed a Research Data Management Working Group to assist members in collecting, organizing, preserving and providing access to the research data and to formulate a cooperative approach for CARL.

The purpose of this report is to provide an overview of the types of data management activities being undertaken in Canada and internationally. This review documents the various options available for libraries, and will pave the way for a more detailed investigation by the Working Group of the potential roles for libraries.

SPAR (A Distributed Archiving and Preservation System)

The Bibliothèque Nationale de France is engaged in the SPAR (A Distributed Archiving and Preservation System) project.

Here's an excerpt from the project's English home page:

After more than a year of study, BnF launched the SPAR project, true digital stack. Its design is based on international standards authoritative in the subject of the sustainability of digital information. In particular, SPAR respects the OAIS standard (ISO-14721:2003), reference model for an open archival information system. . . .

The SPAR project is much more than a simple stack of secure data.

  • It makes multiple copies of the digital objects and provides continuous monitoring on the status of the hardware as well as the media containing the recorded files in order to anticipate new copies before a definitive loss.
  • Through a precise and complete recognition of the formats of the ingested objects, it also guarantees the continuity of access by making the necessary transformations in case of the technological obsolescence of the access software. Hence, for example, when the JPEG image format will become obsolete, SPAR will be able to transform all the appropriate images in a new and more permanent format. . . .

SPAR is a system serving a community. It must guarantee that the documents given back haven't been modified. To this goal, SPAR marks each object with a digital signature. Moreover, to guarantee the access rights of the disseminated digital objects, SPAR uses a rights management system which calculates the usage licenses of the digital objects and applies the necessary restrictions depending on the user . . .

Read more about it at Bringing Seven Centuries into the Future: Bibliothèque Nationale de France: SPAR Analysis.

BagIt: New LC/CDL Format for Transferring Digital Content between Cultural Institutions

The Library of Congress and the California Digital Library have established a new format called BagIt for transferring large data collections between cultural institutions.

Read more about it at "The BagIt File Package Format (V0.94)" and "Library Develops Format for Transferring Digital Content."

RIN Publishes To Share or not to Share: Publication and Quality Assurance of Research Data Outputs

The Research Information Network has published To Share or not to Share: Publication and Quality Assurance of Research Data Outputs. The report has a separate Annex file.

This report presents the findings from a study of whether or not researchers do in fact make their research data available to others, and the issues they encounter when doing so. The study is set in a context where the amount of digital data being created and gathered by researchers is increasing rapidly; and there is a growing recognition by researchers, their employers and their funders of the potential value in making new data available for sharing, and in curating them for re-use in the long term.

Digital Preservation of E-Journals in 2008: Urgent Action Revisited Released

Portico and Ithaka have released Digital Preservation of E-Journals in 2008: Urgent Action Revisited.

Here's an excerpt from the announcement:

In September 2005, library directors from 17 universities and colleges met to discuss the current state of electronic journal preservation and endorsed a statement calling for “Urgent Action” to preserve scholarly e-journals. Over two years later in January 2008, in the Portico and Ithaka invited 1,371 library directors of four-year colleges and universities in the United States to respond to a survey examining current perspectives on preservation of e-journals. A strong response has yielded interesting findings that we now share with the community in the hope they will spark useful discussion among library directors, funders, and administrators regarding strategic library priorities.

The survey finds that a large majority of library directors across the spectrum strongly agree or agree that the potential loss of e-journals is unacceptable, and a significant majority believe their own institution has a responsibility to take action to prevent an intolerable loss of the scholarly record. Most larger libraries responding now support one or more e-journal preservation initiatives; however, the majority of respondents from smaller libraries have yet to support any preservation effort and secure permanent access to e-journals for their institutions. The survey shows that this majority is significantly uncertain about their options for e-journal preservation and how urgent is the need to act.

Evaluation of the JISC UK LOCKSS Pilot Report Released

JISC has released Evaluation of the JISC UK LOCKSS Pilot.

Here's an excerpt from the Executive Summary:

This report provides an evaluation of the UK LOCKSS pilot project as it reaches the end of its pilot phase. LOCKSS (Lots Of Copies Keep Stuff Safe) is an international community-based archiving initiative led by Stanford University in the US with over 170 member libraries worldwide, archiving content from over 200 publishers. It is a community-based archiving service operating on open-source software, giving libraries control over their own archived content with the emphasis on low cost and low maintenance.

In March 2006, JISC in partnership with the Consortium of Research Libraries in the British Isles (CURL) funded membership of a collective UK LOCKSS Alliance for 24 selected UK HE libraries. The Digital Curation Centre (DCC) at the department of HATII within the University of Glasgow received funding to set up the UK LOCKSS Technical Support Service to provide technical advice and general support to pilot members and Content Complete Ltd. (CCL) were funded to negotiate with NESLi2 and other publishers to allow LOCKSS-based archiving. The UK LOCKSS pilot is particularly innovative as the first effort to establish a country wide LOCKSS network. The study found that the UK LOCKSS pilot project had achieved its overall aim of setting up a UK LOCKSS Alliance of 30 HE libraries, but that in relation to the detailed aims and objectives there were a number of issues to be addressed if the project was to become self-sustaining at the end of the pilot phase.

ALCTS Preservation and Reformatting Section Publishes Digital Preservation Definition

The Preservation and Reformatting Section of the Association for Library Collections & Technical Services, a division of the American Library Association, has published its formal definition of digital preservation.

Here's an excerpt from the press release:

The definition was developed to promote an understanding of digital preservation within the library community, as well as our allied professions and the user communities we exist to serve. This definition is presented to mark our current understanding of digital preservation and encourage further development of these ideas.

This definition grew out of a conversation held at the Digital Preservation Discussion Group at the ALA Midwinter Meeting in 2007. An ad hoc task force crafted language that was shared with a number of constituencies during the spring and early summer of 2007. The definition was discussed and approved by the PARS Executive Committee during the 2008 Midwinter Meeting in Philadelphia. The ALCTS Board of Directors approved it during Midwinter, and the definition was presented to and accepted by Council as an informational document. The definition is being incorporated into the forthcoming revision of the current ALA Preservation Policy currently being undertaken by PARS.

The working group studied a number of resources to familiarize itself with the critical elements of digital preservation identified by a broad selection of individuals and agencies. These ideas were cast into language that speaks to a wide variety of stakeholders while also being consistent with the core preservation concepts that have developed in the library and archival communities.

The core concepts are presented in a short, medium and long version to accommodate a variety of needs. The long version includes a number of currently accepted best practices but is not intended to be an exhaustive list. As more is learned about implementing digital preservation programs, the definitions should be reviewed and revised on a regular basis.

The definition will be reviewed and updated as needed.

OCLC Announces Digital Archive Service

OCLC has announced the availability of a Digital Archive service.

Here's an excerpt from the press release:

The service provides a secure storage environment for libraries to easily manage and monitor master files and digital originals. The importance of preserving master files grows as a library's digital collections grow. Libraries need a workflow for capturing and managing master files that finds a balance between the acquisition of both digitized and born-digital content while not outpacing a library's capability to manage these large files. . . .

The Digital Archive service is a specially designed system in a controlled operating environment dedicated to the ongoing managed storage of digital content. OCLC has developed specific systems processes and procedures for the service tuned to the management of data for the long term.

From the time content arrives, the Digital Archive systems begin inspecting it to ensure continuity. OCLC systems perform quality checks and record the results in a "health record" for each file. Automated systems revisit these quality checks periodically so libraries receive up-to-date reports on the health of the collection. OCLC provides monthly updated information for all collections on the personal archive report portal.

For users of CONTENTdm, OCLC's digital collection management software for libraries and other cultural heritage institutions, the Digital Archive service is an optional capability integrated with various workflows for building collections. Master files are secured for ingest to the Digital Archive service using the CONTENTdm Acquisition Station, the Connexion digital import capability and the Web Harvesting service.

For users of other content management systems, the Digital Archive service provides a low-overhead mechanism for safely storing master files.

Audiovisual Research Collections and Their Preservation Published

TAPE (Training for Audiovisual Preservation in Europe) has published Audiovisual Research Collections and Their Preservation.

Here's an excerpt from the introduction:

Digital technology has conquered audiovisual production, post-processing, and archiving. Audio has totally become part of the IT world, and video is about to follow the same way. All dedicated audio formats are dead, and soon the same will be the case for video formats. The pace by which dedicated audio and video formats are becoming obsolete is breathtaking. The problem is not so much the survival of the original documents, but the availability of highly specialised replay equipment which disappears from the market soon after a format has been abandoned commercially. Today audiovisual archives associations estimate the time window still open for the transfer of dedicated analogue and digital carriers into digital repositories to be not more than just 20 years.

Repository Planning Checklist and Guidance Released: Presents Planning Tool for Trusted Electronic Repositories (PLATTER)

DigitalPreservationEurope has released Repository Planning Checklist and Guidance.

Here's an excerpt from the "Executive Summary and Introduction to Platter":

The purpose of this document is to present a tool, the Planning Tool for Trusted Electronic Repositories (PLATTER) which provides a basis for a digital repository to plan the development of its goals, objectives and performance targets over the course of its lifetime in a manner which will contribute to the repository establishing trusted status amongst its stakeholders. PLATTER is not in itself an audit or certification tool but is rather designed to complement existing audit and certification tools by providing a framework which will allow new repositories to incorporate the goal of achieving trust into their planning from an early stage. A repository planned using PLATTER will find itself in a strong position when it subsequently comes to apply one of the existing auditing tools to confirm the adequacy of its procedures for maintaining the long term usability of and access to its material. . . .

The PLATTER process is centred around a group of Strategic Objective Plans (SOPs) through which a repository specifies its current objectives, targets, or key performance indicators in those areas which have been identified as central to the process of establishing trust. In the future, PLATTER can and should be used as the basis for an electronic tool in which repositories will be able to compare their targets with those adopted by other similar (suitably anonymised) repositories. The intention is that the SOPs should be living documents which evolve with the repository, and PLATTER therefore defines a planning cycle through which the SOPs can develop symbiotically with the repository organisation.

ARL Preservation Statistics 2005–06 Published

The Association of Research Libraries has released a PDF version of ARL Preservation Statistics 2005–06, which is freely available.

Here's an excerpt from the "Introduction":

Among the significant developments that took place in research libraries in the 1980s was the emergence of preservation programs as distinct administrative units, separately staffed, funded, and administered. There were 66 such programs reported in 1988, as many as 80 reported in recent years, and 77 in 2005-06.

These rapidly shifting trends have made themselves evident in many categories. Preservation expenditures for ARL’s 111 reporting member libraries were $107,937,836 in 2005-06, which reflects an inflation-adjusted increase of 27% since the survey’s revision in 1996-97.3 Total preservation staff increased to just under 1,800 FTEs in 2005-06, 5.4% more than in 2004-05. Level 1 conservation treatment decreased from 2004-05 levels, while the number of items treated at Levels 2 and 3 increased; total conservation has increased by more than 50,000 volumes in the past year, bringing it higher than it has been in the last four years. . . .

Digitizing bound volumes is gradually emerging as a viable preservation option. In 2005-06, 54 ARL libraries reported more than zero bound volumes digitized. The amount of items digitized varies widely, from one volume at the University of Delaware to 25,121 volumes digitized by the University of Florida.

Sound Directions Project Releases FACET, Preservation Analysis Tool for Audio Works

The Sound Directions: Digital Preservation and Access for Global Audio Heritage Project of Indiana University and Harvard University has released FACET (Field Audio Collection Evaluation Tool), an open-source tool for evaluating preservation issues related to audio works.

Here's an excerpt from the FACET web page:

The Field Audio Collection Evaluation Tool (FACET) is a point-based, open-source software tool that ranks audio field collections based on preservation condition, including the level of deterioration they exhibit and the degree of risk they carry. It assesses the characteristics, preservation problems, and modes of deterioration associated with the following formats: open reel tape (polyester, acetate, paper and PVC bases), analog audio cassettes, DAT (Digital Audio Tape), lacquer discs, aluminum discs, and wire recordings. This tool helps collection managers construct a prioritized list of audio collections by condition and risk, enabling informed selection for preservation. Using FACET provides strong justification for preservation dollars.

Preserving Mixed Analog/Digital AV Archives: PrestoSpace Project Case Study

The Digital Curation Centre has published DCC Case Study—PrestoSpace: Preservation towards Storage and Access. Standardised Practices for Audiovisual Contents in Europe.

Here's the "Executive Summary":

Explicit strategies are needed to manage 'mixed' audio visual (AV) archives that contain both analogue and digital materials. The PrestoSpace Project brings together industry leaders, research institutes, and other stakeholders at a European level, to provide products and services for effective automated preservation and access solutions for diverse AV collections. The Project’s main objective is to develop and promote flexible, integrated and affordable services for AV preservation, restoration, and storage with a view to enabling migration to digital formats in AV archives.

Alternative File Formats for Storing Master Images of Digitisation Projects

Koninklijke Bibliotheek has published Alternative File Formats for Storing Master Images of Digitisation Projects.

Here's an excerpt from the "Management Summary":

The main conclusions of this study are as follows:

Reason 1: Substitution

JPEG 2000 lossless and PNG are the best alternatives for the uncompressed TIFF file format from the perspective of long-term sustainability. When the storage savings (PNG 40%, JPEG 2000 lossless 53%) and the functionality are factored in, the scale tips in favour of JPEG 2000 lossless.

Reason 2: Redigitisation Is Not Desirable

JPEG 2000 and JPEG are the best alternatives for the uncompressed TIFF file format. If no image information may be lost, then JPEG 2000 lossless and PNG are the two recommended options.

Reason 3: Master File is the Access File

JPEG 2000 lossy and JPEG with greater compression are the most suitable formats.

Planets Project Releases White Paper: Representation Information Registries

The Planets (Preservation and Long-term Access through Networked Services) project has released White Paper: Representation Information Registries.

Here's the "Executive Summary":

This document is a report on the state-of-the-art in the field of Representation Information Registries (RIRs). RIRs are widely recognised as a critical component of digital preservation architecture in general, and a number of such registries are being developed as part of the Planets architecture in particular. This document discusses the development of the concept of representation information, and of the use of registries as a means of exposing that information for use by digital preservation services; it describes the RIR implementations which currently exist or are under development globally; it assesses planned and potential future developments in this area; it discusses the role which RIRs play within the Planets project, and concludes with recommendations for future areas of research within Planets and beyond.

NDIIPP's Carl Fleischhauer on Video Formatting and Preservation

The National Digital Information Infrastructure and Preservation Program at the Library of Congress has released Carl Fleischhauer's presentation on "Video Formatting and Preservation" at the Digital Library Federation 2007 Fall Forum.

Here's an excerpt about the presentation from the March 2008 issue of the Library of Congress Digital Preservation Newsletter:

Fleischhauer discussed content wrappers, bitstream encodings, metadata and format profiles for born-digital content. He also spoke about emerging reformatting practices at the Library’s new facility for audiovisual collections and a handful of notable NDIIPP projects.

NEH Awards $474,474 in Digital Humanities Start-Up Grants

The National Endowment for the Humanities has awarded $474,474 to Digital Humanities Start-Up Grants recipients.

Here's an excerpt from the press release:

Note: The We the People program encourages and strengthens the teaching, study, and understanding of American history and culture. Grants bearing this designation have been recognized for advancing the goals of this program.

ALASKA

Fairbanks

University of Alaska, Fairbanks $50,000
Digital Humanities Start-Up Grants
Project Director: Siri Tuttle
We the People Project Title: Minto Songs Project Description: The collection, digitization, organization, and archival storage, as well as dissemination among the Minto Athabascan community, of recorded performances of Alaskan Athabascan songs.

ARIZONA

Tucson

University of Arizona $25,000
Digital Humanities Start-Up Grants
Project Director: Douglas Gann
Project Title: Virtual Vault
Project Description: Electronic access to the world's largest collection of whole pottery vessels from the American Southwest through digital renderings of Arizona State University's Pottery Vault and relevant prehistoric archaeological sites as well as interviews with anthropologists, conservators, and Native American potters.

ILLINOIS

Lake Forest

Lake Forest College $25,000
Digital Humanities Start-Up Grants
Project Director: Davis Schneiderman
We the People Project Title: Virtual Burnham Initiative
Project Description: The development of the Virtual Burnham Initiative (VBI), a multimedia project that would examine the history and legacy of Daniel H. Burnham's and Edward H. Bennett's Plan of Chicago (1909).

MARYLAND

College Park

University of Maryland, College Park $11,708
Digital Humanities Start-Up Grants
Project Director: Matthew Kirschenbaum
Project Title: Approaches to Managing and Collecting Born-Digital Literary Materials for Scholarly Use
Project Description: A series of planning meetings and site visits aimed at developing archival tools and best practices for preserving born-digital documents produced by contemporary authors.

MASSACHUSETTS

Boston

University of Massachusetts, Boston $24,748
Digital Humanities Start-Up Grants
Project Director: Joanne Riley
We the People Project Title: Online Social Networking for the Humanities: the Massachusetts Studies Network Prototype
Project Description: The development and evaluation of a social networking platform for the members of the statewide Massachusetts Studies Project.

Norton

Wheaton College $41,950
Digital Humanities Start-Up Grants
Project Director: Mark LeBlanc
Project Title: Pattern Recognition through Computational Stylistics: Old English and Beyond
Project Description: Development of a prototypical suite of computational tools and statistical analyses to explore the corpus of Old English literature using the genomic approach of tracing information-rich patterns of letters as well as that of literary analysis and interpretation.

MISSISSIPPI

Mississippi State

Mississippi State University $50,000
Digital Humanities Start-Up Grants
Project Director: Paul Jacobs
Project Title: Distributed Archives Transaction System
Project Description: Development of open source web tools for accessing online digitized collections in the humanities via a system that communicates with multiple database types while protecting the integrity of the original data sets.

NEW YORK

Brooklyn

Unaffiliated Independent Scholar $23,750
Digital Humanities Start-Up Grants
Project Director: Daniel Visel
Project Title: Sophie Search Gateway
Project Description: The development of an interoperable portal within the Web authoring program, "Sophie," for locating and incorporating multi-media sources from the Internet Archive.

Hempstead

Hofstra University $23,591
Digital Humanities Start-Up Grants
Project Director: John Bryant
We the People Project Title: Melville, Revision, and Collaborative Editing: Toward a Critical Archive
Project Description: The development of the TextLab scholarly editing tool to allow for analysis of texts that exist in multiple versions or editions, beginning with the Melville Electronic Library.

New York City

New York University $49,657
Digital Humanities Start-Up Grants
Project Director: Brian Hoffman
Project Title: MediaCommons: Social Networking Tools for Digital Scholarly Communication
Project Description: Development of a set of networking software tools to support a "peer-to-peer" review structure for MediaCommons, a scholarly publishing network in the digital humanities.

RHODE ISLAND

Providence

Brown University $49,992
Digital Humanities Start-Up Grants
Project Director: Julia Flanders
Project Title: Encoding Names for Contextual Exploration in Digital Thematic Research Collections
Project Description: The advancement of humanities text encoding and research by refining and expanding the automated representation of personal names and their contexts.

TEXAS

Austin

University of Texas, Austin $49,251
Digital Humanities Start-Up Grants
Project Director: Samuel Baker
Project Title: The eCommentary Machine Project
Project Description: Development of a web-based collaborative commentary and annotation tool.

VIRGINIA

Charlottesville

University of Virginia $49,827
Digital Humanities Start-Up Grants
Project Director: Scot French
We the People Project Title: Jefferson's Travels: A Digital Journey Using the HistoryBrowser
Project Description: Development of an interactive web-based tool to integrate primary documents, dynamic maps, and related information in the study of history, with the prototype to be focused on Thomas Jefferson's trip to England in 1786.

Tool for Digital Preservation: Recover Dead Websites or Rebuild Websites with Warrick

Warrick is an open source software tool from the Old Dominion University Computer Science Department for recovering or reconstructing Websites using composite data from Google, Internet Archive, Live Search, and Yahoo. It can used at the Warrick Website or downloaded.

Read more about it at "About Warrick" and "Warrick."

Sun Centre of Excellence for Libraries to Be Created in Alberta

Sun Microsystems has announced that it is partnering with the University of Alberta Libraries and the Alberta Library to create the Sun Centre of Excellence for Libraries.

Here's an excerpt from the press release:

Sun Microsystems of Canada Inc., the University of Alberta Libraries (UAL) and The Alberta Library (TAL) today announced the creation of a new Sun Centre of Excellence for Libraries (COE). The initiative will enhance and support respective organizational projects, as well as an extensive, province-wide, multi-faceted digital library. As part of the COE the participants intend to provide a seamless search and retrieval experience resulting in unprecedented access to information for students, faculty and the public, as well as creating an enduring preservation environment.

"This initiative will facilitate new levels of access to a tremendous amount of unique information that hasn’t been widely available," said Ernie Ingles, Vice Provost and Chief Librarian, University of Alberta. "It will further our goal to act as a trusted regional repository for digital materials by facilitating approaches to the discovery, storage, and archival preservation of digital resources that will benefit all Canadians." The University of Alberta Libraries, the second largest academic library system in Canada, has more than one million unique digitized pages of content in four major collections to contribute to the new digital library.

Using a range of Sun systems, software and thin client technologies, The Alberta Library (TAL) will integrate current digital collections and electronic information resources from the Lois Hole Campus Alberta Digital Library, an Alberta Government initiative that is providing post-secondary students, faculty and researchers in every corner of the province with access to vast holdings of digital resources. The digital library currently contains more than 4.5 million licensed items, including academic journals, encyclopedias, magazine and newspaper articles, literary criticisms and video clips from 35 post-secondary institutions. The COE will also help TAL improve province-wide access to library catalogues and secure information-sharing. . . .

The COE will support distance learning and research within e-learning environments by providing access to digital collections preserved by Alberta university libraries, archives and museums. It will also yield solutions for long-term archiving of digital resources, and digital rights management. The support and technology provided by Sun will ensure the infrastructure can evolve to meet future needs and continue to support research, collaborative learning and general discovery. . . .

The Centre of Excellence for Libraries is expected to be operational by summer 2008.