Category: Digital Curation & Digital Preservation

The Planets project has deposited a "Digital Genome" time capsule in the Swiss Fort Knox.

Here's an excerpt from the press release:

Over the last decade the digital age has seen an explosion in the rate of data creation. Estimates from 2009 suggest that over 100 GB of data has already been created for every single individual on the planet ranging from holiday snaps to health records—that's over 1 trillion CDs worth of data, equivalent to 24 tons of books per person!

Yet research by the European Commission co-funded Planets project, co-ordinated by the British Library, highlights deep concerns regarding the preservation of these digital assets. Findings suggest that as hardware and software are superseded by more up-to-date technology, and older formats become increasingly inaccessible, the EU alone is losing over 3 billion euros worth of digital information every year.

Looking to ensure the preservation of our digital heritage, on 18 May 2010 the Planets project will deposit a time capsule containing a record of the "Digital Genome" inside Swiss Fort Knox—a high security digital storage facility hidden deep in the Swiss Alps—preserving the information and the tools to reconstruct highly valuable data long after the lifeline of supporting technology has disappeared.

Inside the Digital Time Capsule:

• Five major at risk formats—JPEGs, JAVA source code, .Mov files, websites using HTML, and PDF documents
• Versions of these files stored in archival standard formats—JPEG2000, PDFA, TIFF and MPEG4—to prolong lifespan for as long as possible
• 2500 additional pieces of data—mapping the genetic code necessary to describe how to access these file formats in future
• Translations of the required code into multiple languages to improve chances of being able to interpret in the future
• Copies of all information stored on a complete range of storage media—from CD, DVD, USB, Blu-Ray, Floppy Disc, and Solid State Hard Drives to audio tape, microfilm and even paper print outs . . .

Since 2007 the volume of data produced globally has risen from 281 exabytes to over 700 exabytes—much of this is now considered to be at risk from the repeated discontinuation of storage formats and supporting software. Current studies suggest that common storage formats such as CDs and DVDs have an average life expectancy of less than 20 years, yet the proprietary file formats to access content often last as little as five to seven years and desktop hardware even less. Backing up this data is a start, but without the information and tools to access and read historical digital material it is clear huge gaps will open up in our digital heritage.

To meet this threat, in 2006 the European Commission established the Planets project—Preservation and Long-term Access through Networked Services—bringing together a coalition of European libraries, archives, research organisations, and technology institutions including the Austrian National Library, the University of Technology of Vienna, and the British Library to develop the software solutions to guarantee long-term access. Marking the end of the first phase of the project the deposit of the Planets "Digital Genome" in Swiss Fort Knox will help to highlight the fragility of modern data and help to protect our digital heritage from a whole range of human, environmental and technological risks.

Library and Archives Canada has released Survey of Digital Preservation Practices in Canada: Final Report.

Here's an excerpt:

In 2008, Library and Archives Canada commissioned a survey on digital preservation practices in Canada in order to gain a clearer understanding of the current state. The intent was to determine existing practices and resources for digital preservation as well as to identify gaps and challenges. This report presents the results of this survey. . . .

The survey was divided into six sections: (1) Introduction, (2) Information about the repository, (3) General policies and practices, (4) Preservation practices, (5) Preservation resources, and (6) Challenges. Respondents who rated their organization's preservation capacity as very low were re-routed to an abridged version of the questionnaire.

The survey received 61 full responses from a variety of types of organizations: libraries, archives government departments/agencies, museums, research institutes, across a number of sectors: academic, governments and not-for profit. Although invitations were sent to several organizations in private industry, no responses were received from this sector.

Respondents' repository collections ranged from discrete collections of digitized monographs, images, or audio files, to data repositories, to broader scope, multiple format collections. The repository collections were also wide ranging in terms of size with the smallest collection having 70 objects and the largest collection containing over 8 million objects. 95% of respondents indicated that the content in the repositories was predominantly Canadian. That is, the content was produced in Canada or was about Canada.

JISC has released Keeping Research Data Safe 2.

Here's an excerpt:

The first Keeping Research Data Safe study funded by JISC made a major contribution to understanding of long-term preservation costs for research data by developing a cost model and identifying cost variables for preserving research data in UK universities (Beagrie et al, 2008). The Keeping Research Data Safe 2 (KRDS2) project has built on this work and delivered the following:

• A survey of cost information for digital preservation, collating and making available 13 survey responses for different cost datasets;
• The KRDS activity model has been reviewed and its presentation and usability enhanced;
• Cost information for four organisations (the Archaeology Data Service; National Digital Archive of Datasets; UK Data Archive; and University of Oxford) has been analysed in depth and presented in case studies;
• A benefits framework has been produced and illustrated with two benefit case studies from the National Crystallography Service at Southampton University and the UK Data Archive at the University of Essex.

Plants has released The Digital Divide: Assessing Organizations' Preparations for Digital Preservation.

Here's an excerpt:

• The volume of digital content that organizations expect to archive will increase 25-fold over the next ten years.
• While seventy per cent of organizations hold less than 20 terabytes (TB) of data now, by 2019 seventy per cent of organizations expect to hold over 100TB.
• Digital information comes in a range of types, and while over 80% of organizations already need to preserve documents and images, by 2019 over 70% will need to preserve databases, websites, audio and video files as well.
• Only 27% of organizations think that they have complete control over the file formats that they will accept and store in their digital archives. Since the choice of format affects how easy it is to preserve digital content, producers need to be more involved in digital preservation.
• The digital preservation message has spread far and wide: 93% of respondents indicated that their organisation is aware of the challenges of managing digital information for the long-term.
• Organizations are taking account of digital preservation: 76% include it in their operational planning, 71% in their business continuity planning and 62% in their financial planning.
• By setting out a digital preservation policy, 48% of organizations are actively planning how to tackle digital preservation.
• Organizations are only starting to commit to funding digital preservation, as just 47% have allocated a budget to it.

Read more about it at Survey Analysis Report.

Planets (Preservation and Long-term Access through NETworked Services) has released seven training videos, an annotated reading list, and a set of technical summaries about digital preservation.

Here's a list of the digital videos:

• Introduction to Digital Preservation: Why Preserve? How to Preserve?
• The Preservation Action Cycle: Introduction to Planets
• How to Preserve?
• Tools: How to Understand Files
• Testbed: A Controlled Environment for Experimentation and Evaluation in Digital Preservation
• Digital Preservation: How to Plan: Preservation Planning with Plato
• Tools: How to Integrate the Components of Digital Preservation

Anna Gold has self-archived "Data Curation and Libraries: Short-Term Developments, Long-Term Prospects" in DigitalCommons@CalPoly.

Here's an excerpt:

This paper was prepared as background for a talk given at AGU 2009 on "Data & Libraries." It summarizes the developments and events from late 2006 through early 2010 that are shaping library roles in scientific data curation while underscoring the range, complexity, and varying granularity of systems, actions, and efforts involved. The main conclusions are: (1) leaders of major research libraries have committed their institutions to support data curation. (2) The library profession has demonstrated significant conceptual progress in characterizing and understanding data curation both in theory and in practice. (3) There has been progress since 2006 in legitimizing library roles in data curation through formal education and certification programs as well as by integrating data curation into established library services and systems. Certain questions remain unresolved: how will data taxonomies or ontology, schemas or data models and their databases fit into data curation practices? Librarians, however, can draw on a growing body of experience and the support of a community of practice as they contribute to data curation, while researchers and those who fund research can turn with growing confidence to libraries and librarians for data curation support.

Herbert Van de Sompel. Michael L. Nelson, and Robert Sanderson have announced the release of the MementoFox Add-on.

Here's an excerpt from the announcement:

We are excited to share some news about the Memento (Time Travel for the Web) effort. Memento proposes to extend HTTP with datetime content negotiation as a means to better integrate the present and past Web. The Memento effort is partly funded by the Library of Congress.

=>The MementoFox add-on for FireFox browsers has been released. It allows time travel on the Web in a manner compliant with the Memento framework.

(*) The MementoFox add-on can be downloaded at https://addons.mozilla.org/en-US/firefox/addon/100298.

(*) Suggested Web time travels that can be undertaken using the add-on are described at http://www.mementoweb.org/demo/. They involve navigations for both the document Web and the Linked Data cloud.

=> There is also a Memento plug-in available for the MediaWiki platform. The plug-in provides support for Memento-style navigation of a Wiki's history pages.

(*) The MediaWiki plug-in can be downloaded at http://www.mediawiki.org/wiki/Extension:Memento.

(*) If you run a MediaWiki platform, please install this plug-in and let us know the URI of your Wiki.

See also: Memento project website.

Charles Beagrie Ltd has released Ensuring Perpetual Access: Establishing a Federated Strategy on Perpetual Access and Hosting of Electronic Resources for Germany.

Here's an excerpt:

This study was conducted as basis for all further steps towards a national hosting strategy. It was financed jointly by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), the Fraunhofer-Gesellschaft, the Helmholtz Association, the Leibniz Association and the Max Planck Society. The study is intended to be the starting point from which to arrive at concrete ideas and activities related to a coordinated national hosting strategy. The intensive, often controversial but always constructive and continuing discussion now expands out of the working group into the public area to be continued there in the same manner. Invited for further discussion are all stakeholders with responsibility in the German science system to establish and finance sustainable structures for perpetual access as well as long-term preservation for electronic resources.

The Digital Curation Centre has released Web Archiving.

Here's an excerpt from the announcement:

The DCC has produced a report that provides a snapshot of the state of the art of Web archiving in early 2010, noting areas of contemporaneous research and development. It should be of interest to individuals and organisations concerned about the longevity of the Web resources to which they contribute or refer, and who wish to consider the issues and options in a broad context. The report begins by reviewing in more detail the motivations that lie behind Web archiving, both from an organisational and a research perspective. The most common challenges faced by Web archivists are discussed in section 3. The following two sections examine Web archiving at extremes of scale, with section 4 dealing with full-domain harvesting and the building of large-scale collections, and section 5 dealing with the ad hoc archiving of individual resources and small-scale collections. The challenges associated with particular types of difficult content are summarised in section 6, while methods for integrating archived material with the live Web are reviewed in section 7. Finally, some conclusions are drawn in section 8.

The Blue Ribbon Task Force on Sustainable Digital Preservation and Access has released Sustainable Economics for a Digital Planet: Ensuring Long-term Access to Digital Information.

Here's an excerpt from the press release:

While much has been written on the digital preservation issue as a technical challenge, the Blue Ribbon Task Force report focuses on the economic aspect; i.e. how stewards of valuable, digitally-based information can pay for preservation over the longer term. The report provides general principles and actions to support long-term economic sustainability; context-specific recommendations tailored to specific scenarios analyzed in the report; and an agenda for priority actions and next steps, organized according to the type of decision maker best suited to carry that action forward. Moreover, the report is intended to serve as a foundation for further study in this critical area. . . .

Value, Incentives, and Roles & Responsibilities

The report of the Blue Ribbon Task Force focuses on four distinct scenarios, each having ever-increasing amounts of preservation-worthy digital assets in which there is a public interest in long-term preservation: scholarly discourse, research data, commercially-owned cultural content (such as digital movies and music), and collectively-produced Web content (such as blogs).

"Valuable digital information spans the spectrum from official e-documents to some YouTube videos. No one economic model will cost-effectively support them all, but all require cost-effective economic models," said Berman, who was director of the San Diego Supercomputer Center at the University of California, San Diego, before joining Rensselaer last year.

The report categorizes the economics of digital preservation into three "necessary conditions" closely aligned with the needs of stakeholders: recognizing the value of data and selecting materials for longer-term preservation; providing incentives for decision makers to preserve data directly or provide preservation services for others; and articulating the roles and responsibilities among those involved in the preservation process. The report further aligns those conditions with the basic economic principle of supply and demand, and warns that without well-articulated demand for access to preserved digital assets, there will be no supply of preservation services.

"Addressing the issues of value, incentives, and roles and responsibilities helps us understand who benefits from long-term access to digital materials, who should be responsible for preservation, and who should pay for it," said Brian Lavoie, research scientist at OCLC and Task Force co-chair. "Neglecting to account for any of these conditions significantly reduces the prospects of achieving sustainable digital preservation activities over the long run."

Task Force Recommendations The Blue Ribbon panel report cites several specific recommendations for decision makers and stakeholders to consider as they seek economically sustainable preservation practices for digital information. While the report covers these recommendations in detail, below is a summary listing key areas of priority for near-term action:

Organizational Action

• develop public-private partnerships, similar to ones formed by the Library of Congress
• ensure that organizations have access to skilled personnel, from domain experts to legal and business specialists
• create and sustain secure chains of stewardship between organizations over the long term
• achieve economies of scale and scope wherever possible

Technical Action

• build capacity to support stewardship in all areas
• lower the costs of preservation overall
• determine the optimal level of technical curation needed to create a flexible strategy for all types of digital material

Public Policy Action

• modify copyright laws to enable digital preservation
• create incentives and requirements for private entities to preserve on behalf of the public (financial incentives, handoff requirements)
• sponsor public-private partnerships
• clarify rights issues associated with Web-based materials

Education and Public Outreach Action

• promote education and training for 21st century digital preservation (domain-specific skills, curatorial best practices, core competencies in relevant science, technology, engineering, and mathematics knowledge)
• raise awareness of the urgency to take timely preservation actions

The report concluded that sustainable preservation strategies are not built all at once, nor are they static.

"The environment in which digital preservation takes place can be very dynamic," said OCLC's Brian Lavoie. "Priorities change, policies change, stakeholders change. A key element of a robust sustainability strategy is to anticipate the effect of these changes and take steps to minimize the risk that long-term preservation goals will be impacted by short-term disruptions in resources, incentives, and other economic factors. If we can do this, we will have gone a long way toward ensuring that society's valuable digital content does indeed survive."

About the Blue Ribbon Task Force on Sustainable Digital Preservation and Access The Blue Ribbon Task Force on Sustainable Digital Preservation and Access was launched in late 2007 by the National Science Foundation and The Andrew W. Mellon Foundation, in partnership with the Library of Congress, the Joint Information Systems Committee of the United Kingdom, the Council on Library and Information Resources, and the National Archives and Records Administration. The Task Force was commissioned to explore the economic sustainability challenge of digital preservation and access. An Interim report discussing the economic context for preservation, Sustaining the Digital Investment: Issues and Challenges of Economically Sustainable Digital Preservation, is available at the Task Force website. Please visit the website for more information about the Task Force and its upcoming symposium, called A National Conversation on the Economic Sustainability of Digital Information, to take place April 1, 2010 in Washington D.C. A similar symposium will be held in the United Kingdom on May 6, 2010, at the Wellcome Collection Conference Centre, in London. Space is limited so early registration is advised. More information is available online.

The MetaArchive Cooperative has released A Guide to Distributed Digital Preservation.

Here's an excerpt from the announcement:

This volume is devoted to the broad topic of distributed digital preservation, a still-emerging field of practice for the cultural memory arena. Replication and distribution hold out the promise of indefinite preservation of materials without degradation, but establishing effective organizational and technical processes to enable this form of digital preservation is daunting. Institutions need practical examples of how this task can be accomplished in manageable, low-cost ways.

This guide is written with a broad audience in mind that includes librarians, archivists, scholars, curators, technologists, lawyers, and administrators. Readers may use this guide to gain both a philosophical and practical understanding of the emerging field of distributed digital preservation, including how to establish or join a network.

The International Internet Preservation Consortium has launched a web archives registry.

Here's an excerpt from the announcement:

The registry offers a single point of access to a comprehensive overview of member web archiving efforts and outputs. Twenty-one archives from around the world are currently included; updates will be added as additional archives are made accessible by IIPC members.

In addition to a detailed description of each web archive, the following information is included:

• Collecting institution
• Start date
• Archive interface language(s)
• Access methods (URL search, keyword search, full text search, thematic, etc.)
• Harvesting methods (National domain, event, thematic, etc.)
• Access restrictions

The registry was put in place by the IIPC Access Working Group, which focuses on initiatives, procedures and tools required to provide immediate and future to access archived web material. The registry will also provide a basis for IIPC to explore integrated access and search in the future.

The Library of Congress has launched a digital preservation podcast series.

Here's an excerpt from the press release:

The Library of Congress presents a new podcast series, featuring interviews with prominent digital preservation practitioners and thought leaders. These podcasts offer a chance to hear experts talk about their lessons learned and goals for future projects.

The debut podcasts are interviews with Patricia Cruse and Martin Halbert. Cruse is the director of the California Curation Center, formerly known as the California Digital Library's Digital Preservation Program. She talks about her professional achievements and personal interest in making government information widely available to the public. Halbert is the newly appointed dean of libraries at the University of North Texas and one of the co-founders of the MetaArchive Cooperative. In his podcast he talks about institutional collaboration and how pooling resources helped build large-scale online resources such as the Trans-Atlantic Slave Trade Database.

The podcasts are available on the Library of Congress website and by subscription through iTunesU.