Implementation Services

DIAS (Digital Information Archiving System)

Overview

Maintain digital information for the longterm with IBM DIAS

Functionality

The DIAS solution was originally created for a project at the National Library of the Netherlands (Koninklijke Bibliotheek) in 2000.

DIAS, the IBM Digital Information Archiving System, was developed to assure long-term storage and usability of digital information. It is specifically meant to store digital information, preserve it and make it available in a usable manner over a long term (>100 years).

Figure: IBM Digital Information Archiving System (DIAS)
Legend: SIP=Submission Information Package, DIP=Dissemination Information Package,
AIP=Archival Information Package, ISIP=Interface Submission Information Package, IDIP=Interface Dissemination Information Package

DIAS is based on the Open Archival Information System (OAIS) — Reference, initially developed by NASA and since 2003 an ISO standard (ISO 14721).

DIAS provides a flexible and scalable open digital deposit solution for storing and retrieving massive amounts of digital assets (documents, photographs, videos, films , databases and complete applications like multimedia presentations or simulation models).
The DIAS-Core is industry-neutral and content agnostic based on self-describing digital assets. So DIAS can manage any kind of digital asset regardless of its content.

Its basic building block can easily be integrated with a variety of environments.
It is supported by well defined interfaces and a service-oriented infrastructure based on open standards, e.g. XML, SOAP, J2EE framework. This provides an open and standardized environment to configure the client specific ingest (Delivery and Capture) and access (Packaging&Delivery) processes.

As data standards are continuously changing and/or extended with new insight. The data model needs to accommodate these changes. Therefore DIAS has next to its core model the ability to extend the model with custom data elements. These elements can be configured for particular knowledge domains, for example DIAS-PLM, or specific clients needs like Universal Object Format, which was part of the kopal project in Germany. This provides flexibility and makes it possible to customize the solution to the individual needs of our clients.

DIAS provides both manual and automated ingest of digital assets.
But it is more than just an archiving system because it supports both physical and logical digital preservation to guarantee long-term usability of your information. Once the asset is successfully stored, DIAS manages the associated technical metadata required to evaluate usability over time. This technical metadata is used to define preservation actions, migration and/or emulation, when required.

Stored assets can be accessed either via a web-based interface (for assets having standard file types) or via a specific work environment.

Furthermore the solution includes functionalities for storage virtualization supported by IBM Tivoli Storage Manager. It manages storage, checks the media, identifies media degradation and automatically migrates digital assets from one storage pool to another.

With DIAS IBM offers you a complete deposit solution, based on a simple data model that provides flexibility for customization, scalability and extendibility to meet your current and future business needs.

Assure not only long-term storage but also long-term usability of your digital information .

DIAS- Industry solutions

Customize IBM’s DIAS according to your industry needs.

IBM is continuing to develop new solutions, which support long-term usability of information in order to meet different industry needs. Because DIAS is content agnostic, IBM can optimize its long-term usability solutions to your specific industry needs. The core component of IBM’s industry-specific DIAS implementations is the DIAS-Core.

Aerospace - Product Lifecycle Management solution

DASSAULT AVIATION has chosen IBM to provide a long-term archiving solution, DIAS Product Lifecycle Management (DIAS PLM), to ensure the long term preservation of aircraft design records.

DIAS (Digital Information Archiving System) already was a content agnostic, long-term archiving solution, mainly for the libraries and archives. DIAS PLM is now a solution that is available for different industries. Several enhancements will be made by the integration of IBM Research developments. This is an important milestone in IBM´s effort to provide generic long-term archiving solutions to different industries.

DIAS PLM will support the certification requirements for Aerospace companies as well as the long-term digital usability requirements for product lifecycle management in other industries. In order to obtain certification for its aircraft, an Aerospace company must be able to preserve its design records for the life of a program plus 3 years. In the case of aircraft designs this is in the order of 70 years. Additionally preserving the digital records has become critical with the move to pure 3D, 100% digital designs. DASSAULT AVIATION, a leader in the move to pure 3D designs, will implement the IBM solution for its Falcon 7X program. This agreement was jointly announced at the Paris Air Show on June 16th by Tom Kilkenny, General Manager, Global Aerospace and Defence Industry IBM and Jean Sass, Director of Information System, DASSAULT AVIATION.

Libraries- Archiving and Preservation Systems

Another IBM’s industry solution is DIAS 1.0, as implemented at the National Library of the Netherlands, designed to meet the special needs of libraries.
As the National Library of the Netherlands is a safe place for the digital articles of most of the world’s largest scientific publishers, they have committed themselves to provide long-term usability for these digital assets. Currently the DIAS system contains over 15 million articles and is still growing by the day. Overall DIAS makes it possible to ingest over 50.0000 articles at any given day.

The basis for DIAS version 2.0 has been the experience we gained, for almost a decade now, with DIAS 1.0.. It is currently deployed within the German kopal project. It is a cooperative solution of the German National Library (DNB), the Goettingen State and University Library (SUB), the Gesellschaft fuer wissenschaftliche Datenverarbeitung mbH Goettingen (GWDG) and IBM.

DIAS 2.0 has an advanced data model supporting preservation planning and action.
It also has superior data management capabilities through the support of a customizable data model and integrated support for XML. It provides a multi-organization set-up in which different organizations can make use of DIAS at the same instance, each with their own customizable ingest and access processes. The kopal Library for Retrieval and Ingest (koLibRI), has been developed to support customizable ingest and access processes for the interaction with DIAS within the kopal project. koLibRI can be reused as a whole or in parts and is available as open source software.

DIAS 2.0 supports the Universal Object Format, which has been defined within kopal. It is based on the Metadata Encoding and Transmission Standard (METS, Version 1.4) specification as well as on the Long-term preservation Metadata for Electronic Re-sources (LMER, Version 1.2) specification. It is used to store the structure of the metadata within the ISIP, AIP and IDIP information packages within DIAS 2.0.

DIAS – Technical features

Ensure long-term information storage and future usability with IBM DIAS.

Technology

The DIAS core technical components are developed based on the OAIS Reference Model. All DIAS versions are based on standard middleware components. The IBM DB2 Content Manager is an important standard middleware component of DIAS that contains a Library Server for the storage of metadata and a Resource Managers to store digital assets. The Content Manager has a standard interface with the Tivoli Storage Manager. The result is a technical solution with components that are shown in the figure below:

DIAS core components

Figure: IBM Digital Information Archiving System (DIAS)- Core technology
Legend: ISIP=Interface Submission Information Package, IDIP=Interface Dissemination Information Package, CM= Content Manager, TSM= Tivoli Storage Manager

• Loader
  The Loader component is realized as a Loader Scheduler and a Loader Processor.
  The Loader Scheduler checks for the arrival of Submission Information Packages (SIP's). In case a new Storage Information Packages arrives it checks which Loader Processor is available and able to handle the ingest operation.
  The Loader Scheduler manages the Loader Processors and distributes work over the active Loader Processors. Loader Processors transform the SIP object into metadata and an Archival Information Package (AIP) object. The metadata is stored on the Library Server and the AIP object is stored on the Content Manager Resource Manager. The Content Manager (CM) Resource Manager has an interface with the Tivoli Storage Manager (TSM) to delegate storage of an AIP object on Tivoli Storage Manager managed devices. It is ensured that either the whole operation is successful or the whole operation is rolled back (transaction control).
  The Loader Scheduler and Loader Processor are implemented as Java Servlets. There can be one Loader Scheduler in the system and one or more Loader Processors. Typically there is one Loader Processor per hardware server.
• Content Manager Library Server
  The CM Library Server is a standard component of the IBM DB2 Content Manager for Multiplatforms product. The Library Server contains the metadata of the stored objects. The schema of the metadata model is implemented in the Library Server.
• Content Manager Resource Manager
  The CM Resource Manager is a standard component of the IBM DB2 Content Manager for Multiplatforms product. The Resource Manager handles object storage, staging and destaging as well as purging and storage migration. It manages a cache of recently retrieved objects to reduce retrieve response time. It interfaces with the Tivoli Storage Manager to allow storage migration to TSM managed devices.
• Tivoli Storage Manager Storage Server
  The TSM Storage Server is realised with the Tivoli Storage Manager standard IBM product. It allows object storage on TSM managed devices. The CM Resource Manager uses the standard TSM backup/archive client software to communicate with the TSM Server.
• Retriever
  The Retriever component is implemented as a Java Servlet and runs on each hardware server except the server on which the CM Library Server runs. The Retriever receives retrieve requests via a URL and processes the request. It produces a DIP and invokes Java Server Pages (JSP) to return retrieve results (in XML and HTML format).
• Access Manager
  The Access Manager is a Java Library that translates the OAIS business objects into objects that can be stored in the Content Manager. It provides generalized access to the Content Manager in the form of utility classes to the Loader and Retriever components.
• Logging
  The Logging component consists of a client/server solution. The server is realized with the DB2 server. The clients are able to create logs and add them to a logging table. In addition, there is business logging in the form of events during loading and retrieval.
• Reporting
  Reporting is realized with Business Objects (BO) and BO WebIntelligence. The BO components allow creation of reports based on definitions in a Universe. The Universe is defined as a virtual view on logging tables, status tables and operational performance measures tables.
• Monitoring and Control
  Monitoring and Control are a bundle of tools and utilities for monitoring system operation, start-up and shutdown and backup and restore of the system. Operational performance measures are stored in a table to be accessible by the Reporting components.
• Administration
  Administration provides an administrative interface to the authorized Administrator to delete Assets from the system. The deletion can either be the deletion of only the AIP object of the Asset or the complete Asset. Complete asset deletion is only allowed if the Asset is at the leaf of the Asset version tree or is the only Asset. When permitted, complete asset deletion results in removing all metadata of the selected asset, including the AIP object if it was not already deleted.

DIAS Building Blocks

All DIAS components are based on standard available software components.

Current DIAS Version –Highlights

DIAS-Core Version 2.0 provides the following functions and features in the order from business to technology:

• OAIS compliant- open archiving solution;
• URN (Uniform Resource Name: RFC2141) indexed digital archive;
• Multi-organization support, allowing different organizations to share a common DIAS system;
• Support for DIAS-METS v1.0 format based on METS v1.4 and LMER v1.2 (but not limited to those metadata models; DIAS can be configured to support other models as well);
• Support for migration of assets;
• Browser based retrieval and access for supported file types;
• Standard and Custom reports in a web based reporting environment;
• Supports many storage media and devices through Tivoli Storage Manager;
• Prepared and focused for Long Term Preservation (LTP), e.g. Preservation Manager and Universal Virtual Computer (UVC);
• Supports physical preservation;
• Load balanced and scalable ingest and access;
• Distributed AIP storage;
• Integrates with standard security concepts based on LDAP standard;
• Backup and restore through proven Tivoli Storage Manager solution;
• Open, scalable, flexible solution built on open industry standards like J2EE and XML