Merging multimedia presentations and semistructured temporal data: a graph-based model and its application to clinical information.

Objective: In this paper, we focus on the issue of providing physicians with the capability of representing in a seamless way both temporal aspects of multimedia semistructured data and their temporal presentation requirements.

Background: Semistructured data are data having some structure, that may be irregular or incomplete and does not necessarily conform to a fixed schema. Semistructured data often contain the description of histories of the considered real world. The eXtensible Markup Language (XML) is becoming a cross compatible and standardized means for representing semistructured clinical data. In the field of medical informatics, there are many ongoing activities concerning XML. In the field of multimedia database systems, the topic related to the integration of several media objects (with their temporal aspects) have been considered both for data modeling and querying issues and for modeling multimedia presentations.

Methodology: We first propose the Multimedia Temporal Graphical Model (MTGM), by representing a clinical database for cardiology patients undergoing cardiac angiographies and then describe it in a formal way. We deal with the problem of expressing MTGM data by XML and of managing MTGM clinical data through an XML-based system. We provide both a technique for translating (a part of) an MTGM database into an XML document and some techniques allowing us to obtain presentations defined by means of the Synchronized Multimedia Integration Language (SMIL) from MTGM presentations.

Results: MTGM allows one to represent and store clinical information in a semistructured, temporal, and multimedia database. The physician can define multimedia presentations based on the stored data. Multimedia presentations are then stored in the same MTGM database together with temporal clinical information and are thus represented according to the same data model. A prototype based on an XML native database system has been designed and implemented.

Discussion And Conclusions: In this work we have considered the theoretical and methodological issues concerning the definition of a general data model for describing temporal and multimedia features of semistructured clinical information. Other research and application oriented features, which have not been considered in MTGM, could be investigated for completing MTGM with regard to its applicability to clinical domains: MTGM does not allow one to express times at different levels of granularities, i.e. with different time units, or with indeterminacy; besides the considered valid time, it could be interesting to manage also other temporal dimensions such as the transaction and availability times. Besides being useful for managing multimedia data stored according to widely accepted standards as MPEG and DICOM, nowadays semistructured data, and XML in particular, are becoming the most important way for expressing and exchanging medical knowledge and data: MTGM can be considered as a data model allowing the seamless representation of both (multimedia and temporal) clinical data and knowledge.