A Prototype Client/Server Application for Biomedical Text

Long LR.
Thoma GR.
Berman LE,


ABSTRACT

At the Lister Hill National Center for Biomedical Communications, a research and development division of the National Library of Medicine (NLM), a prototype image database retrieval system has been built. This Medical Information Retrieval System (MIRS) is a client/server application which provides Internet access to biomedical databases, including both text search/retrieval and retrieval/display of medical images associated with the text records. The MIRS Graphical User Interface (GUI) allows a user to formulate queries by simple, intuitive interactions with screen buttons, list boxes, and edit boxes; these interactions create Structured Query Language (SQL) queries, which are submitted a database manager running at NLM. The result of a MIRS query is a display showing both scrollable text records and scrollable images returned for all of the "hits" of the query. MIRS is designed as an information-delivery vehicle intended to provide access to multiple collections of medical text and image data. The database used for initial MIRS evaluation consists of national survey data collected by the National Center for Health Statistics, including 17,000 spinal x-ray images. This survey, conducted on a sample of 27,801 persons, collected demographic, socioeconomic, and medical information, including both interview results and results acquired by direct examination by physician.

Keywords: medical, database, client/server, Internet, information, informatics

1. BACKGROUND

1.1 Medical databases

The status of databased information in current medical practice and research has been summarized by C.G. Chute of the Mayo Clinic in these words:

Medicine has emerged in the late twentieth-century as an information-intensive practice. Human memory, anecdote, and folklore can no longer train and equip medical practitioners to do their best possible job nor can they serve the interest of the patient.[1]

The storage and retrieval of this data, originally consisting only of text available through local access to mainframe computers, has evolved into systems which allow use of significantly less expensive technology, new access methods, and the storage and retrieval of data of modalities other than text. In this background introduction, we briefly discuss the status of text-only medical databases and introduce the opportunities and problems presented by the emerging use of multimedia data in medical data collections.

1.1.1 The legacy: text-only databases

Collections of text-modality medical data have been widely used for clinical and research purposes for decades. A prime example is the family of MEULARS databases maintained by NLM for providing access to bibliographic information and abstracts from the biomedical literature. Other examples are the numerous databases of patient information maintained in individual hospitals, and the large databases of medical data maintained by the Government for operation of the Medicare program. The technology for use of these databases is not static; various researchers are continuing to explore new ways to exploit existing collections of this data, to disseminate it using new communications channels, and to seek new ways to extract important information.

At NLM, for example, the modem-based access to MEULINE, which allows use of MEULARS databases, is now being augmented by Internet capability so that access will be possible through Web browsers. Researchers at Washington University[2] have attacked the problem of unifying information distributed across heterogeneous databases within a hospital for the purpose of allowing a single, comprehensible view of the data. McDonald[3] has argued that databases can definitely have an impact on clinical treatment; the Medicare database, for example, has shown that the real risks of some kinds of surgery are much larger than indicated by clinical trial data, and he identifies the need for better data analysis methods, drawing an analogy with the discoveries astronomers have made with their own tools of spectral analysis and non-linear data fitting methods.

An example of data accessible by MIRS is the National Health and Nutrition Examination Survey (NHANES) data collected by the National Center for Health Statistics (NCHS). This data, collected in surveys carried out at intervals separated by a few years, consists of many hundreds, if not thousands, of observations for each survey participant. The second such survey, NHANES II (1976-80), collected participant demographic data, as well as several categories of medical information, including allergy skin testing data, anthropometric data, audiometric test data, health history, hematology and biochemistry data, dietary information, and the results of a physician's examination. This data has been archived and distributed by NCHS, in the form of 9-track data tapes usually read by mainframe computers.

1.1.2 Multimedia databases

The most common multimedia databases currently in use are possibly the database systems used by the nation's medical centers which have Picture Archiving and Communication Systems (PACS); PACS archive and retrieve medical imagery--typically in huge volumes; as of 1992, the UCLA PACS system was acquiring 2.0 Gigabytes of image data daily[4]. A problem at PACS centers is unifying the patient information contained in the Hospital Information System (HIS) database, the Radiology Information System (RIS) database, and the imagery contained in the PACS. To our knowledge no unified approach has achieved general acceptance to the current time. In fact, the problem of interfacing these various databases has been identified as a major open task.[5]

A small number of medical databases supporting more than text data have emerged in recent years. Included among these are the ARCADIA system, developed in Milan, for the access of angiographic data and images[6], which uses the object-oriented ONTOS DBMS running on a Unix platform, and Image Engine[7],[8], developed at the University of Pittsburgh, which allows retrieval of various types of medical imagery on a MacIntosh computer.

1.1.3 The problem of access and use

The NHANES II text data discussed in Section 1.1.1 has been extensively used for research, in spite of the fact that the distribution method has been limited to 9-track tapes: a MEULINE search in 1994 returned over 800 citations corresponding to studies using this data. Perhaps not so well known is the fact that film imagery was also acquired in this survey: many participants had both cervical and lumbar spine xrays taken. In contrast to the text data use, however, NHANES films have been borrowed only nine times since 1974.

As a result of an effort to preserve and interpret the NHANES II xrays, the entire collection of 17,000 films has now been digitized, making the image data and the text data available in digital form for the first time. An opportunity now exists for making both the images and the text available as an integrated set.

The NHANES II collection is representative of the problem posed by sets of multimodality data with unexploited informational potential because of the lack of a system for retrieving and analyzing such data in an integrated manner on practical hardware platforms. We believe it is not unique in that respect. The MIRS system has been created to remove the barriers which have impeded the use of such collections.

2. MIRS GOALS

The goal of MIRS is to "Deliver the information": to streamline the accessibility of collections of medical text and visual data which have been up to now inaccessible, or accessible only with unintegrated tools. The emphasis of MIRS is medical information delivery--we conceive of MIRS as being fundamentally an information vehicle--hence we place great importance on incorporating all the necessary features, interfaces, and software hooks which will allow MIRS to deliver data judged to be of significant medical informational content by specific targeted user groups. A second general goal of MIRS is to be as generic as possible: MIRS should provide access to many classes of medical data.

Specific subgoals to achieve these general ends are as follows:

  • a. Support access using the client/server method across the Internet.
  • b. Support access to a wide class of medical databases containing both text and images. It should be relatively painless to add new databases to MIRS.
  • c. Provide a powerful database engine to support complex query capability with high performance.
  • d. Provide a user interface which allows users to create queries with minimal effort and which does not require memorization of database structure, table or field names.
  • e. Provide a communications mechanism which will fully exploit available physical bandwidth.
  • f. Provide a query results interface which integrates both text and images and associates text and related images with visual cues.
  • g. Provide capability to rapiULy review large numbers of text records and/or images in the database.
  • h. Provide capability to save results and review later.
  • i. Provide capability to analyze results to extract the important informational content for achieving user end goals, or to export the query results to familiar analysis tools.
  • j. Provide image processing capability to aid in acquiring visual information.

3. MIRS DESIGN

3.1 Overview

In this section we discuss how MIRS has been designed to address each of the subgoals listed in Section 2.

  • a. Client/server Internet access. MIRS is designed with client/server architecture using TCP/IP protocol for communication.
  • b. Support for access to a wide class of databases. MIRS is designed to support access to generic medical text/image databases; this is perhaps the most significant characteristic of MIRS and is discussed in detail in Section 3.2 below.
  • c. Database engine for query support. MIRS uses the Illustra DBMS. Illustra is a commercial product described by its manufacturers as an "object-relational" database management system. Besides the capabilities of an ordinary relational DBMS, Illustra incorporates features defined in SQL 3, including user-defined functions, support for complex data (such as R-tree search capability), and object-oriented features such as inheritance of properties of one table from another[9]. Illustra uses a client-server architecture with standard TCP/IP communications protocol.
  • d. User interface for minimal effort querying. MIRS provides a Graphical User Interface which allows the user to build SQL queries by using buttons, list-, and edit-boxes to reduce the query-building process to a small number of keyboard and mouse actions. Queries can be saved to files and later recalled for direct use or modification. Further, the MIRS screens determine at run-time which database is being used and provide the user the information and choices appropriate for that database. See Section 3.3 for additional details.
  • e. Communications mechanism which exploits available physical bandwidth. MIRS uses the TCP/IP communications channels built into the Illustra DBMS for communication of low-bandwidth data (text data). For high-bandwidth data, MIRS uses a specialized sockets interface developed inhouse, described in Section 3.4 below.
  • f. Integrated display of images and text data resulting from queries. MIRS displays its outputs on a screen which presents multiple images and shows text record results associated with the image marked as the "current image" by an image cursor. The MIRS screens are described in detail in Section 3.5 below.
  • g. Capability to review rapiULy large numbers of text or image records. The user may move rapiULy through the images or the text records, and MIRS will maintain screen indicators which show the user which image is associated with the text record currentlybeing displayed. Again, see Section 3.5 for a description of the MIRS screens.
  • h. Capability to save and review results. The current MIRS provides very limited capability to save query results, and no review capability. We recognize full save/review capability as an essential part of a practical system which must be provided.
  • i. Capability to analyze results. MIRS provides no analysis capability at the present time; capability to do statistical analysis of query results and/or export MIRS results to analysis packages are features to be provided.
  • j. Image processing capability. Capability for contrast enhancement of the MIRS images to maximize the informational value of the images is planned.

3.2 Generic nature of the MIRS databases

MIRS is intended to allow for the easy addition of collections of medical data consisting of both text and images, other than the NHANES II data set. A MIRS database must conform to these rules:

  • a. There must be a key field in the database to which all other fields are related.
  • b. There may be an arbitrary number of non-hierarchical data fields related to this key.
  • c. There may be an arbitrary number of hierarchy trees related to this key. In the current MIRS, the number of hierarchy levels in each tree is limited to three.
  • d. There may be an arbitrary number of image files related to this key.

In the implementation of a MIRS database, the non-hierarchical data becomes one relational database table; each hierarchy tree becomes one table; and data which points to image files is collected in one table. Figure 1 gives a picture of a generic MIRS database. This is the general structure to which any collection of text/image data has to conform to become a database accessible to the MIRS client.

Structure of a MIRS database showing an arbitrary number of hierarchy trees.Figure 1.

For illustration, consider how the NHANES database fits into MIRS:

  • a. All NHANES data fields are related to a single key, the Survey Person (SP) Number (the unique numeric identifier of the person to which all the fields in an NHANES record apply).
  • b. NHANES non-hierarchical data which is related to the SP Number are the demographic data: age, sex, race, nationality, region of country, height, and weight. There are many other items of demographic data collected in the NHANES II survey; a subset of the total collection of items are in the current MIRS NHANES database.
  • c. One hierarchy tree is in the current MIRS NHANES database. This is the Physician's Exam data, which is related to the SP Number (it is the results of the physician's exam for that person). This data uses all three hierarchy levels supported by MIRS. Traversing one path up the tree for illustration, we move from "Back" to "Abnormality or Curvature" to "Kyphosis"; conceptually, a value (such as "Yes") would be stored in the database at the terminal point of this path, indicating whether the physician's examination determined that the person had the medical condition of kyphosis. The NHANES II survey collected data which could be represented by several additional hierarchy trees in the MIRS database, such as Anthropometric Data, Medical History Data, and Medical Questionnaire Data.
  • d. In the MIRS NHANES database, each person may have one associated cervical spine x-ray image, one associated lumbar spine x-ray image, both, or none. Hence, up to two images are related to each SP Number. The specific information about how many images, and which types, are associated with any particular SP Number is carried along with the non- hierarchical data fields.

Figure 2 shows the NHANES II database realization of the generic MIRS database structure.

Graphic representation of part of MIRS NHANES II database structure.Figure 2.

3.3 Generic nature of the MIRS query-building screen

The MIRS screen for constructing queries is called the Search Database screen. The screen is logically divided into two parts: a top part where the user builds relational clauses (e.g. "age > 45", "kyphosis = yes") and a bottom part, where the user combines these clauses with Boolean connectors to build an SQL query which is then submitted. The top part of the screen has list boxes containing the field names, relational operators, and field values appropriate for the particular MIRS database being used. Also, each of these list boxes on the screen is labelled appropriately for the particular MIRS database being used. If the user selects another MIRS database (which is a MIRS option on the initial login screen), the labels on these boxes as well as their contents change appropriately for the new database.

The ability of this interface to adapt to a new database on-the-fly is provided by three design features:

  • a. All GUI components are pre-generated and positioned on the screen at compile time. This includes all list boxes used in the top part of the screen. Most of these components are marked as "hidden until at run-time a particular user action causes them to be shown on the screen. Other user actions may cause them to be hidden again. See Figure 3.
  • b. The interaction between the GUI components and user actions are mediated by event-hanULers in the program code. An event-hanULer exists for each component which needs to be dynamically shown or hidden. Whenever the user takes an action requiring such dynamic showing or hiding, the appropriate event-hanULer is called.
  • c. For all MIRS databases, the (database-specific) contents of each of the list boxes on the screen and the labels on the boxes are contained in an ASCII script which is read in by MIRS upon program initialization. This script also contains information defining which other list boxes should be shown as a result of a selection made in one list box.

Schematic representation of the MIRS Search screen.Figure 3.

Figure 4 represents the interactions between the ASCII script, the event-hanULing code, and the dynamic display on the screen.

Intersection of the MIRS script file for screen config., client code, and Search Database screen.Figure 4.

3.4 MIRS communication features for image communications

In order to increase the performance of MIRS in the delivery of image data, we have incorporated into MIRS an experimental communications method, the "multisocket" file delivery method, developed at NLM.[10] This method uses multiple TCP/IP channels to segment a file, transmit it, and reassemble it at the receiving end. Because overall wait time for acknowledgments is reduced in this method, higher transmission rates are achievable than when using conventional one-channel TCP/IP methods, such as FTP. Internet tests we have performed with this method have achieved speed-ups of 2.5:1 or more as compared with conventional FTP.[10] The multisocket method has been implemented as an experimental "multisocket server" which is currently running on two computers in our lab. An Application Program Interface (API) has been written for this server, and it is through use of this API that MIRS uses the multisocket method.

The interaction of MIRS with the multisocket servers is as follows. When the user has formulated and submitted an SQL query, the MIRS client (which may be situated remotely to NLM on the Internet) sends the query to the MIRS server at NLM. The MIRS server responds by querying the appropriate MIRS database for the text data which satisfies the query, along with the file names of the images required to satisfy the query. This list of file names is returned to the MIRS client. The MIRS client then contacts the multisocket server on the NLM machine which contains the particular images named; at present, large multi-megabyte images are kept in magneto-optical storage on a jukebox hosted by the liberty machine; small images are kept in magnetic storage on a RAID system hosted by the archive machine. This is illustrated in Figure 5.

MIRS communications archetecture.Figure 5.

3.5 Generic nature of the MIRS query-results screen

Query results are presented on the MIRS Database Information screen. The top part of this screen is a horizontally-scrollable, multiple-image display; the bottom part of the screen is a vertically-scrollable text display of a single text record. Images may be quickly viewed by using a slider bar. Text records may be viewed in sequence by using the next record/prev record buttons. A control at the upper left allows the user to change the image display to show other types of images associated with the text records. An image cursor is implemented as a colored border which surrounds the image selected by the user as "current", by simply clicking the mouse on that image. When an image is identified as "current", the text record display immediately updates to provide the descriptive text information appropriate for that image. Similarly, when the user uses the next record/prev record buttons to move through the text records, the image cursor automatically moves to identify the image corresponding to the text record on display.

This screen currently only has capability to display grayscale images in 128 shades of gray. Each image which appears on the Database Information screen must be preformatted in a MIRS-compatible format; at present, MIRS only recognizes the GIF file format for these images. The image sizes MIRS will support on this screen are arbitrary, within bounds determined by the screen spatial resolution, and by the fact that the screen space needs to be shared with the text record display and graphical controls. Within these constraints, MIRS has the capability to display "generic" gray-scale images in GIF format.

In addition MIRS provides a Full Resolution Display screen. This capability is for showing a high-resolution version of a single image which appears on the Database Information screen. This is applicable for images such as the cervical and lumbar spine xrays, which have large spatial resolution (1463x1755 and 2048x2487 pixels, respectively), preventing multiple images from being shown on the Database Information screen in full resolution. Instead, Database Information shows lower resolution versions of such images; the user may select any of these images and use a graphical control to bring up the Full Resolution Display to show the full resolution version.

3.6 Examples of MIRS screens

Figure 6 shows an example of a MIRS Search Database screen for the NHANES database. In this example, the user has created a query to search for all database records corresponding to black females at least 45 years of age having the spinal condition of kyphosis. Several list boxes have been created on the screen in response to user actions: in the DEMOGRAPHIC FEATURES part of the screen, the user has selected "Sex" from the Feature list; this caused the list box titled Value to appear to the right; from this Value list, the user has selected "Female"; likewise, in the EXAMINATION FEATURES, the user has selected "Physician's Exam" from the list box titled Tape; this caused the Topic list box to appear; selecting an item from the Topic list caused the Subtopic list box to appear, and so forth.

As the user makes selections from this lists, relations (such as "Age > 45") appear in the Query Construction Area at the bottom left. Finally, the user joins the relations with Boolean connectors to form the query; in this case, four relations have been joined with AND. The final query appears in the Constructed Query box.

Example of MIRS Search Database screen.Figure 6.

Figure 7 shows an example of a MIRS Database Information screen for the NHANES database. This screen is the output of a query submitted by the user. In this case the query was a search for all database records corresponding to white females at least 50 years of age. Cervical x-ray images are shown in the upper part of the screen in a horizontally-scrollable display, while the text corresponding to the "current image" (highlighted with an image cursor which frames the image) appears in the lower part of the screen. The text may be scrolled vertically to show additional textual data.

Example of database information screen.Figure 7.

3.7 MIRS client platform

The current MIRS client is designed to run on a Sun Sparc 10 class workstation. The initial target user community for MIRS is the research community in the medical school/university environment. Additional MIRS access may be provided in the future on a wider class of platforms, or by using platform-independent means such as the World Wide Web.

4. CONCLUSION

MIRS is a prototype client/server system with the goal of providing a solution to the problem of extracting medical information from remote collections of medical text and image data. MIRS provides a generic method for data organization, retrieval, presentation, and analysis which is intended to apply to a wide class of medical datasets. MIRS capabilities to retrieve and present data have been implemented for a set of NHANES II survey data, and work is planned to add capabilities for data analysis, saving of results, and image processing, as well as the addition of other databases to MIRS.

5. ACKNOWLEDGMENTS

We wish to acknowledge the efforts and dedication of Leif Neve, Patty Bailey, and Yun Cai, of Century Computing, Inc. in developing the MIRS system.

6. REFERENCES

1. C.G. Chute, "Clinical data retrieval and analysis: I've seen a case like that before," Annals of the New York Academy of Sciences, (670), pp. 133-140, 1992.

2. K.A. Marrs, S.A. Steib, C.A. Abrams, and M.G. Kahn, "Unifying heterogeneous distributed clinical data in a relational database," Proceedings of SCAMC 1993, pp. 644-648.

3. C.J. McDonald and S.L. Hui. "The analysis of humongous databases: problems and promises," Statistics in Medicine, Vol. 10, pp. 511-518, 1991.

4. A.W.K. Wong and H.K. Huang, "Subsystem throughputs of a clinical picture archiving and communications system," Proceedings of SPIE Medical Imaging VI, Newport Beach, CA. 1992.

5. H.K. Huang, R.L. Arenson, S.-L. Lou, A.W.K. Wong, K.P. Andriole, T.M. Bazzill, and D. Avrin, "Multimedia in the radiology environment: current concept," Computerized Medical Imaging and Graphics, Vol. 18, No. 1, pp. 1-10, 1994.

6. F. Pinciroli, C. Combi and G. Pozzi. "ARCADIA: A system for the integration of angiocardiographic data and images by an object-oriented DBMS," Computers and Biomedical Research, Vol. 28, pp. 5-23, 1995.

7. H.J. Lowe, B.G. Buchanan, G.F. Cooper, and J.K. Vries. "Building a medical multimedia database system to integrate clinical information: an application of high-performance computing and communications technology," Bulletin of the Medical Libraries Association, Vol. 83, No. 1, pp. 57-64, January 1995.

8. H.J. Lowe. "Image Engine: an object-oriented multimedia database for storing, retrieving and sharing medical images and text," Proceedings of SCAMC 1993, pp. 839-843.

9. M. Stonebraker. "Object-relational DBMS: the next wave," Illustra Information Technologies, Inc, Oakland, CA, 1995.

10. L.R. Long, L.E. Berman, G.R. Thoma. "An Application-Level Technique for Transmission of Large Images on the Internet," Proceedings of SPIE: Multimedia Computing and Networking 1995, Vol. 2417, San Jose, CA, pp. 116-129, February 6-8, 1995.


Return to top Return to top