Kenton Lee

MediaWarehouse: Integrating Multimedia Data Through The X Window System

by Kenton Lee

Presented at the X Consortium's Technical Conference, February, 1996.
Copyright © 1996 Kenton Lee, Palo Alto, California, USA. All rights reserved.

Key words: X Window System, X11, Motif, multimedia, plug-in viewers, database


Abstract

MediaWarehouse is a new application program for Silicon Graphics' X Window System desktop. Its purpose is to help users manage their multimedia data.

Multimedia data includes the audio, movies, images, 3-D graphics, text, and hypertext. Users often copy this data from one X-based application to another. MediaWarehouse helps users interchange multimedia data between applications, save multimedia data into a persistent database, and browse through the database.

This paper briefly reviews the major features of MediaWarehouse and details some of the more interesting X-related components.


Contents

Introduction

In recent years, many X Window System applications have also processed multimedia data such as audio, movie, images, 3-D graphics, text, and hypertext. Users often want to use the same pieces of multimedia data in many different applications. The MediaWarehouse is a tool to help users organize their multimedia data and interchange it between multimedia X Window System applications.

The MediaWarehouse contains four major components:

This paper has two goals:

  1. First, to show you how a program like MediaWarehouse can help you with your work.
  2. Second, to show you the designs of the major X-oriented features. Even if you do not choose to use MediaWarehouse, you may be able to use similar designs in your products.

This is not a tutorial on using MediaWarehouse. Complete documentation on MediaWarehouse's functionality is included with the product.[Ferguson]

User Interface

The MediaWarehouse graphical user interface (GUI) was developed using the X Window System[Scheifler], the X Toolkit intrinsics[McCormack], the OSF/Motif widget set[OSF], and the ViewKit object-oriented application framework[ViewKit]. In this section, we'll present a brief overview of the MediaWarehouse's major features.

Figure 1 shows MediaWarehouse's GUI. The GUI is designed to be small enough to remain fully visible on the user's screen, even when he or she is simultaneously interacting with other applications that take up most of the screen's real estate. On a typical UNIX workstation, MediaWarehouse's main window uses only about ten percent of the screen space.


Figure 1: MediaWarehouse User Interface
[MediaWarehouse user interface]

The primary user concept in the GUI is the data object. Data objects are pieces of multimedia data that users may interchange between applications. The data may be stored in files or may simply be memory that is passed between applications via inter-process communication (IPC). Within the GUI, one data object may be selected at a time. Data objects may be grouped into collections called shelves. One shelf may be selected at a time.

In Figure 1, the menu bar a the top of the GUI presents some of the major functions of the MediaWarehouse, including:

Beneath the menu bar is a viewer area that shows the contents of the currently selected data object. The MediaWarehouse provides a variety of media-specific viewers that are described in the Viewer Framework section below.

Beneath the viewer is an option menu that allows the user to select a shelf. The contents of the selected shelf are displayed iconically in the lower portion of the GUI. The shelf is the drop site for importing data objects via OSF/Motif drag-and-drop. The individual data objects may also be selected or exported via drag-and-drop. Other data object import and export mechanisms are discussed in the Multimedia Data Interchange section below.

The icons identify the data types of the individual data objects. The icons are the same as those used in Silicon Graphics' Indigo Magic desktop environment.[Smith] Indigo Magic provides a set of standard icons and data types and users (and application developers) can add additional icons and data types as well.

The user's shelves and data objects are stored in MediaWarehouse's database between user sessions. The database is discussed in more detail in the Database section below.

MediaWarehouse users typically create separate shelves for their major projects or tasks. They initially add groups of objects to the MediaWarehouse, then later import new objects as they are created. Objects can then be exported to other applications when they are needed. MediaWarehouse's viewing and searching features allow the user to quickly find the objects they want.

The following sections discuss the designs of the major MediaWarehouse features.

Multimedia Data Interchange

Users often use the same pieces of multimedia data in more than one application program. They typically interchange the data between the applications using:

MediaWarehouse does support file import and export for applications that must use file-based techniques. For most users, however, the file system adds a cumbersome extra step to the data interchange procedure. Most users will probably prefer to use one of the direct interchange techniques.

The last three techniques are based on the protocols defined in the X Consortium's Inter-Client Communication Conventions Manual (ICCCM).[Rosenthal] These techniques interchange data between X clients using protocols based on target type names. In order for these protocols to work with multimedia data, the participating applications must agree on a set of target type names that describe the multimedia data types.[Lee]

The ICCCM defines target names for simple types such as STRING and COMPOUND_TEXT, but, unfortunately, not for multimedia data types. To support multimedia data interchange, Silicon Graphics has proposed a set of multimedia target type names to add the ICCCM. The list includes most of the popular audio, image, movie, 3-D graphics, and hypertext data types. SGI's multimedia applications are already using many of these targets, thus allowing users to easily interchange multimedia data by any of the ICCCM techniques.

Data objects are stored in MediaWarehouse's database with their target type names. The target type is determined either by the ICCCM import target type or, in the case of imported files, by applying heuristics to the file contents (the file typing heuristics are part of Indigo Magic[Smith]). The target type names in the database are used both when exporting the data objects and when viewing them with MediaWarehouse's in-line viewer.

GoldenGate Automatic Data Format Conversion

One common problem that users of multimedia data have with inter-client data transfer is format incompatibility. The source of a drag-and-drop or cut-and-paste operation may be able to supply the data in a particular format (e.g., GIF image), but the destination may only accept a similar, but incompatible format (e.g., TIFF image). Since the advertised and desired targets do not match identically, the standard X Window System drag-and-drop or cut-and-paste protocols will fail. This is especially a problem with sophisticated commercial software that often support only proprietary data formats.

Silicon Graphics has solved this problem by extending the X Toolkit and Motif with the GoldenGate automatic data conversion framework[Rand]. When the user initiates a drag, cut, or copy, GoldenGate augments the source application's advertised targets list to include all the types to which GoldenGate can convert the original data. The destination application receives the larger augmented list, thus allowing successful data transfers in many more cases.

GoldenGate uses an extensible set data format converters. Many common types are supported in the base converters, but software developers are encourage to add their own to increase the usability of their software.

The GoldenGate framework is built in to Silicon Graphic's Motif and X Toolkit shared libraries, so applications do not have to provide any special code to use it. They may optionally enable or disable GoldenGate through an application resource. MediaWarehouse has this support enabled by default.

Viewer Framework

One of MediaWarehouse's most popular features is the in-line viewer. Whenever a user selects a data object, MediaWarehouse loads an appropriate type-specific viewer and displays the data object in the viewer. The viewers are very lightweight, allowing users to quickly preview objects before exporting them to other applications.

Figure 2 shows some examples of the viewer user interfaces. Note that the viewers are all resizable, and users may choose to resize them for viewing large objects.


Figure 2: Example Viewer User Interfaces
[Inventor 3D viewer]

Inventor 3D Viewer

[Audio viewer]

Audio Viewer

[HTML viewer]

HTML Viewer


MediaWarehouse is distributed with viewers for the following data types:

The viewers are fast, light weight, and read-only. Users must export the data to a full featured application for editing or other complex operations. Still, the viewers are very useful. For example:

The media viewers are implemented as System V dynamically loaded shared objects (DSOs). The DSOs are loaded at run time when needed for a particular media type. Since the viewers are dynamically loaded, users may change the viewers, add new ones, or use the viewers in other applications.

As mentioned in the previous section, MediaWarehouse uses the data objects' target type names to determine which viewer to use to view a particular data object. MediaWarehouse uses an X Toolkit[McCormack] resource file to map the target names to DSO path names. A portion of the default resource file is shown in Figure 3. Since the X Toolkit reads in the resource file at run-time, users can use the standard X Toolkit resource mechanisms to add to or override the mappings.


Figure 3: Viewer Mapping Resource File
STRING.viewer: /usr/lib/mediaw/libTextViewer.so
HTML.viewer: /usr/lib/mediaw/libHTMLmediaw.so
INVENTOR_2_1.viewer: /usr/lib/mediaw/libInventorViewer.so
VRML_1_0_FILE.viewer: /usr/lib/mediaw/libInventorViewer.so
SGI_RGBIMAGE.viewer: /usr/lib/mediaw/libImageViewer.so


The programming interface for the viewer DSOs is fully specified in the MediaWarehouse's documentation. Each viewer DSO provides four functions:

The MediaWarehouse loads and binds these functions at run time using the path name found in the resource file. If no viewer DSO is specified or if the DSO cannot be loaded, MediaWarehouse instead uses a simple default viewer that just displays that target type name and the data object's size (in bytes).

Database

The MediaWarehouse currently uses a simple database with basic persistence and searching facilities. We're considering a much more sophisticated asset management system for the future.

The database stores each data object's name, data type, and keyword information. Users may edit the name and keywords via a dialog box. Users may also search for data objects by name and keyword.

The database is saved to disk between user sessions. During user sessions, any changes (data object additions, deletions, edits) are periodically and automatically saved using X Toolkit timeout procedures.

We're considering more powerful database features for future versions of MediaWarehouse. For example:

Reusability

MediaWarehouse was designed in an object-oriented fashion with reusability and upgradeability in mind. Each of the major components (viewer framework, data interchange framework, database) is self contained.

The viewer framework, in particular, is now being used by several other Silicon Graphics projects. For example, future versions of the file selection box component and the directory viewer application are using the viewer framework. We've also developed an interface to allow the Netscape Navigator to use the viewer framework as a plug-in.

Of course, the Golden Gate framework is also widely used by other applications.

Conclusion

MediaWarehouse is a new application program for Silicon Graphics' X Window System desktop. It helps users to manage their multimedia data. MediaWarehouse includes four major components:

This paper has focused on the designs of the major X-oriented features. More complete user documentation is available separately.[Ferguson]

References

[Ferguson]
Wendy Ferguson, MediaWarehouse User's Guide, Document Number 007-2851-002, Silicon Graphics, Inc., 1995.
[Lee]
Kenton Lee, "X Window System Multimedia Interchange Targets," The X Advisor, Vol., No. 5, October, 1995.
[OSF]
Open Software Foundation, OSF/Motif Programmer's Guide, Release 1.2, Prentice-Hall, 1993.
[McCormack]
Joel McCormack, Paul Asente, and Ralph Swick, X Toolkit Intrinsics - C Language Reference, included with the X Consortium's X11R6 release.
[Rand]
Douglas Rand, "GoldenGate: Automatic Conversion and the MediaWarehouse," The X Resource, Issue 16, December, 1995.
[Rosenthal]
David Rosenthal, Inter-Client Communication Conventions Manual (ICCCM), included in the X Consortium's X11R6 release. A copy is also included in Part III of [Scheifler].
[Scheifler]
Robert Scheifler and James Gettys, X Window System (third edition), Digital Press, 1992.
[Smith]
Amy Smith and Betsy Zeller, "Indigo Magic: An (un)Common Desktop Environment," The X Journal, Vol. 4, No. 3, January, 1995.
[ViewKit]
ViewKit is an object-oriented C++ framework for developing Motif-based application programs. It is available from Silicon Graphics and ICS.

Trademarks

MediaWarehouse, Indigo Magic, and ViewKit are trademarks of Silicon Graphics, Inc. X Window System is a trademark of the X Consortium. OSF/Motif is a trademark of the Open Software Foundation. UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company, Ltd.


Author Information

Kenton Lee is an independent software consultant specializing in X Window System and OSF/Motif software development. Ken may be reached on the World Wide Web at http://www.rahul.net/kenton/index.shtml or by electronic mail to kenton @ rahul.net.

Ken has published over two dozen technical papers on the X Window System. Most are available over the World Wide Web at http://www.rahul.net/kenton/bib.html.

The work described in this paper was done for the User Interface Technology Group at Silicon Graphics, Inc. in Mountain View, California. The User Interface Technology Group is managed by Mike Chow and Kumar Vora. Engineers in the group include Donald Beaudry, David Curley, Alastair Gourlay, and Douglas Rand.


Please send me your comments on this paper:

Name: E-mail:

[X Consulting] [Home] [Mail] [X Papers] [X WWW Sites]