Kenton Lee

X Window System Data Multimedia Interchange Targets

by Kenton Lee

October, 1995

Copyright © 1995 Kenton Lee, All Rights Reserved.


The X Window System is now being used in a wide variety of application programs. One application area that is currently very popular is that of multimedia and graphics programs.

Users often run several such programs simultaneously. Their work may, for example, involve combining the media data produced by several X programs into a single unified database. For example, when I author World Wide Web documents such as this column, I use separate programs to edit my text, graphics, and multimedia. Naturally, I would like to use simple techniques such as drag-and-drop or cut-and-paste to integrate the various data into my WWW document.

Unfortunately, the state of the art in X inter-client communication makes multimedia data interchange somewhat difficult. Let's look at the major interchange protocols and some of their problems. The major protocols are:

While the X Toolkit and Motif APIs to these protocols are fairly easy to use, much the semantics of the data transfer are left up the source and destination programs. Because there are no widely accepted standards for transferring many types of data, interoperability problems frequently occur.

In this month's column, I'll discuss the major problem areas and techniques you can use to improve data interchange interoperability of your programs.

I won't be discussing the details of the interchange APIs. If you're not already familiar with them, some good tutorials are available over the Internet.[Benson][De Lisa]

A Review of X Data Interchange Protocols

Most programmers probably use the X Toolkit's API for the primary and clipboard selections[McCormack] and Motif's API for drag-and-drop [OSF]. While the APIs for the different interchange protocols differ, they are semantically very similar. The major run-time steps in all the protocols are:

  1. Source program is identified
  2. Selected data within source program is identified
  3. Destination program is identified
  4. Optional:
    1. Destination requests list of available targets from source
    2. Source sends list of available targets
  5. Destination requests specific target
  6. Source sends the data associated with that target
  7. Destination receives the data and processes it
  8. Source is notified that transfer is complete and may clean up

The above protocol is simple and mostly straight forward. Still, many programs, especially those from different software vendors, cannot easily interoperate. The major reasons for this probem are:

We'll discuss these two problem areas in greater detail below.

Target Names

The X Consortium's Inter-Client Communication Conventions Manual (ICCCM) defines several standard target names. Unfortunately, these cover only the most common types of interchange data, e.g.:

Because the standard list is so small, few commercial X programs attempt to interchange any more than the simplest data types. Most only interchange STRING and COMPOUND_TEXT.

Clearly, there is a need for a wider variety of standardized targets. I'll present some recommendations below.

Interchange Semantics

Just choosing target names, however, is not enough. Each target must define the full semantics for the data interchange. Some common problem areas here are:

The semantics of each target must be well defined to both the source and the destination programs. Remember that the X Window System is an open environment. Your users will expect your products to interoperate properly with products from other vendors. Since you will probably not be able to control or test all combinations of products, you should strive to use well defined, standard targets whenever possible. Proprietary targets are sometimes useful, but supporting standard ones as well is always good idea.

A Proposed Interchange Standard

Recently, I've been working with Mike Chow and Doug Rand of Silicon Graphics' User Interface Technology group to develop a standard list of interchange targets. The list of targets is presented in the next section. These cover the major data types used by commercial multimedia and graphics programs. Silicon Graphics is working with the X Consortium to add these targets to the ICCCM. Hopefully, this will occur in the next release of X.

Most of these targets are actually the names of files containing the data. Passing file names rather than the data itself has several advantages:

  1. Using files for large pieces of data is usually more efficient (in both system time and user time) than passing the data through the X server. Passing large amounts of data can significantly degrade the performance of your X server.
  2. File formats for many data types are well defined by either standards bodies or corporate sponsors. On the other hand, stream formats are less well defined, so even if the destination program receives the data, it may not be able to decode it.
  3. Software to parse files is widely available. File format converters are also widely available. Software to parse memory buffers is less widely available and often more difficult to use.
  4. Programs are often only interested in files. Why make them read and write the files when they're only interested in file names anyway?

Since X is a networked window system, there is a chance that the source and destination programs are running on different computers and thus will not be able to directly access the same file name spaces. Programs should try to avoid problems here by supporting the HOSTNAME target as well. Destination programs can use this to try to access files on remote machines.

File name targets are identified by a _FILE suffix to the target name. Names without this suffix are data streams. We only provided stream targets for data formats that have well defined stream formats. These may not be the same as simply reading the bits read out of the file.

As I mentioned in the previous section, the source and destination must agree on who owns the file when a file name is used. The safest approach, which we have adapted, is that the destination program owns the file. If the source program needs the file, it should copy it first. The destination program should delete the file when it is done with it.

Note that version numbers are part of the target names. While this does, unfortunately, create more target names, it greatly simplifies the interchange protocol.

Proposed Additions to the ICCCM Targets List

Audio Targets

Image Targets

Movie Targets

3D Targets

HTML Targets

The Future: Parameterized Selections

The X11R6 X Toolkit extends the selection API to allow the destination program to specify parameters in addition to the target name when requesting selection data (XtSetSelectionParameters() and XtGetSelectionParameters()).[McCormack] While few programs are currently using this feature, it does give us some hints about the future of X data interchange. There are probably two main ways that selection parameters will be used.

The simpler technique is to use the parameters to request only a portion of the selection. For example, you might ask for only the first 10 frames of a movie or only the first 100 characters of text.

A more general technique is to use the parameters to request transformations or conversions of the data. For example, the source program may advertise the MPEG_1_VIDEO movie data target. The destination may only be interested in MPEG movie data with a certain frame rate. The destination could request MPEG_1_VIDEO and then resample the data to the desired frame rate. Alternatively, it could send the desired frame rate as a parameter and have the source convert the data. The advantage of converting at the source side is that the source may need to perform a conversion anyway for some data types. Directly converting the desired destination parameters reduces the loss of information during the conversion. Also, many transformations reduce the size of the data, improving the efficiency of the data transfer.

Hopefully, parameter description languages will be standardized in the near future. Until then, only cooperating programs can use proprietary parameters.

Conclusion

While X has supported a variety of data interchange techniques for many years, the lack of standards for complex media types has forced programmers to use these techniques for only the simplest types, e.g., text strings. Some programs do use interchange more complex data, but they must use proprietary protocols. Programmers are reluctant to support these, since they only work with certain programs and/or on certain operating systems.

In this paper, I've presented the expanded targets list that the X Consortium is considering for the next version of the ICCCM. These should greatly enhance the usability of X programs. Please give them a try and encourage your X Consortium representatives to vote for them.


References

[Benson]
Chan Benson, "Implementing Cut and Paste in the X Environment,". A PostScript version of this document is available from ftp://ftp.x.org/R5contrib/Cut_and_Paste.ps.Z.
[De Lisa]
Gene De Lisa, "A Step by Step Introduction to Motif Drag & Drop," The X Advisor, July, 1995.
[ICCCM]
A copy of the ICCCM is included in Part III of [Scheifler]. You may also FTP a PostScript version from ftp://ftp.x.org/pub/R6untarred/xc/doc/hardcopy/ICCCM/.
[McCormack]
Joel McCormack, et al, "X Toolkit Intrinsics - C Language References," included with the X Consortium's X11R6 release.
[OSF]
Open Software Foundation, OSF/Motif Programmer's Guide, Release 1.2, Prentice-Hall, 1993.
[Scheifler]
Robert Scheifler and James Gettys, X Window System (third edition), Digital Press, 1992.


Ken Lee is an independent software consultant specializing in X Window System application software. He has been developing UNIX graphical user interface software since 1981. Ken may be reached by Internet electronic mail at kenton @ rahul.net or on the World Wide Web at http://www.rahul.net/kenton/.

Ken has published over two dozen technical papers on X software development. Most are available over the World Wide Web at http://www.rahul.net/kenton/bib.html.