Published in The X Journal July, 1994.
Copyright © 1994 Kenton Lee. All rights reserved.
Key words: X Window System, X11, software engineering, software development tools, programming.
As you surely know by now, the X Window System has become very popular in recent years. It is now the base for almost all commercial UNIX products with graphical user interfaces
Dozens of tutorials have been written to help the beginning X Window System programmer and many of these books are very good. There is, however, a very large gap between the needs of a new X programmer and the needs of sophisticated commercial software products. Unfortunately, few texts discuss the problems an X programmer is likely to face while developing high-quality commercial products.
Over the last couple of years, Steve Mikes (editor of The X Journal) and I have been discussing this literature gap. I've written a few articles for The X Journal on X software engineering issues[1-3]. Recently, Steve and I decided that a regular series of short articles on this subject would also be valuable. This is the inaugural article in that series.
Future articles in this series will discuss a variety of software engineering issues related to X-based application software. Some possibilities, in no particular order, include:
For the first subject in this series, I'll discuss some development environment basics. While often given little thought, creating a solid development environment is often the single most important part of any software development project. Even the most experienced X programmers occasionally get a little sloppy with environment issues and this often causes many difficult to diagnose software problems.
Development environment problems do not occur only in X software, of course. Many such problems occur frequently in X application development, however, and certain development environment tools and techniques were designed specifically by and for X programmers. I'll discuss several of both below.
I'll assume you're developing your code using a C or C++ compiler and the UNIX operating system (by far the most popular X programming environment). Your code is probably divided into a number of source code files. You'll obviously be using the X libraries; you may also be using several system libraries and/or libraries that were locally developed.
As your project progresses, these components will be constantly changing. You'll write new code or change old code. You may upgrade compilers or system libraries. You may add new local libraries.
Our goal is to generate correct and reproducable executable programs, even when several programmers are simultaneously working on the programs. Of course, you don't want to rebuild everything from scratch every time a module changes: a large program may take an hour or more to build from scratch. How do you decide which components need to be recompiled when a change occurs?
To avoid confusion, lets first define some terminology.
The most important development environment problem is that of maintaining the integrity of your libraries and executables during incremental development. Frequently (perhaps thousands of times in the course of a project), you'll change some of your source files and want to re-build your executable program. While recompiling all your source files is usually sufficient to generate correct executables, this technique is much to slow for use on most computers. Instead, most programmers strive to recompile only the source code that has changed since the last compilation.
Most X programmers use make (or imake, discussed below) to figure out which object, library, and executable files need to be recompiled whenever a code or header file is changed. make contains a simple set of built-in rules for determining which compiled files need to be recompiled. For example, make will recompile foo.o if the corresponding foo.c has been modified more recently.
Unfortunately, the rules built in to make are not sufficient for a correct build. The default build rule is to compile a .c file if it has a more recent date than a .o file with the same base name. If, for example, your .c file has a different name from the .o file, make will not properly build the .o file unless you add a new rule to your Makefile.
A more common problem area is that of header files. If a .c file includes one or more .h files, the corresponding .o file will frequently be incorrect if any of the .h files are changed. In this case, too, you must add rules to your Makefile specifying that this .o file depends on the .h files.
Your Makefile must also contain rules (and the corresponding dependencies) for combining .o files into libraries and into executables, of course.
Any UNIX programmer's tutorial should discuss the basics of creating Makefile dependency and rules specifications. Rather than repeating everything here, I'll let you study those.
The header file dependency problem mentioned in the previous section is, by far, the most serious dependency problem. Well written X applications tend to be highly modular. If each module defines its interfaces in a header file, there will probably be many header file dependencies.
To help manage this problem, the X Consortium's software distribution includes a great utility called makedepend. makedepend will search your .c files to find header file dependencies, including nested header files, and add the appropriate dependency rules to your Makefile. makedepend accepts the same command line arguments as C and C++ compiles, including -I search paths, so it is very easy to integrate into your Makefiles.
makedepend is not the only automatic dependency generation tool, though it is one of the best. Another popular alternative is the -M command line option that some C and C++ compilers support.
Dependencies aren't the only problem you could have with header files. Another common problem is that of confusion between multiple versions of header files.
If you're like me, you'll probably have more than one version of your project's header files lying around. You might also have more than one version of the X header files, e.g., one for your OpenWindows environment and one for your Motif environment.
When you compile your program, you should specify a search path for header files that allows the compiler to find the right files. Unfortunately, search paths are easy to get wrong.
If you suspect that you're including the wrong header files, there are a couple of easy ways to find out which ones you're really using. First, if you used makedepend to generate your dependencies, you can just look in your Makefile. The dependency is the first header file that makedepend found. This is usually the same header file that your C or C++ compiler preprocessor would find.
A more accurate, though more time consuming technique is to use the -E command line option available on most C and C++ compilers (along with your normal compiler command line options). The -E option tells the compiler to run the preprocessor only and write the results to the standard output. The preprocessor output will tell you exactly which header files you're using.
C and C++ compilers automatically search /usr/include for header files and /usr/lib for libraries. X libraries and header files, however, are often installed in non-standard locations, thus requiring additional Makefile rules and/or macros for all your applications. If you build your applications on several different platforms (hardware, operating system, etc.), each with different non-standard X library and header file locations, you'll soon find that a large percentage of each of your Makefiles is dedicated to these rules and macros.
Also, you'll often want to use different compiler options (or different compilers) on different platforms, further complicating your Makefiles.
Since most of the above is really the same information repeated in all your Makefiles, a better solution would be to move all of these definitions into separate configuration files. The imake program, supplied with the X Consortium distribution, does just this and more. Using imake, you create Imakefiles containing a minimal amount of information about your application's source files and build rules. Imake converts these Imakefiles into regular Makefiles that you can use with make.
Imake can get complicated, so I won't give a tutorial here. Instead, I recommend that you read Paul DuBois' tutorial, which is fairly complete.
Most UNIX systems come with a basic configuration management system such as RCS or SCCS. Others are available, some with more advanced features, both free over the Internet or as commercial products. All non-trivial software projects should take advantage of these (and most probably do).
One problem with these systems, or at least the more popular ones, is that they can be very wasteful of disk space in some cases. If several programmers are working on the same files, they usually must create their own copies of each of the source files. The same problem occurs if one programmer is maintaining multiple versions (perhaps for different platforms) of the same source code. If the source code is large, as it is for the X Window System as a whole, the amount of wasted disk space is very large.
The X Consortium distribution includes a simple tool called lndir that helps avoid this problem. lndir creates a shadow source tree, consisting of individual symbolic links to the real source files. You can easily build different versions of your application in the shadow trees.
Some organizations have devloped simple shell scripts that extend this concept to integrate the shadow trees with their configuration management systems. With these scripts, they can check files out and back in while working in a shadow tree. I'm not aware of any freely available versions of these scripts, but a good shell programmer should be able to put something together pretty quickly
There are more sophisticated systems are available to solve this, and other, configuration management problems. Many, however, suffer from too many features, resulting in excessive complexity and poor performance. The above utilities provide a large benefit in a very simple package.
One last build environment problem that I'd like to mention is that of shared libraries. Shared libraries are very popular with developers because they save disk space and speed up the application linking process. Unfortunately, dependency based build tools usually aren't well integrated with shared libraries, so you may have problems if you're not careful.
One common problem is that you'll often find (sometimes after a lot of searching) that your application is not using the library that you thought it would. Another problem is that you may update header files and make the shared libraries obsolete, but foreget to rebuild the shared libraries.
Unfortunately, shared library implementations vary quite a bit from platform to platform, so I can make only a few simple, general suggestions on debugging shared library problems:
Build environment problems can be very common when developing sophisticated application software, such as many X applications. I've listed some of the more common problems and ways to avoid them. Fortunately, the X Consortium has developed and distributed a set of tools that can help you avoid them, too. The tools are simple, yet very powerful, and should be a part of every X programmer's arsenel.
Ken has published over two dozen technical papers on the X Window System. Most are available over the World Wide Web at http://www.rahul.net/kenton/bib.html.
Ken may be reached by Internet electronic mail to kenton @ rahul.net or the World Wide Web at http://www.rahul.net/kenton/.
For more information on the X Window System, please visit my home page..