X Windows and GTK+ are not the bloated monsters you think they are. Here's how we modified GTK+/X for our device's GUI.
Consumer embedded devices require a graphical user interface (GUI) tailored to specific device characteristics. To offer a handful of examples:
Touchscreens should use large buttons to accommodate the width of fingers and inaccurate measurements
Softkeys (general-purpose buttons next to the screen) require a dedicated on-screen softkey region
A keyboard-centric device should organize data into vertical lists whereas a mouse-centric device may organize data by geographic space
Devices with low-contrast screens should avoid swaths of gray
A major factor in the success of Palm Pilots is a UI paradigm designed to fit the Palm device. The plain widget look provides on-screen simplicity, the menu system offers a good compromise between offering user options and optimizing screen real estate, and the applications are designed around stylus input. One of the reasons why WinCE and Palm-sized PC devices did not initially do as well is that a general-purpose GUI design borrowed from desktop paradigms is not appropriate for embedded devices.
A Linux GUI consists of several individual parts, some of which may be interchangeable. The GUI provided by desktop Linux distributors like Red Hat, SuSE, or Debian includes a windowing system for basic drawing and overlapping window support, a widget set (or sets) for applications to use, a window manager to implement windowing policies and to decorate windows, and a desktop environment to tie everything together into a coherent whole. For an embedded GUI, the windowing system and widget set are the most important components.
There are several embeddable widget sets for Linux, including Qt/E, FLTK, and GTK+. These can run on several possible windowing systems, including the X Window System and a direct framebuffer interface. An embedded systems developer must ask: Which combination of options is right for my system? Which widget toolkit should I adopt? How much engineering effort is needed to tweak to spec? What footprint will all this require?
This article describes our experience designing and implementing a custom GUI for a consumer Linux device. We decided to use GTK+ running on the X Window System. It took five engineer-months to produce our stable, customized and reduced-footprint GUI totaling 2.9MB (including X).
Custom UI
We designed a custom UI to match our device's I/O characteristics. Our device has a small, low-quality screen and no mouse. A row of softkeys under the screen provides primary user input (Figure 1). The bottom 14 pixel region of the screen is reserved for displaying monikers above softkeys. This is the softkey bar.
Figure 1: Device mockup.
Rather than create new paradigms that would confuse users, the rest of the UI is drawn as a typical point-and-click interface, complete with buttons and scrollbars. We maintain the user's control metaphor of physically "pushing" widgets by substituting "push-via-softkey" for "point-and-click". Each button is arranged horizontally with an arrow dropped under it that extends across the window border into the softkey bar. Figures 2-5 show the elements of our UI design, including buttons.
Figure 2: Buttons extend an arrow into the softkey bar. While pressed, the button and its arrow are filled with gray.
Figure 3: Text entry fields use a bold-face font for the label and a plain-face font for the user's text. This uses minimal screen real estate to differentiate between the two elements.
Figure 4: Scrollbars visually fit into the scrolled panes they control, giving an uncluttered look.
Figure 5: Application windows and dialogs are shaped and drawn with different titlebar designs. The topmost window gets the softkey bar.
After several rounds of testing the usability and aesthetic appeal of various mockups, we hammered out this UI design and the accompanying developer style guide. Our relief in finishing the design was cut short with thoughts of actually putting the UI together. We had conscientiously avoided considering implementation when drafting this UI. What programmer in his right mind would WANT shaped windows, arrows spanning several widgets, and numerous fonts? Now it was time to roll up our sleeves, pick a GUI, and get to work.
Choosing GTK+/X
Linux has never adopted any One True UI. There is a myriad of interfaces for both desktop and embedded products. While this fragmentation has been an obstacle in standardizing a Linux desktop, Linux's detachment from its UI has been a benefit to embedded devices. Old UIs are easily removed and newer ones are readily adopted. When designing a device requiring a custom interface, it is good to have a wide range of UI options.
A quick survey of embeddable Linux widget sets gives a list including Gtk+, Qt/E, FLTK, OpenGUI, MiniGUI, PicoGUI, and PicoTK. Most of these toolkits require a separate windowing and/or display system like X Windows, Microwindows, or direct framebuffer access.
Our GUI requirements include:
Full-featured widget set
Display several applications simultaneously
Allow third-party add-on applications
License is compatible with proprietary software
Stable enough to withstand buggy applications
Codebase that we can modify as needed
Royalty-free, without license costs to us or after-market developers
Additionally, we would prefer to use a mature GUI with a sizable application base. This gives a pool of applications that could be easily adapted to the device, as well as a large community of after-market developers.
Using the above criteria, we narrowed our GUI choices to GTK+ or Qt/E.
Qt is a GUI recently developed by TrollTech and used by the KDE desktop. Is has an OOP-intensive C++ framework. Qt/Embedded (Qt/E) is a reduced version of Qt that runs on a framebuffer. It sports a clean widget set, a signal/slot architecture, and graphics abilities including TrueType fonts and alpha blending. The Qt Palmtop Environment (QPE) is an application infrastructure that includes Qt/E running on a framebuffered client/server windowing architecture, along with a host of PIM applications.
GTK+ is a GUI with a C-based object-oriented architecture. The GNOME desktop uses GTK+. GTK+ features clean abstractions between the GTK+ library which manages widgets and signals, the Gdk library which manages windowing and drawing, and the Glib library which provides low-level data structures and utilities. GTK+ was developed on the X Window System, but has been ported to many systems, including a framebuffer (GTK+/fb) and Microwindows, a small windowing system (see Introduction to Microwindows Programming Jan/Feb 2001).
We briefly toyed with the idea of using GTK+/fb or GTK+/Microwindows. There are two problems with GTK+/fb: (1) applications cannot share the display, and as a consequence (2) if an application crashes, it brings down the entire GUI. Compiling applications together into a monolithic single framebuffered application only exacerbates (2), and makes it hard to allow add-on applications. While using Microwindows would dodge these problems, it is not yet mature.
We refined our decision to Qt/E running within QPE versus GTK+ running on the X Window System. The decision to consider X may surprise many embedded developers.
X is a 20+ year old network-transparent client-server windowing system. A single "display server" manages client display requests through an interprocess communication protocol. This makes it remarkably crash-proof. X does not include any widgets, but underlies many popular widget toolkits. Although it is well-established and well-supported, it is often assumed that X is bloated and incurs too much overhead to be useful in embedded devices. Jim Gettys, an original X author, rejects this belief, saying "most of what you hear about X being too big are from people who know little or nothing about the topic" (further info).
Recent efforts to streamline X have been successful, giving good performance on embedded hardware. We selected Keith Packard's TinyX, now a part of the standard XFree86 source tree, as our flavor of X.
We ultimately chose GTK+ 1.2.8 running on X Windows, in turn running on a framebuffer. The decision to use GTK+ instead of Qt/E was primarily a conservative preference to stick with a proven solution, but it was also informed by key differences between GTK+ and Qt/E . . .
Ownership: All things being equal, we prefer to place our trust in publicly-maintained codebases. TrollTech owns Qt/E and develops it internally without public exposure. GTK+ and XFree86 are highly visible Open Source projects with large, active development communities.
License: GTK+ is licensed under the LGPL. This allows proprietary 3rd-party software to use our derived GUI. Qt/E is released under a dual-license scheme which either releases it under the GPL or requires licensing and royalties. These terms would prevent us and other developers from releasing proprietary software for our device without paying TrollTech.
Size: Although TrollTech claims that Qt/E can be as small as 1MB, internal dependency conflicts prevented it from compiling at this size. The iPaq QPE distribution includes a 3.3MB Qt/E library and a 718KB QPE library (analogous to Xlib). TinyX and our modified GTK+ required 2.9MB (compiled for the same architecture).
Stability: Although Qt is a mature widget set, Qt/E is still not entirely stable. The QPE demo is good, but applications and QPE itself occasionally crash. GTK+ rarely crashes, but when it does X is unaffected.
Language: In our experience, C is a better language than C++ for small teams working on embedded devices. Qt/E is written in C++. GTK+ and X are written in C.
There has been a surge of interest and work in making X appropriate for embedded devices. Jim Gettys defended the future of PDAs running X, stating that "I believe very strongly that either GTK+/fb or Qt/E are dead ends" for PDAs because they cannot easily share applications with desktop systems and lack network transparency. He goes on to describe the ongoing work to put Xlib "on a diet."
The standard TinyX distribution worked perfectly on our device. To further reduce the size of TinyX, we moved the color management system (CMS) into its own library. Not using the CMS saves 86KB. Other planned reductions include removing unused, large functions like arcs and widelines, and converting macros into functions.
Tailoring GTK+ -- "Tailoring" is a good description for the next stage of our UI process. We started with the standard distribution of GTK+ 1.2.8 and snipped away excess, altered existing code, and added some new features. These modifications ranged from minor to extreme. Some even broke core GTK+ assumptions.
Removing Widgets -- The easiest change was trimming out unused widgets. The list of widgets is too long to reproduce here. It includes obviously unused widgets like GtkGamma and GtkHRuler, outdated widgets like GtkList (replaced by GtkCList), and widgets not required by our style guide like GtkFrame (why waste screen real estate on extra borders?).
Widget Sizing and Drawing -- The next round of changes was altering how widgets sized and drew themselves. GTK+ provides a theme engine mechanism for controlling widget look-and-feel. It enables run-time selection of fonts, spacing, and drawing code. This mechanism did not meet our needs for two reasons. First, it wasn't flexible enough. Many spacing and drawing constants were hard-coded. Second, a theme would be an additional chunk of code and parameters that would sit alongside the GTK+ library and increase the GUI footprint.
It required a fair bit of sleuthing to unearth all the factors that go into a widget's size. Consider the parameters that go into GtkButton's sizing and drawing: border width (inherited), x and y thickness (theme engine), default spacing (constant), default left and top shift (constant), child spacing (constant), focus (in code), and relief (drawn by theme engine). We confronted this complexity by simplifying code to ignore GTK+'s extreme flexibility in favor of our tight requirements.
While good OOP methodology dictates that widget changes should be implemented in a new subclassed widget, space constraints required us to tweak code in-place.
GtkWindow -- GTK+ assumes that when one widget contains another widget (a "has-a" relationship) the two widgets are nested. In our UI, softkey bars belonging to application and dialog windows are not contained within those windows, yet adhere in other respects to the GTK+ containment relationship. A perfect solution was not obvious, but we were able to reconcile this conflict in a minimum of new code by adding special-case checks for softkey bar child widgets to a few key functions.
The owning window listens for softkey key events and passes them to its softkey bar. Softkey arrows "poke" through the owning window's border and and display feedback when the softkey is pressed, so softkey callbacks are registered with the parent window to draw to the window's "button" areas when these events occur. We provide API at the level of the top-level windows to manage creating the softkey bar, populating it with softkeys, and attaching signals.
A more innocuous hack was managing the special case of an application window containing just one scrollable region. Drawing a window border around a scrolled region border would waste valuable screen real estate. Thus, when a window only contains a scrolled widget, it does not display a border.
Font Management -- Finding a lightweight mechanism to display many fonts on the screen turned into a hairy problem. The difficulty is a combination of Gtk's heavy-weight widget "style" and X's archaic font management scheme.
As mentioned before, GTK+ has a theme engine that controls widget look-and-feel. Just before a widget is displayed, it gets a style object, GtkStyle, which is either a pointer to a parent's style or a new style that applies to that widget and its children. This style is determined by default settings, rc text files, and the application. To change a widget's font, you must clone the widget's style and load a new font using an X font name like "-adobe-helvetica-bold-r-normal--12-*-*-*-p-*-iso8859-1".
There are several problems with this practice. GtkStyle is a large object. If a screen has many widgets with different fonts and each has a unique GtkStyle object, we waste memory. Also, X does not support relative font changes. You cannot ask X to "take this font and make it bold" because X considers different font faces to be different fonts. It is assumed that you will either hard-code the desired font or parse a font name, change it, and verify that the result is on the font server.
We wrote a more elegant font management system that circumvents GtkStyle. It lets a programmer request a widget font by attribute relative to the base font rather than by absolute X font name. For example, to display a widget in a bold face one step larger than the base font size, call:
We implemented this font system by adding a GdkFont * font member to GtkWidget, the parent class of all widgets. If widget->font is set, widget->font is used; otherwise, widget->style->font is used.
An infuriatingly thorny issue involved letting programmers set widget font attributes relative to the base font at any time, even before the theme engine has determined the base font. We store requested font changes until a widget is shown, at which time its base font is known and we can apply the changes and use the actual font.
Window Manager -- The final step of implementing our UI was creating a window manager, a program that enforces display policy. We chose to modify Aewm, a tiny Open Source X Window manager written by Decklin Foster of Red Bean software.
We hard-coded our windowing behavior preferences into Aewm. For example, the topmost window always gets focus and the root background is white.
Our design calls for application windows with vertical titlebars and dialogs with horizontal titlebars. Our modified version of GTK+ communicates this window type information to the window manager via X atoms, and the window manager draws the titlebars. GTK+ also passes an atom requesting the titlebar font.
In addition to its normal functions, our window manager serves as a communication daemon for managing intra- and inter-application windowing. Each user application window establishes a communication socket with the window manager and may name itself (e.g. "FooApp"). An application can request that it be lowered to the bottom of the window stack, or that a named application be raised. If a dialog opens on top of the topmost application window, the window manager tells the application window that it no longer has focus and should draw its widgets in an insensitive state. An application may request window manager eye-candy like a "zoom" special effect.
Footprint
After the dust had settled from the frenzy of implementing our UI, we were left with a stable system that looked and behaved exactly like our draft UI.
Compiled for the ARM architecture, our GUI's footprint is:
GTK+ still includes code that we will never use and could be removed. We estimate that another engineer-month of effort will reduce GTK+ to 850KB, bringing the GUI footprint to 2.8MB.
Performance
Our UI gives reasonable run-time performance on an ARM7 CPU (roughly equivalent to a 75MHz Pentium). Widget response time to user events is lightning fast. New screens are built and drawn at acceptable speeds.
We have been plagued by slow launch times. Our most complicated application takes as long as 2.4 seconds to load and display. Of this, 1.5 seconds are spent building and displaying the UI.
This lag stems from running GTK+ on X, which writes to a framebuffer, on under-powered hardware. No one factor is to blame. When we ran applications on our desktop machines displaying remotely to the device, initialization and drawing times were negligible. Conversely, there was good performance when running applications on the device displaying remotely to our desktop machines. Neither packet transmission, nor drawing to the framebuffer, nor GTK+'s computations were a bottleneck. The slowdown appears to be a consequence of these factors in tandem. We probably max out the under-powered CPU. It is also likely that there are also memory bandwidth constraints. At initialization time, GTK+ constructs many objects, GTK+ and X are communicating via shared memory, X is writing to the framebuffer, and the framebuffer is constantly written to the display. The memory bus runs at 18MHz and the framebuffer claims fully one quarter of this bandwidth, so the memory-intensive operations of initialization are understandably slow.
Of course, the obvious but pricey solution would be to use better hardware. We estimate that on an iPaq (100MHz memory bus, 206MHz StrongARM) applications would load 2.5 times faster, giving a respectable 1 second worst-case launch time. But since our normal operation is perfectly reasonable, we will try other solutions first.
X is costly at at initialization time. The client GTK+ application connects with the server, authenticates itself, and negotiates bit depth and other resources. The benefits of using X outweigh this overhead. X provides a robust client-server model that allows add-on applications. We can show different applications' windows at the same time, which cannot be done for most framebuffered solutions like GTK+/Fb (QPE is a notable exception).
A 2.4 second lag is not the end of the world. On a 700MHz Windows NT machine, neither Word, nor Excel, nor Internet Explorer loads in less than 1.5 seconds. 'Kedit', a KDE application, loads in 1.37 seconds on a 500MHz PIII (http://www.suse.de/~bastian/Export/linking.txt). We realize this may not be a legitimate comparison, however. Users accept lag on powerful desktop machines but expect an instant response on portable devices.
Why can PalmOS launch applications instantly? PalmOS achieves excellent speed on a small scale by using a thin OS running everything in a single address space. These are limiting factors for creating complex software. An application can crash the OS. There is no multitasking, hence no multiprocess solutions to problems. A modern operating system like Linux enables development of the complex software we want to create, but the trade-off is system overhead which is more visible on slower hardware.
We have adopted two interim strategies for dealing with the launch time problem. The first strategy is to amuse the user with eye candy as the application launches. A slow response can be forgiven as long as something happens to show that the computer is working on your behalf. The other strategy is to predictively launch commonly-used applications in the background.
Optimizations
We were able to target a few areas of GTK+ for future optimization.
We profiled the effect of removing floating point calculations from GTK+. Our ARM7 lacks an FPU so floating point calculations are a big performance hit. GTK+ uses floating point variables in a few, but commonly-used, widgets. We concluded that removing these calculations would give a 3-12% speed improvement, depending on the application.
Pixmap-rich applications were unacceptably slow. Upon investigation, we discovered that the problem was largely caused by inefficient pixmap handling methods in GTK+ and X. The X pixmap (XPM) format is designed for compatibility, not for speed, and GTK's handling of it is not fully optimized. More importantly, the pixmaps are passed from the GTK client to the X server one pixel at a time. It would be far more efficient for the client to pass the pixmap by reference to the X server, allowing direct display of a precalculated binary image.
We fixed obvious pixmap problems with GTK+ and wrote a temporary hack to grab post-rendered pixmaps from the X server and use the raw data as a replacement to XPMs. These raw images could be forcibly written to the X server. While this is an ugly solution that nobody in his right mind should use, it was 80% faster than using XPMs and demonstrates that an alternate pixmap solution could yield much better performance.
Conclusion
Consumer devices deserve a lovingly hand-crafted GUI. The Linux community has produced a number of Open Source GUIs that can be easily tailored to meet even the most bizarre design. The real difficulty is in selecting a GUI; after this, the implementation follows requirements in a fairly straightforward manner.
GTK+ was the best GUI toolkit for our needs, and X Windows provided a stable client/server model that was worth the cost of footprint and launch speed. While we are proud that we got a 2.9MB custom GUI up and running on our hardware with five engineer-months of effort, most of the credit is due to an Open Source community that produces fine software that can be squished and prodded into embedded devices.
About the author: Chuck Groom (cgroom@bluemug.com) is a project engineer for Blue Mug, Inc. One of his pet peeves is that too many embedded devices are technically brilliant but useless. He ponders ways that free software can make the world a better place.
Use of this site is governed by our
Terms of Use
and Privacy Policy.
Except where otherwise specified, the contents of this site are copyright © 1999-2007
Ziff Davis Publishing Holdings Inc.
All Rights Reserved. Reproduction in whole or in part without permission is prohibited. Linux is a registered trademark of Linus Torvalds. All other marks are the property of their respective owners.
About the author: Chuck Groom (
news feed