Should you integrate a GUI into an embedded system by adding a dedicated processor or upgrading the existing one? Here are some options and trade-offs.
A graphics panel is often an afterthought for embedded systems. In some cases, the main product doesn't have a graphics panel, but one can be purchased as an optional extra. I often see this in medical devices where, for example, numerical data representing a patient's condition can be seen on seven-segment displays on the main device, while an optional add-on can take that data and display it in a graph. Another common practice is to not include a graphics panel on a current product, but call for one in the next generation of the design.
Unfortunately, implementing a graphical user interface (GUI) is more than a matter of adding one more peripheral to the address/data bus and controlling it with the same microcontroller from the previous design. Graphics generally demand more processing speed and more code space. A device with ten buttons and a simple four-line LCD display might contain 32KB of code. A graphics display with a few fonts and bitmaps could easily add 100KB to the code size. Suddenly, that 8-bit microcontroller with on-chip flash and RAM is no longer up to the job.
One option is to upgrade the microcontroller for the entire design and port the control code from the previous generation. The new chip then does the work of the old one and also drives the graphics.
An alternative is to make minimal changes to the current design and add a second processor to handle the graphics. A spare serial port can provide the communications path to the graphics processor. The original design will need some software updates to transmit and receive event data to and from the graphics controller, but, for the most part, won't require modification. This approach may not be the cheapest in terms of unit cost, but for some low-volume products, tearing up the old design is the bigger expense.
Amulet Serial Module
One option when you want to manage your graphics from a serial port is the Amulet Technologies GUI.1 This product contains an LCD, a touchscreen, and a small board to drive the LCD. In addition to managing the data and clock signals required by the LCD, the board provides a graphical library and some simple graphical objects.
You don't program this board in C. Instead, you load an HTML file to be stored in flash over the serial port. The serial port can then provide a conduit for messages that change values on the display. Such a message might be a temperature value sent to update the height of a bar. Messages received from the Amulet indicate when an event, such as a user pressing a button, has occurred.
The downloaded HTML file is not pure HTML, but the variations are slight. Amulet calls it µHTML (micro-HTML). While the format is similar to HTML, the user's experience is not mediated by a browser. The HTML format is simply a processor-independent way to describe the layout of a screen or a number of screens and locate text, bitmaps, and graphical widgets on those screens.
If you've defined a number of screens, navigating among them doesn't require any communication external to the Amulet. A user event, such as a changed value on a slider, will trigger communication. This minimizes the frequency of interruptions to the normal work of the microcontroller. In the other direction, the processor may transmit new values to the Amulet to update values, text, or widgets on the display.
This design creates a useful division between the control processor and the graphics engine. Since an HTML file is the medium for graphics control, the graphics board also doesn't require any code. You have to program the control processor to perform some serial communications, but you don't have to bother with line-drawing routines and the like. More conventional graphics solutions offer a software library that you integrate with your own program. Amulet provides the low-level graphics as a hardware solution.
If you change the layout or redesign some icons, you just disconnect the graphics module from the controlling processor and move it to a serial port of the PC. A new HTML file downloads, and now your new user interface is in place. The upshot? Purely visual changes don't require modifications to the microcontroller software.
Since the Amulet interacts with the rest of the system via serial protocol, integration with a specific processor, RTOS, or compiler is not an issue. For this reason, the Amulet is by far the fastest design solution I have seen for adding a GUI to an existing system. However, speed of design is not always the most important factor. Amulet's biggest limitation at the moment is that it only supports monochrome LCDs.
Another drawback is that the complexity of the GUI is limited by the HTML format. If the layout changes dramatically at run time, you will lack access to some of the values that are constant in the HTML file.
There is also a trade-off between design time and cost per unit of product. One-size-fits-all can never compete with roll-your-own if your volumes are high and margins tight. The Amulet is a good fit if the volumes are lower, time to market is paramount, or graphics programming expertise is in short supply.
Segger emWin Graphics Library
The alternative to a dedicated graphics subsystem is to drive the graphics from the main processor. You usually still need a graphics controller, such as the Epson SED series, to manage the clocking of data into the LCD screen. However, the line drawing and bitmap copying routines all run on the main processor. If you choose this architecture, you may well consider purchasing a third-party graphics library. The library I have been using recently is from Segger, a German company that sells a C-based library called emWin.2
The emWin library is not as feature-rich as some of the competition, but it uses fewer resources. You can run the emWin library on an 8-bit micro as long as the screen updates don't need to be very fast.
The library allows you to render text and bitmaps and provides utilities for converting fonts and bitmaps from a PC. Its drawing routines allow you to create arbitrary lines and curves. At a higher level, it provides widgets such as buttons, sliders, windows, list boxes, and progress bars. A window manager allows windows to be moved around. (While this feature is indispensable on a desktop, most embedded applications don't need it.)
The most noticeable shortcoming is the lack of a drag-and-drop GUI-building tool. Many libraries let you design the GUI on the host system by dragging the widgets onto a simulated screen and then altering their attributes from menus. When the designer is happy with the layout, the tool automatically generates code for the target system.
Drag-and-drop tools make it easy to produce the first draft of an application, but I find them less useful as the application becomes more complex. Say you want four buttons in a row. All you have to do is drag a button onto the simulated screen and then change a few of its attributes, such as color and text. Repeat four times and the GUI is complete. Now say you want N buttons, where the value of N varies at run time. Maybe there's one button for each incident recorded in the device's log, with the number of recorded incidents changing over time. The text on the button is the name of the incident, so that text varies at run time, too. Since the screen is not static, doing the layout as a single snapshot has limited utility.
A far more significant feature than a GUI builder is the ability to run your applications on a PC. The Segger library allows you to do this, and you may also use a bitmap as a background to complete the imitation of the real device. It is also possible to incorporate off-screen keys to imitate a device that has input keys other than the touch panel.
Segger Evaluation Boards
Segger also sells evaluation boards that are based on Mitsubishi and NEC microcontrollers and use the Epson SED series of graphics controllers. Because they supply the LCD, touchscreen, microcontroller, and software tools, these evaluation boards allow quick development of a GUI prototype. Having a prototype built on real hardware can be useful, even if the final product will employ a different processor or screen.
The only catch is that the compilers supplied with the evaluation boards are time licensed, so you have to decide fairly quickly whether you want to use the micro on the board in the final product. If they provided a GNU toolchain with their evaluation system, the developer would have more flexibility. Segger supplies the schematics for their evaluation boards, so if you choose to use the processor and graphics controller from their board, then designing the graphics components of your own system should prove straightforward.
I have always been a big fan of evaluating GUI designs on a PC, but an evaluation board like this allows you to investigate the size and speed of the code, as well as physical issues such as whether touch-sensitive buttons are too close together and whether the display is readable from a distance. These physical issues are difficult to assess on a PC.
emWin Event Model for Touchscreens
While any graphics library can be used to output graphics, those that can process touchscreen events are far more powerful. In a touchscreen application, you need a mechanism to indicate that a certain action should happen whenever a specific button is pressed or a specific widget is acted upon by the user.
I prefer to store a function pointer in the button and point it at a callback function that performs the button's job. Each button could have a different callback function, or a number of buttons could share the same one if their functionality were similar.
This model is easy to program, but it involves storing a function pointer for each button. If you want to capture multiple events such as button-down, button-up, slide-on, and slide-off, the list of stored pointers grows quickly, using up valuable RAM. The emWin library does not follow this model and opts for something a little less programmer friendly, but far less memory hungry.
The emWin library allows one callback function per window. That function handles all of the events within that window. Each object (button, slider, radio button) has an identifier and that identifier is used to distinguish which action should be taken when an event occurs. Each window's callback function usually contains a large switch statement to enable jumps to the appropriate routine for each action.
C vs. C++
I am a big fan of C++, especially for graphics work; inheritance lends itself well to graphical objects, which makes the job of producing and using a graphics library much easier. Of course, many developers— myself included—are reluctant to use a library that mandates C++ in an embedded system. It's not always as bad as we think; the language's size and speed issues can be avoided if the library vendor and application designer exercise some restraint. The people at Swell Software, for example, use the Embedded C++ subset to avoid the bloat issues associated with exceptions and templates in the PEG graphics library.3
I think it is important to have a C option available to graphics programmers, even if it is less elegant than the C++ option. Sometimes a reliable C++ compiler isn't available. Sometimes transitioning a team of C programmers to C++ is too onerous. Segger's emWin is a C library. While you can call emWin from C++, the API is a set of C functions, not a set of objects. Competitors such as PEG from Swell Software and Zinc from Professional Software Associates offer C++ products that require a C++ toolchain.4
Threading
I generally consider it good practice to use a single task to process all actions that change the display. However, some designs enable each of a number of tasks to control a different area of the display. It's important for library vendors to tolerate both models, so that each developer can make his own decision about the task architecture. With emWin, the developer can turn on support for reentrancy in his code. Reentrancy is disabled by default because the additional locking would generate extra code. To support reentrancy, some integration with your chosen RTOS is required.
Solutions for Embedded Graphics
We just looked at two out of an increasing number of interesting hardware and software solutions for graphics systems. Desktop developers never have to write line-drawing or flood-fill algorithms, so why should you? Products like these allow embedded developers to withdraw themselves from low-level details and focus on the higher level application functionality.
Endnotes
1. Amulet Technologies [back]
2. SEGGER Microcontroller Systems [back]
