Note: when I first started Diminuto, it began as both an exercise in using the open source buildroot tool and as an associated Linux systems programming library and toolkit written in C. But with later projects like Arroyo, Cascada, and Contraption, the library and toolkit became much more useful than the artifacts that came out of buildroot. I've left in the Makefile buildroot rules, but have not tested them in ages and would not expect them to work. The library and toolkit I use all the time. So the default behavior of the Diminuto Makefile is to just build the library and toolkit, and it is just the library and toolkit that is used in subsequent projects. As such, the library is a constantly moving target, and I generally only test it in the context of my latest project. You can find a tarball of the library and toolkit here.
Diminuto is my attempt to create a platform for teaching embedded and real-time software design and assembly language programming using open source software and commercially available hardware. It is a work in progress. Diminuto runs on an Atmel AT91RM9200-EK evaluation board.
This board has an Atmel AT91RM9200 system-on-a-chip based on the ARM ARM920T microprocessor core which is part of the ARM9 family. The ARM9 microprocessor design has 247 different licensees and is widely embedded in consumer electronic devices including mobile phones, PDAs, internet tablets, games, and printers. The ARM9 implements the ARMv4 processor architecture. I chose the processor, board and software to meet the following goals:
- The hardware is commercially available and a single teaching laboratory can host several stations at a reasonable cost.
- The instruction set of the processor is amenable to learning assembly language programming;
- The processor is widely used in actual devices, especially consumer electronic devices.
- The platform is sufficient to run Linux, but is resource constrained enough that developing for it isn't like developing for the typical Pentium-based server.
- The platform is capable of running smaller, simpler real-time operating systems without undue pain.
- The platform is a system-on-a-chip that incorporates a rich set of contemporary, useful devices.
- The platform supports Ethernet and removable storage media.
- The platform includes a boot-loader in ROM.
- The tool chain provides C and C++ and can be expanded to include other languages.
- The platform exposes the hardware both physically and programmatically.
- The hardware platform is an unlikely theft target (unlike a consumer device such as a Nokia N8xx or a LinkSys box).
- The form factor is typical of embedded projects in the real world.
The AT91RM9200-EK comes with U-Boot (a boot-loader in ROM), a bootable Linux 2.4 image and its associated initrd ramdisk image. But Linux 2.4 is no longer mainstream, and the ramdisk image is minimal.
- The tarball of the patches and utility source code for the Diminuto library and toolkit can be found via a link in the note at the top of this page . It is quite likely this is all you need unless you are using buildroot on the old AT91 evaluation kit. If you are using the old AT91 EK, be aware that Diminuto library is a constantly moving target. The pre-built initramfs root file system image probably has a much older version of the library. You should consider rebuilding the library and replacing it and the few binary executables that depend upon it in your initramfs root file system image.
- The bootable Linux 188.8.131.52 image can be found here. It includes an integrated initramfs root file system, so a separate ramdisk is not required. This kernel has the AT91 patches for 184.108.40.206 applied, as well as two additional patches (to the very early boot code, and to the linker script) that I found necessary.
- A log of Diminuto booting is here. It displays all of the U-Boot environment too.
- A tarball of the i386-hosted cross-compiling tool chain used to compile applications is here. It was built to reside under /opt/diminuto/buildroot but can be installed elsewhere and run using the GCC sysroot flag. (The executables you want are under the usr/bin directory and the CROSS_COMPILE string is "arm-linux-".)
- Desperado is an open source collection of C++ classes implementing design patterns I've found useful in embedded systems. I have built and run all but the JVM-interface portion on Diminuto.
- Here is a collection of photographs and screen snapshots of the board and the running system.
Diminuto boots Linux with its root file system as a single image via TFTP, enabling a stable teaching laboratory system to be maintained on a central server, while making it easy for students to build their own kernels and root file systems. The kernel supports ext2, ext3, and VFAT (a.k.a. Windows FAT-32) file systems on USB memory sticks and SD cards (both of which I've used on the board), making it easy for students to store their work persistently on inexpensive removable media.
The following U-Boot environmental variables are used to provision the system:
- serverip is set to the IP address of the TFTP server, e.g. 192.168.1.222 (this is required by U-Boot).
- ipaddr is set to the IP address of the target board, e.g. 192.168.1.223 (this is required by U-Boot).
- netmask is set to the network mask of the target board, e.g. 255.255.255.0 .
- gatewayip is set to the IP address of the gateway for the local subnet, e.g. 192.168.1.1 .
- broadcastip may be set to the IP broadcast address if it is something other than the standard value, e.g. 192.168.1.255 .
These U-Boot environmental variables are read by Linux during boot and are used to provision the Ethernet interface. They may be persistently saved by the U-Boot bootloader in on-board flash memory, simplifying board setup.
I have successfully compiled the C++ Desperado unit tests using the tool chain and run most of them on the board. (This was done as a test of the completeness of the tool chain and its C library.) Ficl, the Forth-based embeddable scripting langauge used by Desperado, is supported. The native interface to Java currently is not. (Porting a JVM to Diminuto is yet to come).
Diminuto was built from the following open source sofware components.
- U-Boot 1.1.1 (a boot-loader provided with the hardware);
- Buildroot 22987;
- GCC 4.2.4;
- uClibc 0.9.29 (a reduced memory footprint implementation of the standard C library which supports all of Desperado);
- libstdc++ v3 (this is a full version of the C++ library in support of Desperado);
- Binutils 2.18;
- Linux 220.127.116.11 with the applicable AT91 and my own patches; and
- Busybox 1.11.1 (a program that provides many utilities in a single binary).
This distribution has the following open issues (that I know and care about, anyway):
- There are two ARM application binary interfaces or ABI (in part, how code generated by a compiler for an application talks to the kernel), the "Old" ABI or OABI and the newer "Embedded" ABI or EABI. I wasn't able to get the EABI working reliably. Seems to be a problem in how signals are handled in the kernel.
- I haven't figured out how to get the Gnu debugger working correctly, using gdb on the host and gdbserver on the AT91RM9200-EK target. This may be pilot error on my part, but gdb works just fine with the gdbserver implementation embedded in my BDI 3000 JTAG debugger.
- The compiler option -O3 can produce mis-behavior from code compiled with arm-linux-g++ that does not occur if the same code is compiled with arm-linux-gcc. I've convinced myself that this is not a bug in the C++ (or C) compiler, but a bug on my part involving a casting of pointers in which I unwittingly violated anti-aliasing rules. This manifested in weirdness with the desperado_offsetof macro in generics.h. Both issues have been addressed, but I found them to be very subtle. This illustrated to me the hazards of porting a large legacy code base to a new compiler version; there is a lot of "But it worked before!".
- There is significant weirdness in the Platform::factory() static method in Platform.cpp. I developed a workaround (a.k.a. a hack). I'm sorely tempted to call this one a compiler bug, but I'm sure I'll eventually read how I was doing something subtly wrong. Turning optimization on or off had no effect on it.The upshot is that dereferencing a base class pointer to a derived class object caused a Segmentation Violation. Simply storing the base class pointer in a global variable (to prevent any optimization) before dereferencing it fixed the problem. Spooky.
The AT91RM9200 is a system on a chip that features the following embedded components:
- ARM920T 32-bit microprocessor core;
- 180 MHz clock for a claimed 200 MIPS;
- Memory Management Unit (MMU);
- JTAG debugging port;
- 16KB instruction cache,
- 16KB data cache,
- USB host and device controllers;
- Ethernet controller;
- four USARTS;
- Serial Peripheral Interface (SPI);
- system timer;
- real-time clock;
- and a bunch of other stuff.
The AT91RM9200-EK evaluation board features:
- 32MB of RAM;
- 8MB of NOR flash;
- one 10/100Base-T Ethernet port;
- two DB-9 serial ports;
- a USB host double port;
- a USB device port;
- a MultiMedia Card Interface supporting both MMC and SD cards;
- a video controller;
- some LEDs;
- power supply circuitry so that only a single 12V DC power brick (supplied) is needed;
- a JTAG connector.
My development setup included the following (all U.S. dollar amounts approximate):
- Required: Atmel AT91RM9200-EK evaluation kit ($1250 retail in quantities of one, available off the web from any number of suppliers);
- Required: a shared Linux server on which to run the tool chain and a TFTP server (I used a Dell 530 at $900, but any existing Linux server would probably work);
- Required: serial console, either a terminal, a PC, a reverse telnet server, a USB serial adaptor, etc. (I used an old laptop);
- Required: anti-static mat and grounding wrist strap (from Amazon for $26 and $12 respectively; get used to using these or suffer the consequences);
- Recommended: PCB standoffs, little legs to replace the lame rubber feet that come with the evaluation kit ($2 at Radio Shack);
- Recommended: a single shared JTAG debugger (I used an Abatron BDI-3000 , the gold standard at about $2800; having a single unit of any brand for a multi-station lab will be occasionally necessary for some kernel or device driver hacking; it's good experience too, since they are widely used in the real world);
- Optional: personal fan ($12 from Amazon.com; the AT91RM9200-EK runs pretty cool, so if your A/C is adequate, don't bother);
- Optional: anti-feline shield ($10 from Target, shown here in use);
Here are some suggestions for projects on Diminuto:
- Build a root file system on the SD card using the full-blown glibc (oops, already done: see Arroyo).
- Port a JVM (maybe Kaffe) and write a simple web server in Java as a sample application.
- Get the video display and a USB keyboard working and turn them into a system console.
- Get a USB mouse working too and port X.
- Port some well-known CPU, I/O, and network benchmarks and run them.
- Get the D programming language running in the tool chain and develop a distributed socket-based client/server application between the board and a server as a sample application.
- Port a web server (or check out the one in Busybox) and write a web-based administration tool as a sample application.
- Get the latest (what ever it is) Linux kernel and/or Busybox running.
- Write a tiny real-time OS in a mixture of assembler and C and get it running.
Here are some articles that my alter-ego, Chip Overclock, has written about Diminuto:
- Diminuto and the War Against Abstraction
- Choosing Hardware for Diminuto
- Diminuto Right Out of the Box
- Choosing Software for Diminuto
- Diminuto Nuts and Bolts
- Running with Scissors Around Diminuto and Arroyo
- Memory Mapped Devices on the Beagle Board with Android
Here are some related books that I have on my bookshelf:
- ARM Architecture Reference Manual, Second Edition, D. Seal ed., Addison-Wesley, 2000
- ARM System-on-a-Chip Architecture, Second Edition, S. Furber, Addison-Wesley, 2000
- ARM System Developer's Guide, A. Sloss et al., Morgan Kaufmann, 2004
- ARM Assembly Language Programming, P. Cockerell, MTC, 1987
- ARM Assembly Language - an Introduction, J. Gibson, 2007
- AT91RM9200, 1768G-ATARM-29-Sep-06, Atmel, 2006-09-29
- ARM920T Technical Reference Manual, Revision 1, ARM DDI 0151C, ARM, 2001
- Building Embedded Linux Systems, K. Yaghmour, O'Reilly, 2003
- Embedded Linux Primer, C. Hallinan, Prentice Hall, 2007
- Programming Embedded Systems with C and GNU Development Tools, Second Edition, M. Barr et al., O'Reilly, 2007
- Embedded System Design on a Shoestring, L. Edwards, Newnes, 2003
- Understanding the Linux Kernel, Third Edition, D. Bovet et al., O'Reilly, 2006
- The Linux Kernel Primer, C. Rodriguez et al., Prentice Hall, 2006
Linux Device Drivers