Adapting the SmartEiffel compiler to the PalmOS platform involves a great many tradeoffs and a special toolchain. In practice, once makefiles are set up, the process is mostly transparent... but because a great deal is going on "under the hood", it behooves the ePalm developer to understand the evolution of the tangled ePalm toolchain.
One reason that SmartEiffel was an easy choice as an Eiffel compiler for ePalm is that it is easy to modify its toolchain, runtime, and compilation process. We'll begin at the beginning: modifying the SmartEiffel runtime to produce a C file which could be legally compiled by the PalmOS toolchain.
SmartEiffel, by default, uses the system's traditional compilation toolchain-- gcc, gdb, and friends.
Of course, this generates executables for the development platform, not for a PalmOS target. An entire new C compilation toolchain has to be installed. The ePalm project uses the "prc-tools" toolchain, a set of free software modifications to the GCC compiler toolchain which allows cross-platform development for the m68k compilation target (see the prc-tools homepage for more information on the prc-tools project).
Because it was beneficial to do some other processing, the ePalm
project uses its own version of the system.se
(located in
$EPALM/smalleiffel_sys/system.se
) which defines two new
targets:
[palmos] c_compiler_type: gcc c_compiler_path: ${EPALM}/tools/epalm_gcc.rb c_compiler_options: -pipe -O2 c_linker_path: ${EPALM}/tools/epalm_link.rb cpp_compiler_type: g++ cpp_compiler_options: -pipe -O2 smarteiffel_options: -no_strip [palmos_debug] c_compiler_type: gcc c_compiler_path: ${EPALM}/tools/epalm_gcc.rb c_compiler_options: -pipe -g c_linker_path: ${EPALM}/tools/epalm_link.rb cpp_compiler_type: g++ cpp_compiler_options: -pipe -g smarteiffel_options: -no_strip
An eiffel .ace
file can then be created for each ePalm project
which uses the palmos
or palmos_debug
targets by including the
following line under the generate
section:
c_mode: palmos
This compiles the Eiffel code into C code and transfers control of
the C compilation to the $EPALM/tools/epalm_gcc.rb
Ruby script.
A number of special-case tools written in the Ruby scripting
language are used to massage the generated C code into a
compilable form, which is then compiled and linked by the script
using the prc-tools toolchain.
So now, it seems, we can generate C code which can be compiled into Palm-hardware-compatible binary executables. But there is much, much more to be done.
As mentioned earlier, the PalmOS environment is not a traditional
desktop development environment, and the prc-tools package shows it
by balking at many traditional unix includes. To get away with
this, ePalm uses a substitute base.c
(and many other base
packages) from the $EPALM/smalleiffel_sys
subdirectory, which has
many of the offending includes removed. Some of the offenders
include
math.h (no cos/sin/sqrt...)
string.h (no strtol/strtoul...)
typedefs added in PalmTypes.h for int8_t and friends
signal.h
sys/stat.h (filesystem)
fcntl.h (file control options)
sys/file.h
inttypes.h
There are other minor problems which may or may not be a factor
(for example, PalmOS defines type Boolean
as 'unsigned char';
Eiffel defines BOOLEAN
as signed char
. In practice, many of
these make little difference.
One which makes a large difference is that an Eiffel INTEGER
is
32-bit, while a PalmOS int
is 16-bit. It is highly recommended
that the ePalm developer use INTEGER_16
for most calculations.
To handle the lack of standard file system I/O combined with
SmartEiffel's absolute need for it (SmartEiffel generates many
calls to fprintf
), a simple empty stub function named fprintf
was created and put into the base.c
file. It does absolutely
nothing. This means that the developer currently has no access to
any diagnostic information that the SmartEiffel-generated program
may produce, but at this time there is no workaround.
Unlike traditional console programs, the entry point for a PalmOS
program is not the venerable main()
function, but a more
complicated entry:
DWord PilotMain( Word launchCode, Ptr cmdPBP, Word launchFlags );
The PilotMain
function serves as the entry point into a PalmOS
program, and thus into an ePalm program. It accepts a launch code
(which indicates the way the program is opened), a pointer
cmdPBP
to any launch-code-specific data, and a set of
launchFlags
which indicade whether global variables are
available, etc.
This actually presents something of a problem to running a program
under SmartEiffel, although it's not immediately obvious. The
Palm operating system can launch an application many different
ways; although users are most commonly familiar with a "normal"
launch, there are many ways in which a program can be launched,
such as sysAppLaunchCmdFind
, in which the program is launched in
order to find a particular piece of data, or
sysAppLaunchCmdGoTo
which directs an application to open and
then go to a particular piece of data.
What we will do with SmartEiffel is hijack the operation of PilotMain, and immediately transfer control of the program to the SmartEiffel runtime:
UInt32 PilotMain (UInt16 cmd, MemPtr cmdPBP, UInt16 launchFlags) { int Result = 0; if (cmd == sysAppLaunchCmdNormalLaunch) { Result = main( 0, NULL ); } return Result; }
When the program is executed, control passes through PilotMain
,
which checks the launch code and, if the launch code indicates a
normal launch, passes control to the main()
function of the
SmartEiffel compiler
The problem isn't the different launch codes, but rather the flags associated with each. With many launch codes, the application's global variables are not available.
This situation is untenable with SmartEiffel, because a few global variables are vital to the operation of a SmartEiffel program. So many of the additional launch codes involve this case that _ePalm currently only handles normal launches_; hopefully a workaround can be found in the future.
Once PilotMain
hands control off to main
, the standard
SmartEiffel execution takes place: the stack is initialized, an
instance of the Eiffel root object is created, and control passes
to the default creation feature of the root class.
The root class, then, must be an instance of a class which can
handle all the interaction and responsibilities of a PalmOS
program. ePalm comes with one such class, the root of all ePalm
applications, PALM_APPLICATION
. The PALM_APPLICATION
class
handles basic setup and teardown of an application and has as its
creation feature its own version of 'pilot_main':
pilot_main is do create last_event.as_new_memory create system_event_handler create menu_event_handler if launch_code = Sys_app_launch_cmd_normal_launch then start_application event_loop stop_application form_manager.close_all_forms end end
Look primarily to the inner loop, ignoring the outside event
handling. The pilot_main
feature calls a deferred
start_application
feature, and begins the 'event_loop'; when
event_loop
finishes, the deferred feature stop_application
is
called, after which the form manager closes all forms and control
passes back to main
, which handles shutdown tasks.
So now we have C code which should call Eiffel code and return. Unfortunately, this works only for very small programs, so more has to be done.
When programs got to be any reasonable size, I quickly discovered a new and different type of error that I had not been aware of before:
/usr/bin/m68k-palmos-gcc -g -o controls controls1.c controls2.c controls3.c /usr/m68k-palmos/bin/ld: region coderes is full (controls section .text) /usr/m68k-palmos/lib/crt0.o: In function `start': crt0.c:62: relocation truncated to fit: DISP16 _GccRelocateData crt0.c:64: relocation truncated to fit: DISP16 __do_bhook crt0.c:67: relocation truncated to fit: DISP16 __do_ctors crt0.c:72: relocation truncated to fit: DISP16 __do_dtors crt0.c:74: relocation truncated to fit: DISP16 __do_ehook /tmp/ccKWcOSW.o: In function `se_malloc': controls1.c:53: relocation truncated to fit: DISP16 malloc controls1.c:62: relocation truncated to fit: DISP16 exit /tmp/ccKWcOSW.o: In function `se_calloc': controls1.c:73: relocation truncated to fit: DISP16 calloc controls1.c:82: relocation truncated to fit: DISP16 exit
The motorola dragonball processor is a curious beast. While it
can handle some 32-bit values (and sizeof
pointer shows that
pointers are 32-bit), local jumps are signed 16-bit jumps, which
means that local jumps must be very close together. When a
program reaches a particular size so that these jumps can't be
made, the compilation process will generate a large number of
these relocation truncated to fit
errors.
To work around this, it's possible to compile code into various sections, each of which makes local jumps into code contained within its own section and longer jumps "between sections".
It turns out that most of the offending code comes from the
generated garbage collection routines. SmartEiffel has a very
clever garbage collector that generates custom C code for every
collected class in the program--which invariably means that more
code is generated than can fit into one section. The solution is
to take a great deal of the boilerplate generated garbage
collection code and code for basic types in SmartEiffel and define
their own sections, so the following code is inserted into the
generated header file by the epalm_gcc.rb
script prior to
compilation:
#define BUILTIN_TYPES_SECTION __attribute__ ((section ("btypes"))) #define GC1_SECTION __attribute__ ((section ("gc1"))) #define GC2_SECTION __attribute__ ((section ("gc2"))) #define GC3_SECTION __attribute__ ((section ("gc3"))) #define GC4_SECTION __attribute__ ((section ("gc4")))
These lines define sections into which various code will be mapped
during the compilation process. Then the epalm_gcc.rb
script
scans the generated header file and, for each function which
belongs in a code section, places the name of the section after
the function declaration:
... void gc_sweep32(fsoc*c) GC1_SECTION; void gc_mark32(T32*o) GC2_SECTION; void gc_align_mark32(fsoc*c,gc32*p) GC4_SECTION; ...
The current script places sweep
functions in GC1, mark
functions in GC2, new
functions in GC3, align_mark
functions in GC4, and builtin type code in the BUILTIN_TYPES
section. This works for now, but is a fairly mechanical and
primitive way of doing it; there should be some way to identify
which functions call each other and maximize the number of local
jumps made (thus minimizing expensive jumps between sections) but
so far that is impractical.
Unfortunately, there is one more catch regarding memory, SmartEiffel, and PalmOS
The SmartEiffel garbage collector is truly a remarkable thing. For more full details on this section, I highly recommend Colnet, Coucaud, and Zendra's paper, "Compiler Support to Customize the Mark and Sweep Algorithm, available from the SmartEiffel site here, which outlines more fully the concepts in the next few paragraphs.
Basically, the SmartEiffel garbage collection mechanism separates objects into "fixed size" objects and "resizeable" objects. In the case of the fixed size objects, each live object type is given its owns Linear Allocation Chunk (LAC) which is just a chunk of memory with a Free-Space Pointer (FSP) pointing to the beginning of free space for similarly-typed objects. This allows the allocator to allocate blocks of a known size very efficiently.
The problem is that, as the paper says, "we found that fixed-size chunks of 8kb were a good tradeoff between fast allocation and tight memory footprint". To make things worse, it takes a bit before garbage is actually collected: "In order not to trigger a full GC cycle whenever no free room can be found either in the LAC or in the type list of free objects... the memory ceiling is considered. It represents... the ammount of allocated memory under which no garbage collection is requested, but a new chunk is malloc'd instead."
In other words, SE-generated code begins by allocating an 8k chunk for each live type. When those chunks are full, instead of triggering a GC cycle, the offending type gets a new 8k chunk until a ceiling is reached.
For a desktop, this situation is quite acceptable. For PalmOS, it
is horrifying. PalmOS has extremely tight memory
requirements--in PalmOS version 3.0 (under which ePalm development
began), executables had a mere 96k of dynamic memory available for
programs, and for application globals and dynamic allocation (in
other words, the pile of memory we'd be using up) only 36k was
allowed. Obviously, leaving each fixed-size class with an 8k
block is prohibitive. The memory requirements have been relaxed
considerably with further releases of the OS, but it was still
necessary to reduce the fixed and relocateable object chunk
sizes. This is handled by once again massaging the header file
(the chunk sizes are defined during compilation, so it was not
possible to make a one-time modification to the SmartEiffel
base.h
file):
#define RSOC_SIZE 4096 #define FSOC_SIZE 1024
This sets more reasonable demands on the memory allocation algorithm. We also define the default memory allocation scheme to use as little memory as possible (triggering GCs earlier).
Currently these chunk sizes are not modifyable by the developer
(they are hard-coded in the replace_gc_sizes.rb
script). Future
versions of the tools will likely include more configurability.
One more modification generated some difficulties. I had intended
to redefine the Eiffel INTEGER
type to be 16-bit (the default
size of an int
under PalmOS). By redefining it in the base C
files, things appeared to work well for some time.
Unfortunately, there was a snag. I had redefined the SmartEiffel
INTEGER
type to generate internal type T10
(the internal type
for INTEGER_16
). But since the Eiffel standard says that
INTEGER
is 32-bit, the SmartEiffel compiler generates internal
type T2
for some declarations of INTEGER
-- and T2
is type
INTEGER_32
.
This presents real problems when linking with external C code. If
the developer declares an external C type to have INTEGER
arguments when he means 16-bit integers, SmartEiffel will happily
generate stub code declaring the arguments to be 32-bit integers.
Because each version of the header happily seems to be
self-consistent, no error messages appear and the code will link
fine. However, since SmartEiffel is generating an argument stack
with 32-bit integer arguments and the C code expects an argument
stack with 16-bit integer arguments, the code will fail
spectacularly for no obvious reason. For this reason it is
highly encouraged to not use the standard Eiffel INTEGER
type in ePalm programs! Always use INTEGER_16
or INTEGER_32
!
At this point, the code should generate and compile. To finish
making an executable file, the epalm_link.rb
script takes over
the compilation process in a few short steps.
First, an application definition file (.def
file) is
processed using the prc-tools multigen
tool to create stubs
(.s
and .ld
files) that define the sections for proper
linking.
Next, the assembly-code .s
file is compiled into an object
(.o
) file for linking.
Finally, the produced .o
file and the .ld
file are
inserted into the argument string for the linking command,
and the files are linked by the m68k prc-tools toolchain to
produce an executable.
Suppose that now our application (which does absolutely nothing!) compiles successfully. We now have a binary executable which should execute.
But there is much more to it than that. PalmOS programs are not
viewed as islands unto themselves; like Windows programs, they are
packaged together with resources that are available to the program
at runtime (such as forms, strings, bitmaps, etc). It is
impossible to upload a solitary binary program to a Palm device;
instead it must be packaged into a .prc
file--a Palm resource
database.
Currently this is handled by further processing in the
epalm_link.rb
script, which uses the pilrc
tool to compile a
textual description of resources (a .rcp
file; see the pilrc
documentation
here
for more information). The pilrc
tool compiles the .rcp
file
into a series of binary resources (.bin
extension).
Finally, the build-prc
tool is invoked to assemble the project
definition (.def
) file, the executable file, and all the
generated binary resources (.bin
) files into a .prc
resource
database file. Finally, the file can be uploaded to the Pilot.
Of course, without any support classes or program code, it won't do a great deal. We now have the ability to cause SmartEiffel to generate compilable PalmOS code... but it remains to create the support structure and libraries to actually develop proper software. We'll cover that in the next section.