Databases

Databases

As mentioned in the introduction, PalmOS programs have no sense of a file system; instead, they rely on in-memory database structures that hold the information required. The term database refers in this case to a set of indexed records, not to any sort of relational database system.

Typically, dealing with PalmOS databases can be a tedious process in C; one gets a pointer to a block of memory, assumes that it must fit a particular struct description, manually marshals data to/from the block of memory, and then releases the block.

Happily, much of that has been automated by the ePalm infrastructure. The basic unit of interaction with the database is through the SIMPLE_DATABASE_RECORD class, which represents a record in a database. A deferred class, SIMPLE_DATABASE_RECORD is used as a parent class for actual records holding similar data.

Rather than force the developer to manually marshal data back and forth, SIMPLE_DATABASE_RECORD implements a few simple features to do the conversion for the application developer (such as read_integer, write_character, etc). The advantage is that things are much easier for the developer; the disadvantage is that there is no real way to use a record "in-place" as you can in C.

The task of customizing a SIMPLE_DATABASE_RECORD class can in large part be automated by a script, make_simple_record.rb. Here's how that's done.

Defining the Record

First we have to define the sort of record we're going to implement. In the case of the demo database applicatio, the record is very simple; it contains an alpha_code (a character), some sort of quantity (an integer), and a name (a string). We define these simply in a text file, 'demo_record.ear':


simple_database_record DEMO_RECORD is
alpha_code : CHARACTER
quantity : INTEGER
name : STRING


This .ear file is then processed by way of a Makefile target:

demo_record.e: demo_record.ear $(RUBY) $(EPALM_TOOLS)/make_simple_record.rb demo_record.ear

...which produces the desired file, 'demo_record.e':


class DEMO_RECORD

inherit
   SIMPLE_DATABASE_RECORD

feature {ANY} -- basic data elements

   alpha_code : CHARACTER
   set_alpha_code (new_value : CHARACTER) is
      do
         alpha_code := new_value
      end

   quantity : INTEGER_16
   set_quantity (new_value : INTEGER_16) is
      do
         quantity := new_value
      end

   name : STRING
   set_name (new_value : STRING) is
      do
         name := new_value
      end

feature {ANY} -- total record size

   record_size : INTEGER is
      do
         Result := 0
         Result := Result + sizeof_character
         Result := Result + sizeof_int16
         Result := Result + sizeof_string( name )
      end

feature {ANY} -- class read/write features

   read_from_pointer( p : POINTER ) is
      local
         cursor : POINTER
      do
         cursor := p
         alpha_code := read_character( cursor )
         cursor := pointer_plus( cursor, sizeof_character )
         quantity := read_int16( cursor )
         cursor := pointer_plus( cursor, sizeof_int16 )
         name := read_string( cursor )
         cursor := pointer_plus( cursor, sizeof_string( name ) )
      end

   write_to_pointer( p : POINTER ) is
      local
         cursor : POINTER
      do
         cursor := p
         write_character( cursor, alpha_code )
         cursor := pointer_plus( cursor, sizeof_character )
         write_integer( cursor, quantity )
         cursor := pointer_plus( cursor, sizeof_int16 )
         write_string( cursor, name )
         cursor := pointer_plus( cursor, sizeof_string( name ) )
      end

   dm_write_to_pointer( p : POINTER; starting_offset : INTEGER ) is
      local
        offset : INTEGER
      do
        offset := starting_offset         dm_write_character( p, offset, alpha_code )
         offset := offset + sizeof_character
         dm_write_int16( p, offset, quantity )
         offset := offset + sizeof_int16
         dm_write_string( p, offset, name )
         offset := offset + sizeof_string( name )
      end

end -- class DEMO_RECORD


The advantages to the developer are that the code above is entirely generated by the four-line description of the record. The script generates accessors and setters for each of the record fields, and then marshaling code which is called by the database framework when writing and reading from the database itself.

Now that we've defined our record, let's look at the rest of the application.

Constants and Resources

Similar pattern here, but I will omit the constants and skip straight to the resource file:


#include "database_demo_resource.h"

form id Main_form AT (0 0 160 160)
begin
   title "ePalm Database Demo"
   list "blank" id Main_form_list at (15 25 130 100)
   button "New" id Main_form_new_button at (10 140 auto auto)
   button "sort" id Main_form_sort_button at (prevright+5 prevtop auto auto)
end

form id Details_form at (0 0 160 160) modal
begin
   title "Record Details"
   label "Alpha Code:" autoid at (5 20)
   pushbutton "A" id Details_pushbutton_a at (100 prevtop 12 auto) group 1
   pushbutton "B" id Details_pushbutton_b at (prevright+1 prevtop prevwidth prevheight) group 1
   pushbutton "C" id Details_pushbutton_c at (prevright+1 prevtop prevwidth prevheight) group 1
   pushbutton "D" id Details_pushbutton_d at (prevright+1 prevtop prevwidth prevheight ) group 1
   label "Quantity:" autoid at (5 35)
   field id Details_quantity_field at (70 prevtop 80 auto) underlined numeric maxchars 5
   label "Name:" autoid at (5 50)
   field id Details_name_field at (70 prevtop 80 auto) underlined maxchars 20
   button "OK" id Details_button_ok at (5 140 40 15)
   button "Cancel" id Details_button_cancel at (prevright+5 prevtop prevwidth prevheight)
   button "Delete" id Details_button_delete at (prevright+5 prevtop prevwidth prevheight)
end

form id Sort_form at (0 75 160 85) modal
begin
   title "Sort by"
   label "Sort by:" autoid at (5 20)
   pushbutton "Alpha" id Sort_first_alpha at (45 prevtop auto auto) group 2
   pushbutton "Quantity" id Sort_first_quantity at (prevright+1 prevtop auto auto) group 2
   pushbutton "Name" id Sort_first_name at (prevright+1 prevtop auto auto) group 2
   label "Then by:" autoid at (5 prevbottom+2)
   pushbutton "Alpha" id Sort_second_alpha at (45 prevtop auto auto) group 3
   pushbutton "Quantity" id Sort_second_quantity at (prevright+1 prevtop auto auto) group 3
   pushbutton "Name" id Sort_second_name at (prevright+1 prevtop auto auto) group 3
   label "Then by:" autoid at (5 prevbottom+2)
   pushbutton "Alpha" id Sort_third_alpha at (45 prevtop auto auto) group 4
   pushbutton "Quantity" id Sort_third_quantity at (prevright+1 prevtop auto auto) group 4
   pushbutton "Name" id Sort_third_name at (prevright+1 prevtop auto auto) group 4
   button "OK" id Sort_button_ok at (5 prevbottom+5 40 15)
   button "Cancel" id Sort_button_cancel at (prevright+5 prevtop prevwidth prevheight)
end

ALERT ID Debug_alert
INFORMATION
BEGIN
    TITLE "Debug info"
    MESSAGE "1: ^1; 2: ^2; 3: ^3"
    BUTTONS "OK"
END


We define two forms: a simple form for listing the database contents and a second form for sorting the database.

The root class

Fundamentally, the root class is not different from before, but here we must do a bit more to actually initialize the application:


class DATABASE_DEMO

inherit
   DATABASE_DEMO_CONSTANTS
   FORM_DISPATCHER_CLIENT
   PALM_APPLICATION
      redefine
         handle_event, start_application
      end
   DATABASE_MANAGER_CLIENT

creation
   pilot_main

feature {ANY} -- redefined application features

   application_database : APPLICATION_DATABASE

   application_database_opened_successfully : BOOLEAN

   open_application_database is
         --tries to open main database.  If this fails, creates the 
         --database and then opens it
      do
         create application_database.from_type_and_creator_using_characters( 'd', 'e', 'm', 'o',
                                                                             'e', 'p', 'l', 'm',
                                                                             "Database_demo",
                                                                             3 )
      end

   start_application is
         --opens main form and opens/creates database
      do
         open_application_database
         create form_handler.make( application_database )
         form_manager.go_to_form( Main_form )
      end

   handle_event( event : EVENT ) is
      --handle PalmOS events
      local
         form : FORM
      do
         inspect event.type
         when Frm_load_event then
            create form.from_form_id( event.data_frmload_formid )
            form.set_as_active_form
            form.set_event_handler( form_handler )
            last_event_was_handled := True
         else
            last_event_was_handled := False
         end
      end

   form_handler : MAIN_FORM_EVENT_HANDLER

end -- class DATABASE_DEMO


We have a feature application_database of class 'APPLICATION_DATABASE'; this represents an in-memory database of the sort mentioned above. The open_application_database attempts to open it by using the from_type_and_creator_using_characters creation feature.

There are two 32-bit IDs associated with a database; the type represents the sort of data contained therein, and the creator represents the application which created or uses the data. In our case, we used made-up values ("demo" for type and "eplm" for creator) but in reality these should be registered via the PalmOS developer's program to ensure no collisions occur.

The main form

The main form here does not do much new except open and manipulate the database, but that's enough--it adds many lines to what could otherwise be a very simple class:


class MAIN_FORM_EVENT_HANDLER

inherit
   FORM_MANAGER_CLIENT
   DATABASE_DEMO_CONSTANTS
   STANDARD_EVENT_HANDLER
      redefine
         handle_frm_open_event, handle_lst_select_event, 
         handle_ctl_select_event
      end

creation   
   make

feature {ANY} --creation

   make( application_database : RECORD_DATABASE ) is
      do
         create details_form.from_form_id( Details_form )
         create sort_form.from_form_id( Sort_form )
         create sort_form_event_handler
         sort_form.set_event_handler( sort_form_event_handler )
         create current_record
         create sort_comparison.make
         database := application_database
      end

feature {ANY} -- event handlers

   handle_frm_open_event( event : EVENT ) is
      do
         populate_main_list
         form_manager.active_form.draw
         last_event_was_handled := True
      end

   handle_lst_select_event( event : EVENT ) is
      --handle selection of an item in the list.  In this case, we 
      --will be editing the details of the selected item
      do
         show_details_for_record_at_index( event.data_listselect_selection )
         last_event_was_handled := False
      end

   handle_ctl_select_event( event : EVENT ) is
      --there is only one button on the main form, so this simple 
      --creates a new database entry and edits it using the details dialog
      do
         inspect event.data_ctlselect_controlid
         when Main_form_new_button then
            create_new_record
            last_event_was_handled := True
         when Main_form_sort_button then
            sort_database
            last_event_was_handled := True
         else
            last_event_was_handled := False
         end
      end

feature {ANY} -- application features

   alpha_code_from_control_id( control_id : INTEGER ) : INTEGER is
      do
         inspect control_id
         when Details_pushbutton_a then
            Result := 0
         when Details_pushbutton_b then
            Result := 1
         when Details_pushbutton_c then
            Result := 2
         when Details_pushbutton_d then
            Result := 3
         else
            Result := 0
         end
      end

   control_id_from_alpha_code( alpha_code : INTEGER_16 ) : INTEGER_16 is
      do
         inspect alpha_code
         when 0 then
            Result := Details_pushbutton_a
         when 1 then
            Result := Details_pushbutton_b
         when 2 then
            Result := Details_pushbutton_c
         when 3 then
            Result := Details_pushbutton_d
         else
            Result := Details_pushbutton_a
         end
      end

   character_from_alpha_code( alpha_code : INTEGER_16 ) : CHARACTER is
      do
         inspect alpha_code
         when 0 then
            Result := 'A'
         when 1 then
            Result := 'B'
         when 2 then
            Result := 'C'
         when 3 then
            Result := 'D'
         else
            Result := '?'
         end
      end

   set_controls_to_record( form : FORM ) is
      require      
         current_record /= Void
      do
         form .set_control_group_selection_by_id( 1, control_id_from_alpha_code( current_record.alpha_code.to_integer ) )
         form.field_from_id( Details_quantity_field ).set_text_to_string( current_record.quantity.to_string )
         form.field_from_id( Details_name_field ).set_text_to_string( current_record.name )
      end  

   set_record_to_controls( form : FORM ) is
      do
         current_record.set_alpha_code( alpha_code_from_control_id( form.object_id_from_index( form.control_group_selection( 1 ) ) ).to_character )
         current_record.set_quantity( form.field_from_id( Details_quantity_field ).contents.to_integer.to_integer_16 )
         current_record.set_name( form.field_from_id( Details_name_field ).contents )
      end

   last_details_dialog_result : INTEGER_16

   edit_current_record is
      do
         --draw once to properly save background
         details_form.draw
         set_controls_to_record( details_form )
         --now draw again to show our changes to the controls correctly
         details_form.draw
         details_form.do_dialog
         last_details_dialog_result := details_form.last_dialog_result
         if last_details_dialog_result = Details_button_ok then
            set_record_to_controls( details_form )
         end
      end

   create_new_record is
         --create a new record in the database and edit it
      do
         current_record.set_name( "bob")
         current_record.set_quantity( 42 )
         current_record.set_alpha_code( '%/1/' )
         edit_current_record
         if details_form.last_dialog_result = Details_button_ok then
            database.new_typed_record( database.max_record_index, current_record );--database.number_of_records + 1, current_record )
            populate_main_list
         end
      end

   set_current_record_to_record_at_index( index : INTEGER_16 ) is
         --set the "current record" to the database record with the 
         --given index
      do
         database.set_typed_record_from_index( index, current_record )
      end

   set_record_at_index_to_current_record( index : INTEGER_16 ) is
         --set the record at the given index to the current record
      do
         database.write_typed_record_to_index( index, current_record )
      end

   show_details_for_record_at_index( index : INTEGER_16 ) is
         --edit the record at the given index
      do
         set_current_record_to_record_at_index( index )
         --edit the record
         edit_current_record
         --if "ok" selected, set the database record at the index to 
         --the current record
         if last_details_dialog_result = Details_button_ok then
            set_record_at_index_to_current_record( index )
            populate_main_list
         elseif last_details_dialog_result = Details_button_delete then
            database.remove_record_at_index( index )
            populate_main_list
         end
      end

   populate_main_list is
      local
         record_iterator : INTEGER_16 
         max_records : INTEGER_16
         entry_texts : ARRAY[STRING]
         current_line : STRING
      do
         create current_line.make( 50 )
         from
            record_iterator := 0
            max_records := database.number_of_records
            create entry_texts.make( 0, max_records - 1 )
            entry_texts.set_all_with( "woof" )
         until
            record_iterator >= max_records
         loop
            database.set_typed_record_from_index( record_iterator, current_record )
            current_line.clear
            current_line.append( current_record.name )
            current_line.append( " (" )
            current_line.append( current_record.quantity.to_string )
            current_line.append( ":" )
            current_line.add_last( character_from_alpha_code( current_record.alpha_code.to_integer ) )
            current_line.append( ")" )
            entry_texts.put( current_line.twin, record_iterator )
            record_iterator := record_iterator + 1
         end
         form_manager.active_form.list_from_id( Main_form_list ).set_list_choices( entry_texts )
         form_manager.active_form.list_from_id( Main_form_list ).draw
      end

   sort_database is
      do
         sort_form.do_dialog
         sort_comparison.set_sort_criteria( sort_form_event_handler.sort_first_by,
                                            sort_form_event_handler.sort_second_by,
                                            sort_form_event_handler.sort_third_by )
--         form_manager.custom_alert( Debug_alert, sort_form_event_handler.sort_first_by.to_string,
--                                    sort_form_event_handler.sort_second_by.to_string,
--                                    sort_form_event_handler.sort_third_by.to_string )
         database.quicksort( sort_comparison, 0 )
         populate_main_list
      end

   sort_form : FORM

   sort_form_event_handler : SORT_FORM_EVENT_HANDLER

   details_form : FORM

   current_record : DEMO_RECORD

   database : RECORD_DATABASE

   last_handle : MEM_HANDLE

   sort_comparison : DEMO_RECORD_COMPARISON

end -- class MAIN_FORM_EVENT_HANDLER


In anticipation of a user wanting to look at the details of a particular record, the make feature creates the details_form dialog from its resource id, as well as creating the sort_form for future use, and an event handler to associate with it. These forms are created once here, rather than repeatedly later, in anticipation of further reuse (rather than repeatedly creating and destroying the sort form, we create it once and then anticipate using it several times). The sort form is associated with its event handler and held ready, and two more special objects are created: a record to hold the current record displayed on screen, and a comparison object used to compare two different records for sorting (more on this later).

The actual event handling is fairly mundane, displaying the details of a record selected from the list or creating a new record. A few conversion features are provided as well.

When the user wants to edit a record, the edit_current_record feature is called, which draws the details form and then sets the details form controls to the values present in the record. The set_controls_to_record and set_record_to_controls features are just reciprocals of each other, synchronizing the on-screen controls to the values present in the record and vice versa; the database has not been modified yet! Only when the set_record_at_index_to_current_record feature is called is this done, when we call database.set_typed_record_from_index( index, current_record).

Look through the above class and note how records are created, read, written, and manipulated. Simple operations are fairly straightforward.

When things need to be sorted, life becomes a bit more tricky. Take a look at sort_database, which brings up a dialog (the "sort form", which lets the user choose the order of fields to sort by. A mysterious "sort_comparison" object sets its criteria based on that chosen by the user, and then database.quicksort is called, using the sort_comparison object as a parameter.

Record comparisons

The comparison object passed to the database.quicksort feature represents a comparison between two records:


class DEMO_RECORD_COMPARISON

inherit

   SORT_CONSTANTS
   DATABASE_DEMO_CONSTANTS
   FORM_MANAGER_CLIENT
   DATABASE_RECORD_COMPARISON
      redefine
         compare_records
      end

creation

   make

feature {ANY} --comparison features

   demo_record_a : DEMO_RECORD

   demo_record_b : DEMO_RECORD

   sort_first_by : INTEGER_16

   sort_second_by : INTEGER_16

   sort_third_by : INTEGER_16

   set_sort_criteria( new_first, new_second, new_third : INTEGER_16 ) is
      do
         sort_first_by := new_first
         sort_second_by := new_second
         sort_third_by := new_third
      end

   make is
      do
         create demo_record_a
         create demo_record_b
         sort_first_by := Sort_by_name
         sort_second_by := Sort_by_quantity
      end

   compare_name : INTEGER_16 is
         --three way comparison based on name
      do
         Result := demo_record_a.name.three_way_comparison( demo_record_b.name ).to_integer_16
      end

   compare_quantity : INTEGER_16 is
      do
         Result := demo_record_a.quantity.three_way_comparison( demo_record_b.quantity ).to_integer_16
      end

   compare_alpha : INTEGER_16 is
      do
         Result := demo_record_a.alpha_code.three_way_comparison( demo_record_b.alpha_code ).to_integer_16
      end

   compare_criterion( sort_by : INTEGER_16 ) : INTEGER_16 is
      do
         inspect sort_by
         when Sort_by_alpha then
            Result := compare_alpha
         when Sort_by_name then
            Result := compare_name
         when Sort_by_quantity then
            Result := compare_quantity
         end
      end

   compare_records( record_a, record_b : POINTER;
                                                  other_info : INTEGER_16;
                                                  sort_info_a, sort_info_b, app_info_handle : POINTER ) : INTEGER_16 is
      do
         demo_record_a.read_from_pointer( record_a )
         demo_record_b.read_from_pointer( record_b )

         Result := compare_criterion( sort_first_by )
         if Result = 0 then Result := compare_criterion( sort_second_by ) end
         if Result = 0 then Result := compare_criterion( sort_third_by ) end
      end

end


The meat of this is in compare_records, an overridden feature which takes two records and returns a standard 'compare'-style result (0 if the records are equal, -1 or 1 depending on which is "greater" in the given comparison). This object is taken by the quicksort routine and called repeatedly as the database is sorted.

It's worth noting that this would be an ideal task for an agent in the new agent mechanism, but unfortunately at the time ePalm was written, SmartEiffel (then SmallEiffel) had no agent mechanism to take advantage of. This option will be considered in the future.

More cecil.se and .ace fun

As one might expect, databases incur a certain amount of support code, which requires exposure both in cecil.se and the .ace file. The cecil.se file only adds exposure for the database manager singleton and comparison features:


epalm_cecil.h
--the features you want to call from C
FORM_DISPATCHER_dispatch_event_from_pointer FORM_DISPATCHER dispatch_event_from_pointer
FORM_DISPATCHER_CLIENT_form_dispatcher FORM_DISPATCHER_CLIENT form_dispatcher
DATABASE_MANAGER_compare_database_records DATABASE_MANAGER compare_database_records
DATABASE_MANAGER_CLIENT_database_manager DATABASE_MANAGER_CLIENT database_manager


...and only minor inclusions in the .ace file:


external -- section for elements written in another language

   cecil ("cecil.se")
   external_header_path: "${EPALM}/smalleiffel_sys/runtime/c ."
   external_c_files: "${EPALM}/src/c_code/database_support.c"
   external_c_files: "${EPALM}/src/c_code/database_sort_compare.c"


Databases are simple at heart, but provide the persistence necessary for almost any truly useful application. The next section describes (but does not walk through) a more practical application--a simple multiple-choice quiz program.

Powered by Zope