Epoc Compressed
eXecutable Loader
version 1.2.1

(c) copyright 2002-2006, Peter van Sebille
Mail me about ECompXL
Peter's Place

Table of Contents
What is it
What's new in this version
Verifying the installation
Building compressed applications and executables
Support for global data and global C++ objects in applications
Verifying that ECompXL has compressed your .app or .exe

How ECompXL works
Symbian OS tool chain
ECompXL tool chain
Running a compressed executable (.exe) on a target device

Understanding ECompXL's license

What is it

ECompXL is a development tool that allows you to compress applications (.app) and executables (.exe) for Symbian OS 7. Applications and executables built with ECompXL run as any other on a target device but, when started, will first decompress themselves on-the-fly before their actual execution. ECompXL uses the standard gzip utility for compression and the zlib library for decompression; a port of zlib is part of the standard Symbian OS 7 SDK.

Once installed, ECompXL is integrated in the standard tool chain and will transparently compress those applications and executables for which you've configured compression support. The compressed applications and executables generated by ECompXL behave exactly the same as their uncompressed counterparts, with the one difference that the first decompress themselves on-the-fly when run on a target device. ECompXL supports all standard target builds, including ARMI, ARM4 and THUMB. As a bonus, ECompXL allows your application (.app) to have global modifiable data and even global C++ objects, something that standard Symbian OS applications are not allowed to.

What ECompXL can not do is compress existing Symbian OS applications or executables; you have to use ECompXL as part of building your application or executable. For an explanation on why this is, read the section on how ECompXL works. ECompXL also does not support static interface dlls (i.e., dlls with numerous exported methods).

ECompXL is released under a MIT-style license, see the file copying for more information.

What's new in this version

Version 1.2.1:
Version 1.2:

Version 1.1:

Version 1.0:
Initial version


Installing ECompXL requires the following three steps:
  1. Unzip (preserving the directory structure) in the top level directory of your Symbian OS development environment. For example, on my machine that directory is q:\, so Symbian OS' standard include directory is q:\epoc32\include and, after unzipping, the binaries for ECompXL are in q:\ECompXL\bin.
  2. Add the ECompXL-bin directory to your PATH. On my machine that would be: q:\ECompXL\bin.
  3. Rename the Symbian tool chain component epoc32\tools\petran.exe to petran_symbian.exe. Don't rename it to anything else than petran_symbian.exe, ECompXL still uses Symbian's petran.exe and relies on this name.
After installation, ECompXL is part of the standard Symbian OS tool chain; you never have to invoke any of the ECompXL executables directly. Now read the section on building compressed applications and executables to enable compression for your components.

Note that the default behaviour of ECompXL is to not compress applications or executables. If you only have installed ECompXL, all the components that you built (exes, apps, dlls etc.) result in the exact same binary as if ECompXL had never been installed.

Verifying the installation

After installing ECompXL, you can do the following to verify all works ok: open a DOS box , change directory to say the root directory of your development environment (q:\ on my machine) and run petran.exe. If things work correctly you should see this:

ECompXL: EPOC Compressed Executable Loader, version xxx
Copyright (c) 2002, Peter van Sebille,

PETRAN - PE file preprocessor V01.00 (Build xxx)
Copyright (c) 1996-2001 Symbian Ltd.

Syntax: petran_symbian.exe [options] pefile output file
        petran_symbian.exe [options] e32imagefile
option: [-v] [[-no]call[entrypoint]] [-priority <priority>]
        [-stack <size>] [-heap <min> <max>] [-uid<n> <uid>]
        [-allowdlldata] [-datalinkaddress <base>] [-fixed] [-moving]
        [-align-const-section] [-const-section-address-mask <mask>]


Building compressed applications and executables

For each .exe or .app that you want ECompXL to compress, you have to add the following statement to your component's .mmp file:

If your component already had an EPOCSTACKSIZE statement then add 0x80000000 to that value. For example:

Now rebuild your component as you normally would. The resulting .exe or .app is in the exact same location you would normally find it but should now be reduced approximately 50% in size.


Support for global data and global C++ objects in applications

Symbian OS executables (.exes) already have full support for global modifiable data and global C++ objects, and ECompXL allows you to have them in your applications (.apps) as well. If you have built your application as described in the previous section, support for global modifiable data in your application comes for free. If you want support for global C++ objects as well, then you have to perform one additional step to ensure that constructors for such objects are called on start-up and destructors are called when your application exits. The simplest way is to do what ECompXL's test app (ECompXL\test\app\hello.cpp in the source distribution) does:
Constructors for your application's global C++ objects are called from the constructor of the TECompXL object and their destructors are called from this object's destructor. Note that this behaviour only applies for target builds; the behaviour in the emulator relies on the host OS  to call global C++ objects' constructors when the app dll gets loaded and to call their destructors when it gets unloaded.


Verifying that ECompXL has compressed your .app or .exe

When building your component, you should see the ECompXL banner fly by with a comment that it has successfully created your compressed .exe or .app.
For example, the source distribution for ECompXL comes with a test executable and application project. You can build the test executable from the command line like this:
cd \ECompXL\test\exe
makmake hello armi
make -f hello.armi

At the end of the output you see something like this:
ECompXL: EPOC Compressed Executable Loader, version xxx
Copyright (c) 2002, Peter van Sebille,
  Running PE2ECompXL.exe to create HELLO.bin.gz
  Running EPOC Petran on loader image ECompXL.exe

PETRAN - PE file preprocessor V01.00 (Build xxx)
Copyright (c) 1996-2001 Symbian Ltd.

ECompXL: glueing HELLO.bin.gz to loader image ECompXL.exe
Successfully created compressed executable HELLO.EXE (compressed 50%)


I've successfully used ECompXL in two projects. Below is a summary of some benchmarks I've performed on a fairly recent SonyEricsson P800 prototype. Ratios in green are better, red are worse. Start time is the difference measured in the parent process just before starting the executable under test and the first statement in the the executable under test (using RDebug::Print() with the value from User::TickCount()). Each run was performed 3 times and their average is listed in the table; the variance between each 3 runs was less than 3%.

File size (KB)
Start time in seconds
(Memory Stick - drive D:)
Start time in seconds
(Internal flash - drive C:)
EMame uncompressed
EMame ECompXL-compressed

File size (KB)
Start time in seconds
(Memory Stick - Drive D:)
Start time in seconds
(Internal flash - Drive C:)
EDoom uncompressed
EDoom ECompXL-compressed

How ECompXL works

Symbian OS tool chain
In order to understand how ECompXL works, you first need to know how the standard Symbian OS tool chain for target builds works. In a nutshell (skipping details not relevant for the discussion), it goes like this:
  1. The GNU compiler (gcc.exe) compiles your source files into object (.o) files.
  2. The GNU linker (ld.exe) links the object files into an intermediate executable file. The format of this executable file is called PE-COFF  which is short for Portable Executable - Common Object File Format. COFF is a file format originating from certain UN*X platforms and Microsoft has derived its WIN32  executable file format PE-COFF from it.
  3. The Symbian OS tool petran.exe transforms the executable file generated by the GNU linker into an E32 executable file and this is the resulting .exe or .app file you can run on a target device.
The bad news is that Symbian's E32 executable file format is proprietary and not described in any public Symbian OS SDK. The good news is that Microsoft's PE-COFF file format is well documented and all source code for the standard GNU tools (gcc, ld etc) are freely available, see references below.

ECompXL tool chain
Rather than using the E32 executable file (whose file format is proprietary), ECompXL uses the intermediate PE-COFF executable file generated by the GNU linker as the basis for generating a compressed executable. In order to decompress this executable at run time, the decompression code needs to be in a normal E32 executable file that can be run as per normal on the target device. The ECompXL solution is to "glue" a compressed ECompXL executable file (derived from the PE-COFF executable file) to a standard E32 executable file containing the ECompXL loader code (which will load and decompress the actual executable).

Building compressed executables and applications using ECompXL goes like this:
  1. The build process works as normal up to the point where petran.exe is called. During installation, Symbian's petran.exe was renamed to petran_symbian.exe, so rather than the original pertran.exe, ECompXL's petran.exe is called. 
  2. ECompXL's petran.exe examines the -stack parameter to see if the most significant bit in this 32 bit value is set. If not, then it simply runs petran_symbian.exe with the parameters that were passed to petran.exe and the resulting executable file (.app or .exe) is the same as if ECompXL had not been installed. If this bit is set, ECompXLTran.exe is run.
  3. ECompXLTran.exe starts by running PE2ECompXL.exe to generate a compressed ECompXL executable file.
    1. PE2ECompXL.exe reads the intermediate PE-COFF executable file, extracts various file headers and sections, and creates an ECompXL executable file consisting of:
      1. A single header describing all the relevant parameters for this executable (the virtual address this executable was linked for, the virtual addresses of the code, data, bss, import and relocation sections and their sizes, and the virtual address of the executable's entry point), immediately followed by;
      2. the code section, immediately followed by;
      3. the data section, immediately followed by;
      4. the import section, immediately followed by;
      5. the relocation section.
    2. PE2ECompXL.exe uses gzip to compress the ECompXL executable file generated in the previous step and then exits.
  4. ECompXLTran.exe continues and copies the ECompXL loader image ( ECompXL\bin\ECompXL.exe or ECompXL\bin\ for respectively .exes and .apps) to the target directory, renames it to the target executable file name and runs  petran_symbian.exe on it to ensure that parameters like UIDs, stack sizes  etc are set according to what was specified in the mmp file. ECompXL's petran.exe has passed all its parameters to ECompXLTran.exe and the exact same parameters are passed to petran_symbian.exe. Before doing so, the most significant bit in the -stack parameter is removed; if the stack size is 0 the -stack parameter is removed all together to let Symbian's petran.exe choose the appropriate default size. Since the ECompXL  loader is a normal E32 executable file, Symbian's petran.exe works as expected.
  5. ECompXLTran.exe "glues" the compressed ECompXL executable file to the ECompXL loader executable file. Glueing is done by literally appending the contents of the compressed ECompXL executable file to the ECompXL loader executable file. 
  6. Finally, ECompXLTran.exe appends the file offset of where the compressed ECompXL executable file got appended to the executable file.

Running a compressed executable (.exe) on a target device
When you start an ECompXL compressed .exe on target, the following happens:
  1. The Symbian OS System Loader:
    1. Reads the E32 image header from the executable file. 
    2. Loads parts of the E32 executable file containing code, data, import, export and relocation sections based on information in the E32 header. Note that the Symbian OS loader doesn't just load the entire file. It only loads those parts as specified in the E32 header, which corresponds to the loader executable (roughly 10KB in size); the glued compressed executable gets completely ignored.
    3. Dynamically links the E32 image in memory against dlls specified in the import section, loading dlls if needed.
    4. Relocates the E32 image according to where it has been loaded into memory by applying the so called "fix-ups" from the relocation section.
    5. Starts execution of the executable at its entry point, which is normally _E32Main(). _E32Main() contains Symbian OS specific start-up code such as calling constructors for global C++  objects and gets automatically included into an E32 executable file as part of the Symbian OS tool chain. _E32Main() then calls the executable's actual entry point E32Main().
  2. After the previous step, the System Loader has finished its job and as far as it is concerned the executable is up and running. E32Main() of ECompXL's loader is now executing and does the following:
    1. Opens its own executable file.
    2. Seeks to the end of this file and retrieves the last 32-bit integer value; this is the file offset at which the compressed ECompXL executable file is stored.
    3. Seeks to that offset and reads the GZIP header.
    4. Reads the ECompXL file header and uses the ZLIB library to decompress it.
    5. Creates a chunk (RChunk) for code and a chunk for data of sizes determined by information in the ECompXL file header.
    6. Loads the compressed code section from file, decompresses it and stores it in the code chunk.
    7. Loads the compressed data and import section from file, decompresses them and stores them in the data chunk.
    8. Loads the compressed relocation section from file, decompresses it and stores it in the heap.
    9. Dynamically links the executable image against libraries as specified in the import section.
    10. Relocates the executable image according to where it has been loaded into memory by applying the so called "fix-ups" from the relocation section.
    11. Cleans up unnecessary data structures (such as the relocation section in the heap) and calls the executable's entry point, which will normally be _E32Main(). _E32Main() performs the normal start-up code for this executable and then calls E32Main().
Almost the exact same happens for compressed ECompXL applications with the following differences:
  1. The ECompXL loader code is an E32 .app rather than an E32 .exe.
  2. The loader code therefor starts from its first exported method (NewApplication()) which gets called by the Application Framework when the latter loads the application dll as part of application start-up.
  3. The loader code, after decompressing, dynamically linking and relocating the glued compressed application, calls the first exported method in that application and returns that value from its NewApplication() entry point.
  4. There is no Symbian OS start-up code in applications and therefor there is no automatic support for global C++ objects (calling their constructors and destructors). Applications created with ECompXL should therefor ensure they call constructors and destructors for global C++ objects themselves. Read the section "Support for global data and global C++ objects in applications" on how to do that.

Portable Executable Common Object File Format
Source code for GNU tool used by Symbian OS

Understanding ECompXL's license

Starting from version 1.1, ECompXL is released under an MIT-style license.