Trusted Computing for the Java(tm) Platform

This guide is a step by step description on how to (re)build a minimalistic compartment image which is able to execute Trusted Java applications. A sample demonstration for the PrivacyCA server application is included.

There are 5 major phases in the build process:

A Gentoo Linux base system is bootstrapped.
A Java development environment based on IcedTea Java is set up.
The basic system structures for the minimalistic target image are built.
The Java application is run and observed for actual resource consumption.
The final minimal compartment is assembled.

1. Linux Base System

The intended minimalistic application compartment is a small-sized Linux-based system consisting of the uClibc system library and a minimal user environment. To build all components for this final image it is usually easier to start from a full Linux system based on uClibc instead of creating everything by crosscompiling from a common GLibc based distribution. Thus, the first step is to bootstrap a Linux system based on uClibc.

The Linux system we bootstrap is Gentoo based. Any Linux distribution can be used as a starting point, but we recommend to also start from a Gentoo Linux system.

1.1. Setup of Base System

We start from the Gentoo provided stage3 installation archive as base of our build system onto which we apply our own customised configuration files provided in separate tarballs. The shell variable $BUILD_DIR is used for the directory where the build system will be installed.

Extract the stage3 archive and the customised configuration files into a new directory.

BUILD_DIR=uclibc-buildroot/
mkdir $BUILD_DIR

tar xvf stage3-x86-uclibc-2008.0.tar.bz2 -C $BUILD_DIR
tar xvf gentoo-config.tar.bz2 -C $BUILD_DIR

Unpack the portage snapshot into the usr/ directory of the future chroot.

tar xvf portage-20090120.tar.bz2 -C $BUILD_DIR/usr

Copy the supplied baselayout-lite tarball and the uClibc patches for gcc into the distfiles directory of your build system to make it available for the portage package manager. Then, extract the configuration files for the minimal environment.

cp baselayout-lite-1.0_pre1.tar.bz2     $BUILD_DIR/usr/portage/distfiles/
cp gcc-4.3.1-uclibc-patches-1.0.tar.bz2 $BUILD_DIR/usr/portage/distfiles/

tar xvf gentoo-minimal.tar.bz2 -C $BUILD_DIR/gentoo-minimal

Copy the IcedTea and OpenJDK tarballs into the distfiles directory of your build system to make them available for later use.

cp icedtea-minimal.tar.bz2 $BUILD_DIR/usr/portage/distfiles/
cp openjdk-b29.zip         $BUILD_DIR/usr/portage/distfiles/

Enter our new environment with the chroot command.

cd $BUILD_DIR/
cp /etc/resolv.conf etc/
mount -t proc none proc/

chroot . /bin/bash

1.2. Runtime Environment Support for Java

The package versions used for the base system are taken from a fixed date. Thus, it is possible that some source files are not available anymore on the Gentoo mirror servers or even on the original download locations. However, the prepackaged base system snapshots referenced in section 5 contain all source archives of their included packages respectively and can thus be used as a package mirror for use with our files.
Rebuild the GNU Compiler Collection and GNU Binutils for using the new CHOST setting which is different from the used stage archive. Afterwards install our modified version of uclibc.

USE="-gcj" emerge -av =gcc-4.1.2 binutils && emerge -v uclibc

The binaries provided in the Gentoo stage are out of date which requires upgrading the packages to the versions contained in the more recent Portage tree snapshot which is used by us. Rebuild the world set with emerge command.

emerge --unmerge mktemp && emerge --oneshot coreutils
emerge --ask --verbose --update --deep --newuse world
dispatch-conf

Remember: emerge prints out individual messages for certain packages containing necessary information on manual steps involved in an upgrade. Read and follow them if necessary. (Warning: Do not delete the setup_icedtea shell function in /etc/skel/.bashrc on the dispatch-conf run.)

Install GCJ by using the gcj-jdk package which supplies wrappers for providing a command line interface similar to the official Java Development Kits. In order to use GCJ, the system gcc has to be set to gcc-4.3. In addition, it is required to set gcj-jdk as the system virtual machine using the java-config utility.

gcc-config i686-pc-linux-uclibc-4.3.1
source /etc/profile

emerge gcj-jdk
java-config --set-system-vm=gcj-jdk

Having prepared a usable basic build environment, the next step is installing the high-level dependencies required by IcedTea. These consist of the Ant build system, the Xalan XSLT processor, and the Xerces XML parser library. For our custom scripts we will additionally need the Z shell later on.

emerge ant-eclipse-ecj xalan xalan-serializer xerces zsh

2. IcedTea Java Development Tools

2.1. Prepare environment and sources

In order to build IcedTea as non-privileged user you need to create a user account. In this guide we use the name minijava.

BUILD_USER="minijava"
useradd --create-home --user-group $BUILD_USER

Extract our IcedTea tarball into the home directory of the future build account and copy the associated OpenJDK sources into it.

tar xvf /usr/portage/distfiles/icedtea-minimal.tar.bz2 -C /home/$BUILD_USER/
cp /usr/portage/distfiles/openjdk-b29.zip /home/$BUILD_USER/icedtea-minimal/

Set ownership of the sources to the build user.

chown -R $BUILD_USER:$BUILD_USER /home/$BUILD_USER/

The base path used by the Java runtime environment for determining the location of its native libraries has to be hard-coded on the uClibc target. This is due to uClibc lacking support for the $ORIGIN syntax in its dynamic linker.

The location of this path has been set to /opt/icedtea as a default value in our uClibc port of IcedTea. We use a symlink for easy switching between different versions of this runtime environment. This symlink has to be set before building IcedTea, because the build process already uses this symlink during its later stages.

Early builds are located in the openjdk/build/linux-i586 subdirectory whereas the final development and runtime environment builds are placed in subdirectories of that directory. So we set a symlink from /opt/icedtea to the future build location, which does not exist yet.

ln -s \
  /home/$BUILD_USER/icedtea-minimal/openjdk-ecj/build/linux-i586/ \
  /opt/icedtea

The IcedTea build process exhibits random problems when run on a symmetric multiprocessing machine, e.g. computers with a multi-core cpu. However, it can be build on such computers by the use of the cpuset mechanism of the Linux kernel. The appendix contains a short guide to set up a cpuset with a single cpu in it and using it to build IcedTea.

2.2. Building IcedTea

Now you can switch to your build user and start building IcedTea. Note, however, that the $JAVA_HOME environment variable needs to be unset for the OpenJDK build system.

su - $BUILD_USER
cd icedtea-minimal/

unset JAVA_HOME

./configure \
  --with-parallel-jobs=1 \
  --with-gcj-home=$(java-config --jdk-home) \
  --with-libgcj-jar=$(java-config --runtime) \
  --with-ecj=/usr/bin/ecj \
  --with-ecj-jar=$(java-config -p eclipse-$(eselect ecj show)) \
  --with-openjdk-src-zip=./openjdk-b29.zip \
  --disable-docs \
  --disable-gcjwebplugin \
  --disable-zero \
  --without-ant-home \
  --host=i686-pc-linux-uclibc \
  --build=i686-pc-linux-uclibc \
  --target=i686-pc-linux-uclibc
make

The generated programs and shared libraries still contain all symbols which leads to an increased size. They can be removed by executing the following commands.

find openjdk/build/linux-i586/j2re-image \
  -type f -executable -exec strip \{\} \;
find openjdk/build/linux-i586/j2sdk-image \
  -type f -executable -exec strip \{\} \;

For using the previously built runtime environment or development kit you need to set the /opt/icedtea symlink to the corresponding directory and set the $PATH and $JAVA_HOME environment variables accordingly. This has to be done as user root inside the build system, so exit the user account first.

exit

mv /opt/icedtea /opt/icedtea.build
ln -s /home/$BUILD_USER/icedtea-minimal/openjdk/build/linux-i586/j2re-image \
  /opt/icedtea

The shell function setup_icedtea() is included in the .bashrc file of the build user for convenient environment setup:

setup_icedtea() {
  local jhome="/opt/icedtea"
  [[ -n "${1}" ]] && jhome="${1}"

  export JAVA_HOME="${jhome}"
  export PATH="${JAVA_HOME}/bin:${PATH}"

  if [[ -d "${JAVA_HOME}"/jre ]] ; then
        export PATH="${JAVA_HOME}/jre/bin:${PATH}"
  fi

  export JRE_BASE_DIR="${JAVA_HOME}"
}

3. Self-contained minimalistic operating environment

The final minimal image is based on Gentoo as well. The required compartment components are copied or cross-compiled from the chroot environment built in the previous sections.

We assume a plain hard disk image with a size of 50Mb as target. (Depending on later use in e.g. virtual machine setups one might have to partition it using fdisk or cfdisk and mount it differently). The path /images is used in this example, but this can be set differently.

Create an empty (disc) image, format and mount it.

IMAGE_DIR=/images
mkdir $IMAGE_DIR

dd if=/dev/zero of=$IMAGE_DIR/minimal.iso bs=1M count=50
mke2fs -F $IMAGE_DIR/minimal.iso

mkdir -p /mnt/loop
mount -o loop $IMAGE_DIR/minimal.iso /mnt/loop

Portage is able to use a directory different from the / root directory of the system as installation base path. It can also use a different root directory to read its configuration files from. This enables keeping different package configuration completely separated. Thus, it is possible to separate the base system from the evolving system, which is created by installing packages into it.

The provided xmerge wrapper script makes extensive use of this feature. In order to use it, it is necessary to set the $SYSROOT and $ROOT environment variables to the path containing our configuration files (source) and the mount point of the image respectively (destination).

It is now possible to build and install all packages required for running a minimal IcedTea-based runtime environment into the image.

gcc-config i686-pc-linux-uclibc-4.1.2
source /etc/profile

export SYSROOT=/gentoo-minimal
export ROOT=/mnt/loop

xmerge -av sys-libs/uclibc sys-apps/busybox sys-apps/baselayout-lite \
  sys-libs/libstdc++ sys-libs/zlib

4. Reduction and assembly of applications

In order to perform the Java runtime class file reduction we provide an experimental shell script which includes all the necessary stages for tailoring the Java runtime class libraries to an individual application. Dynamic profiling is used for obtaining a report of the possible minimal memory footprint. It is necessary to add the -verbose:class option on the call to the java interpreter because our script uses its classloader trace feature. Another requirement is the availability of a working jar tool program. In order to ensure the availability of a jar tool, either use the j2sdk version of IcedTea or have the system gcc set to the version used by gcj.

ln -s /home/$BUILD_USER/j2sdk-image /opt/icedtea

gcc-config i686-pc-linux-uclibc-4.3.1
source /etc/profile

4.1. Example: Testsuite of IAIK jTSS

Our first example is the IAIK jTSS stack which implements the TCG Software Stack in the Java programming language. Fortunately, it includes a set of unit tests which can be used for automatic execution of each of its functions.

setup_icedtea

cd jTSS/tests
export APPLICATION_BASE_DIR=$(pwd)/..

chmod 755 run_tests.sh
cp run_tests.sh trace_tests.sh
sed -i -e 's@java -cp@java -verbose:class -cp@' trace_tests.sh

After adding the -verbose:class argument to the call to the java program in the provided run_tests.sh shell script, it can be run with our jar-strip trace script.

jar-strip -u -a -n -o jTSS_env -f ./trace_tests.sh

The reduced application needs additional files which are not tracked by our trace script.

cp ../lib/*.ini jTSS_env/lib/app/
cp run_tests.sh jTSS_env/

It is also necessary to adapt the start script to include the new classpath.

TOPDIR="${0%/*}"
source "${TOPDIR}"/classpath.env

exec java -cp ${CLASSPATH} iaik.tc.tss.test.tsp.java.TestMain
#java -cp $CLASSPATH_LOCAL iaik.tc.tss.test.tsp.java.TestMain

4.2. Example: APKI PrivacyCA server

Another example is the apki server component which provides an implementation of a PrivacyCA. In this scenario the individual commands are executed on the client side whereas the server side binary is being traced. Therefore, it is necessary to add the -verbose:class argument to the java program call in the server.sh shell script. In order to execute all services on the server part a test script uses all desired commands.

setup_icedtea

cd apki/
export APPLICATION_BASE_DIR=$(pwd)

cp server.sh trace_server.sh
sed -i -e 's@java -cp@java -verbose:class -cp@' trace_server.sh

jar-strip -u -a -n -o apki_env -f ./trace_server.sh

It is necessary to terminate the server program after calling all client commands so the wrapper script can start analysing the server dependencies. In our test setup server and client run on the same host which enables us to do this by sending a SIGKILL signal to the server with the kill command. Otherwise, this would have to be done manually.

sudo -v && \
  ./run_tests.sh && \
  sudo kill -9 $(sudo netstat -l -p |& grep 10000 | \
  awk '{print $7}' | sed -n 's:\([[:digit:]]\+\)/java$:\1:p')

The script only tracks native libraries and Java archive files. Therefore, any additional file resources used by the application have to be copied into the output directory.

cp -r resources certstore quotedata apki_env/
cp server.sh apki_env/
cp lib/*.ini apki_env/lib/app/

Please note that the IAIK JCE is a signed jar archive which may inhibit its functionality when modified. You should also be familiar with its license should you choose to redistribute it. Therefore, we remove this library from the minimised application which requires to add it again when the image is to be used.

rm -f apki_env/lib/app/iaik_jce-3142.jar

The start script must be adapted so that the classpath includes the modified archive files. It should also use exec for the execution of the program so the process ID is kept for potential later use with a daemon service init script.

TOPDIR="${0%/*}"
source "${TOPDIR}"/classpath.env

exec java -cp ${CLASSPATH} iaik.tc.apki.APKIS $@
#java -cp ${TCCERT}:${TSP_LIB}:${JCE_LIB}:classes iaik.tc.apki.APKIS $@

Additionally, the /opt/icedtea symlink must be set to the now reduced runtime environment. In the case of the apki example this would be similar to the following commands (issued by the root user).

rm -f /opt/icedtea
ln -s /home/$BUILD_USER/apki/apki_env/lib/jre /opt/icedtea

4.3. Assembly of Compartment

The final assembled compartment is a merge of the base system built in section 3.3 with the result of the Java application and runtime reduction demonstrated in section 4.1 or 4.2.

Mount an empty minimal environment as built in section 3.3 to /mnt/loop and chroot into it.

cd /mnt/loop
mount -t proc none proc/
chroot . /bin/sh

Tweak system configuration as needed for your application. For example, it is considered good practice to use unprivileged users for different services. You can leave the chroot environment after completing the user management.

USER=username
adduser $USER

passwd
passwd $USER

adduser -D -H apki


HOSTNAME=example_compartment
echo $HOSTNAME > /etc/hostname

exit

Add an application to the compartment. In the following example, the apki server is used. It is self-contained after the reduction and modification step in section 4.2 and already includes an adapted language runtime environment. Therefore, it is possible to simply copy it into the compartment image.

mkdir /mnt/loop/opt
cp -r apki_env /mnt/loop/opt/
APKI_USER_GROUP="$(sed -ne 's/^apki:x:\([[:digit:]]\+:[[:digit:]]\+\):.*$/\1/p' \
  /mnt/loop/etc/passwd)"
chown -R ${APKI_USER_GROUP} /mnt/loop/opt/apki_env

ln -s /opt/apki_env/lib/jre /mnt/loop/opt/icedtea

To control a daemon process it is useful to have an init script which can be used by the base system. An example would be the script we use for apki:

#!/bin/sh

PIDFILE="/var/run/${0##*/}"
SERVERDIR="/opt/apki_env/"
EXECUTABLE="${SERVERDIR}/server.sh"

DAEMON_USER="apki"
DAEMON_GROUP="apki"

JAVA_HOME="${SERVERDIR}/lib/jre" ; export JAVA_HOME
PATH="${JAVA_HOME}/bin:${PATH}"  ; export PATH

start() {
  cd "${SERVERDIR}"
  start-stop-daemon \
    --quiet \
    --background \
    --make-pidfile \
    --pidfile "${PIDFILE}" \
    --chuid ${DAEMON_USER}:${DAEMON_GROUP} \
    --exec "${EXECUTABLE}" \
    --start
}

stop() {
  start-stop-daemon \
    --quiet \
    --pidfile "${PIDFILE}" \
    --signal KILL \
    --stop
}

[ "${1}" = "start" ] && start
[ "${1}" = "stop" ] && stop

Copy the script into the /etc/init.d directory of the mounted image and create symlinks for automatic startup and shutdown of the service.

cp apki.sh /mnt/loop/etc/init.d/
cd /mnt/loop/etc/init.d
ln -s apki.sh S01apki
ln -s apki.sh K01apki

Please note that the environment - especially $LANG $LC_ALL etc. variables - have to be cleaned up if the compartment is to be tested in a chroot jail.

cd /mnt/loop
mount -t proc none proc/
chroot . /bin/sh

unset LANG LC_ALL LC_CTYPE LC_MESSAGES LC_COLLATE
/etc/init.d/apki.sh start

/etc/init.d/apki.sh stop

4.4. Comparison

This table shows the approximate size reduction achieved in our experimental setups.

The footprint of the base system can be reduced by using applications and libraries oriented towards embedded systems. However, the C++ standard library which is required by the Hotspot VM accounts for most of its size being roughly 3.5 MB large.

Reducing the runtime environment to only those native libraries that are crucial for VM execution has a strong impact on its size. In this case our trace script copies only those libraries which were loaded on the reference execution of the downsized application. These libraries - including the main VM library - amount to roughly 4.1 MB in our example cases for apki and jTSS.

Another major influence on the overall size of the runtime environment, additional libraries, and the target application itself are their binary class files of which we include only those which were loaded during the reference execution.

	Standard components	Minimalised components
Base system	34 MB	5.7 MB
JRE	92 MB	7.1 MB
jTSS libs	753 KB	291 KB
apki app	110 KB	60 KB

5. Software Packages

This guide is up to date as of February 2009. The described build process is Gentoo Linux based. As Gentoo is a rolling upgrades distribution some information or packages provided here may already be out of date or incompatible to current packages and thus fixes may be necessary.

The following components are used in this guide (checksums are SHA1):

Gentoo x86 uClibc, stage 3:
0f3ad6c1e53c126166dae51c0513d4ff391f89f3 stage3-x86-uclibc-2008.0.tar.bz2
The most recent standard stage 3 for x86-uclibc is located in the experimental/x86/embedded/stages/ subdirectory on a Gentoo Linux mirror.
Gentoo Portage tree snapshot:
7c67ee153b2c262cfa364b3c8d64616851e05e08 portage-20090120.tar.bz2
The most recent Portage snapshot is located in the snapshots/ directory on a Gentoo Linux mirror.
Custom GCC patches:
2949be07bf99ac55fb68eebfd6c2976c6f3200e1 gcc-4.3.1-uclibc-patches-1.0.tar.bz2
These patches modify GCC 4.3.1 to be compatible with uClibc. The original patchset which can be obtained from the Gentoo CVS Repository does not consider libjava which is part of GCJ. Our modifications take this into account.
Custom Portage configuration files:
ea1ad6049c2752f919c5fad68f196e94713121f0 gentoo-config.tar.bz2
This file is provided by us. It overwrites the standard Gentoo system configuration files with our modifications to allowing building IcedTea directly from GCJ.
Custom Portage configuration files for the minimal compartment:
19a168c73ebb47bd8dfe6337657902bc31d3e5e0 gentoo-minimal.tar.bz2
This file is provided by us. It provides a set of configuration files for a minimal Gentoo Linux system.
Custom Baselayout Lite
fdabfe76ad4012df60f4bdfc20d66c6d60161a55 baselayout-lite-1.0_pre1.tar.bz2
This file contains the old baselayout lite base system files from the Gentoo CVS repository. Custom patches are applied by our ebuild.
Custom IcedTea 1.7
8650b2d8be63a81c590c60823674d7cd8f7227bc icedtea-minimal.tar.bz2
Provides a custom version of IcedTea 1.7 which is able to run on the uClibc system library and has a reduced dependency set.
OpenJDK b29
39cbc64c30d40323da55e9b2c71fe03837378e3e openjdk-b29.zip
Copy of the original b29 snapshot of the OpenJDK development tree.

If you do not want to do every step described in this document, we provide prebuilt images:

Base system image prepared for building IcedTea:
c93f13c2bdc829a8de32a65a83bbb40bd60d4159 prebuilt_base_system.tar.bz2
A snapshot of the system used to build our adapted IcedTea version including all necessary dependencies.
Base system image with pre-built IcedTea:
c9b9f305ef6d4520acd5a3a24971034f05b43568 prebuilt_base_system_icedtea.tar.bz2
Snapshot of the base system including a prebuilt images of the IcedTea runtime environment.
Image of the bare minimal compartment:
5d608680874eecca415c21a0ae79d30ebf16f911 prebuilt_minimal_bare_compartment.tar.bz2
Contains the bare minimal compartment which can be used to execute a minimized application.
Image of the apki minimal compartment:
4d0430384592661ad78111ff7b5fa4aa2c647479 prebuilt_minimal_apki_compartment.tar.bz2
Contains a minimal compartment including a minimised apki PrivacyCA server. Note that you need to install missing libraries manually into /opt/apki_env/lib/app/. For example IAIK-JCE is available from http://jce.iaik.tugraz.at/.

Both images of the base system include the source files of the included software in the usr/portage/distfiles directory.

6. Appendix

This section provides additional background information which may be useful for building this setup from scratch.

6.1. Sync and Upgrade of Gentoo base system

The Portage snapshot provided is most likely out of date. You may sync and upgrade the whole system after first chrooting into it. This, however, will most likely introduce issues. You are on your own, if you choose to deviate from the packages and versions used throughout this guide!

Synchronize the portage tree and rebuild the world package set:

emerge-webrsync
emerge --ask --verbose --update --deep --newuse world

If the used stage is unreasonably old, i.e. using an outdated toolchain (gcc/glibc) you might have to rebuild larger parts of the system set. Please read the relevant upgrade guides from http://www.gentoo.org/doc/en/, especially the Gentoo GCC Upgrade Guide.

Also, the Gentoo package manager does not automatically update configuration files in order to preserve changes made by the user. Therefore, it is important to merge in changes manually if portage reports new configuration files updated by some package.

6.2. Using cpusets for limiting a chroot compartment to a single core

Newer Linux kernels support cpusets to restrict the execution of tasks and their children to specified CPUs. They use the cgroup subsystem, so make sure you have CONFIG_CGROUPS enabled in your kernel configuration. The following example assumes a quad-core CPU.

Cpusets are controlled by a pseudo file system which has to be mounted first.

mkdir -p /dev/cpuset
mount -t cpuset -o cpuset cpuset /dev/cpuset

You can create new cpusets by simply creating new directories in this file system.

cd /dev/cpuset
mkdir single triple

It is necessary to set the memory node which is going to be used by the processes in the different sets.

echo 0 > single/mems
echo 0 > triple/mems

In this example, we use the first three cpu cores for the triple set whereas the single set only gets to use one core.

echo '0-2' > triple/cpus
echo 3     > single/cpus

To make use of the assignment of specific cores for a cpuset we define exclusive access for them.

echo 1 > single/cpu_exclusive
echo 1 > triple/cpu_exclusive

Now you can start adding process ids to the different sets. Our example assigns every process currently running to the triple set. Due to short-living processes the following command can lead to error messages which can be safely ignored.

for pid in $(cat /dev/cpuset/tasks) ; do
  echo ${pid} > triple/tasks
done

To have exclusive access to one core for our chroot, we add the process id of the shell being used ($$) to the single set and chroot into it.

echo $$ > /dev/cpuset/single/tasks
cd /my-build-dir
chroot . /bin/bash
[...]

7. Attributions

Sun, Sun Microsystems, Java, Java runtime environment, JRE, JVM, J2RE, JDK, and OpenJDK are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

The original efforts at IAIK to integrate TC technology into the Java programming language are part of the OpenTC project funded by the EU as part of FP-6, contract no. 027635. The project aims at providing an open source TC framework with a special focus on the Linux operating system platform. Started as an open source project the results can be inspected by everybody, thus adding towards the trustworthiness of Trusted Computing solutions.