sortix--sortix/doc/cross-development

Sortix Cross Development Guide
==============================

The primary way to build Sortix is on Sortix itself. However, the system may not
be fully up to the task yet and your favorite development environment may not
have been ported. You can cross-compile Sortix from a reasonable similar system
such as GNU+Linux. The build process will liberally use common GNU tool options,
so you may wish to install the basic GNU distribution or compatible.

Overview
--------

To build Sortix you need these components:

* Sortix source tree
* Sortix binutils
* Sortix gcc
* GRUB2 (iso support)
* xorriso (iso support)

In this tutorial we will cross-compile a Sortix using by following these steps:

1. Retrieve all the source code.
2. Build additional Sortix-specific tool programs.
3. Create a system root containing all the Sortix system headers.
4. Compile a new compiler that targets Sortix and uses the system root.
5. Actually compile Sortix.

It is necessary to compile binutils and gcc because your system compilers
doesn't produce executables for Sortix, but rather for your current system.

Source Code
-----------

You can find the latest Sortix source code at:
https://gitlab.com/sortix/sortix

You can find the latest Sortix binutils source code at:
https://users-cs.au.dk/sortie/sortix/toolchain/sortix-binutils-latest.tar.xz

You can find the latest Sortix gcc source code at:
https://users-cs.au.dk/sortie/sortix/toolchain/sortix-gcc-latest.tar.xz

It is only possible to build Sortix with this blessed toolchain or a derivative
thereof as it has been customized to understand Sortix and provide a number of
useful extensions. You may need to occasionally upgrade the toolchain, in the
event that it is further modified and Sortix takes advantage of these changes.

Variables
---------

This guide uses shell variables to denote where you have choice. You would
typically decide to set them up somewhat like this:

  # The Sortix source code is in /home/user/sortix
  # The cross-compiler is installed in /home/user/opt/x86_64-sortix

  SORTIX_PLATFORM=x86_64-sortix
  CROSS_PREFIX=/home/user/opt/x86_64-sortix
  SORTIX_SYSROOT=/home/user/sortix/sysroot
  export PATH="/home/user/opt/x86_64-sortix/bin:$PATH"
  export PATH="/home/user/opt/x86_64-sortix/sbin:$PATH"

The following sections describe these variables in detail.

Target Platform
---------------

You need to decide what the platform your final Sortix system will run on. You
can currently decide between i486-sortix and x86_64-sortix. In this guide we
will refer to that platform triplet as $SORTIX_PLATFORM. If you want to build
another platform afterwards, then you will have to follow this guide again.

Cross-Environment
-----------------

You should install your cross-toolchain into a useful and isolated directory
such as $HOME/opt/$SORTIX_PLATFORM. This allows you to easily dispose of the
directory and keeps it neatly isolated from the rest of the system. Let's refer
to that location as $CROSS_PREFIX.

You need to add $CROSS_PREFIX/bin and $CROSS_PREFIX/sbin to your $PATH variable:

  export PATH="$CROSS_PREFIX/bin:$PATH"
  export PATH="$CROSS_PREFIX/sbin:$PATH"

This will modify the $PATH variable in this particular shell session. Depending
on your shell and system configuration you can make this permanent by adding
that line to your ~/.profile. Consult your shell documentation. Otherwise type
it in all Sortix-related shells before doing anything.

Sortix build tools
------------------

You need to install some additional Sortix programs into your local environment
as they are needed to build Sortix. Assuming you cloned the Sortix source code
tree into the ~/sortix directory, you can install them by typing the following
commands:

  cd ~/sortix &&
  make PREFIX="$CROSS_PREFIX" clean-build-tools &&
  make PREFIX="$CROSS_PREFIX" build-tools &&
  make PREFIX="$CROSS_PREFIX" install-build-tools &&
  make distclean

These tools produce platform independent output so you may wish to install them
into $HOME or /usr/local or where it suits you, and add its bin and sbin to your
$PATH.

Setting up basic system root
----------------------------

The compiler has been taught how to correctly build programs for a Sortix
system, however it gets part of this information from the system headers. The
first step is to create a basic system root that contains the headers and
filesystem structure for the final system. We will install the kernel, libc,
libm, and other core library headers into this directory. Run the following:

  cd ~/sortix &&
  make sysroot-base-headers HOST=$SORTIX_PLATFORM

This will automatically set up a basic system root in ~/sortix/sysroot. We will
use this when we build the cross-compiler in a moment. Let's call the system
root directory $SORTIX_SYSROOT.

Cross-Toolchain Dependencies
----------------------------

You may need to install these packages (and their development packages) before
building binutils and gcc, or the packages will refuse to configure or some
obscure error will occur. In particular, you may need to install:

* Bison
* Flex
* GNU GMP
* GNU MPFR
* GNU MPC

You should consult the official binutils and gcc documentation on what exactly
you need to install before building these packages yourself.

Cross-Binutils
--------------

We need a cross-binutils that have been taught the Sortix program format. If you
have extracted the Sortix binutils source code into ~/src/sortix-binutils then
you can build binutils out-of-directory using these commands:

  mkdir ~/src/binutils-build &&
  cd ~/src/binutils-build &&
  ../sortix-binutils/configure \
    --target=$SORTIX_PLATFORM \
    --with-sysroot="$SORTIX_SYSROOT" \
    --prefix="$CROSS_PREFIX" \
    --disable-werror &&
  make &&
  make install

You may need to install some dependencies before you build binutils. You can
remove the temporary ~/src/binutils-build directory when you are done.

Cross-GCC
---------

We need a cross-compiler that has been taught what Sortix is and that supports a
few extensions that have been added to Sortix gcc that isn't present in upstream
gcc. If you have extracted the Sortix gcc source code into ~/src/sortix-gcc then
you can build gcc out-of-directory using these commands.

  mkdir ~/src/gcc-build &&
  cd ~/src/gcc-build &&
  ../sortix-gcc/configure \
    --target=$SORTIX_PLATFORM \
    --with-sysroot=$SORTIX_SYSROOT \
    --prefix="$CROSS_PREFIX" \
    --enable-languages=c,c++ &&
  make all-gcc all-target-libgcc &&
  make install-gcc install-target-libgcc

You may need to install some dependencies before you build gcc, such as libmpc,
libmpfr and libgmp. You can remove the temporary ~/src/gcc-build directory when
you are done. Notice how we don't build all of gcc as this isn't possible yet.
For instance, the C++ library requires the Sortix libc to have been built.

Building Sortix
---------------

You now have the needed components to build a working Sortix system. If you have
completed the above steps correctly, then you can simply do:

  cd ~/sortix &&
  make HOST=$SORTIX_PLATFORM

This will compile a basic Sortix system into ~/sortix/sysroot with a kernel,
headers, libraries, programs, everything you need. This isn't a bootable system
yet. You need the combination of a kernel and an initrd to boot Sortix. The
initrd is a root filesystem entirely in memory that is loaded by the bootloader
in addition to the kernel. You can generate the initrd in builds/ by running:

  cd ~/sortix &&
  make HOST=$SORTIX_PLATFORM initrd

If you want a copy of the kernel and initrd in the current directory (rather
than normally finding them in builds/ and sysroot/), you can run:

  cd ~/sortix &&
  make HOST=$SORTIX_PLATFORM sortix.bin sortix.initrd

You now have a sortix.bin and sortix.initrd pair. You can boot Sortix using a
multiboot bootloader by passing them as a multiboot kernel and multiboot
module/initrd.

If you have xorriso and grub-mkrescue from GRUB 2 installed, then you can
can build a bootable .iso by typing:

  cd ~/sortix &&
  make HOST=$SORTIX_PLATFORM sortix.iso

This will produce a sortix.iso file that is bootable on real hardware and
virtual machines. This works by first building Sortix system and packaging up an
initrd, then it create a cdrom image with a bootloader configured to load the
kernel and initrd stored on the cdrom. If the command fails, see below.

You can clean the source directory fully:

  cd ~/sortix &&
  make distclean

This will clean all the source directories and ports and delete all the builds
and the binary package repositories (if you use ports as below) and it will
delete the system root containing your installed system. Alternatively, you may
be interested in the target `mostlyclean `that doesn't delete binary packages
for ports, as they may take considerable time to recompile. See also the targets
`clean`, `clean-core`, `clean-sysroot`, `clean-repository`, `clean-builds`,
and `clean-ports` which lets you control what is cleaned.

Building Ports
--------------

You can extend your Sortix system with third party software that has been ported
to Sortix. You can find the suitable ports if you visit the download directory
for this release and enter the subdirectory `srctix` which contains compressed
archived source tixes. Each such file is simply a compressed tarball that
contains a single directory with the source code for this port. If you have the
file `libfoo.srctix.tar.xz`, you can extract it into ~/sortix/ports as such:

  cd ~/sortix &&
  mkdir -p ports &&
  cd ports &&
  tar --extract --file $HOME/Downloads/libfoo.srctix.tar.xz

This will create the directory `~/sortix/ports/libfoo` that contains the source
code for the libfoo port. Keep in mind that many ports depends on another more
basic ports and you will need to satisfy the dependencies. The build process
will give an error and tell you which dependency was missing if you haven't
satisfied the dependencies. Other ports have optional dependencies which gives
an inferior installation if the dependencies are not satisfied.

If you have installed the Tix package management (which is done by the
`make install-build-tools` command above), then the top-level makefile will
automatically detect all the installed ports. They will automatically be built
along with the core system the next time you build Sortix and is present in
your initrd and bootable images.

For more information on ports and how they work, please read:

  $MY_LOCAL_DOCUMENTATION_MIRROR/porting-guide

Building some ports may require additional tools to be installed on your system
and other unforeseen problems may arise that means the port doesn't compile
properly on your system. Should a port fail to compile, then the `tix-build`
command will offer you a chance to investigate the situation in a shell.

Troubleshooting
---------------

If producing a bootable cdrom with grub-mkrescue gives the error

  xorriso : FAILURE : Cannot find path '/efi.img' in loaded ISO image

then your GRUB grub installation is defective. You need to install mformat(1) to
use grub-mkrescue.