sortix--sortix/doc/user-guide

Sortix User Guide
=================

Thanks for your interest in Sortix! This document will describe how to use it
and what features are available. Sortix is a small self-compiling Unix-like
Operating System developed since 2011. The system is free software licensed
under the GNU General Public License and other licenses.

Introduction
------------

If you are using a Sortix live medium or a prebuilt harddisk image, you will
likely be met with a bootloader. Normally, GNU GRUB is used as the Sortix
bootloader. In that case, simply pick Sortix from the boot menu. If you have
installed Sortix manually, you will have to configure your bootloader to boot
Sortix.

The bootloader will then load the Sortix kernel and start the system. The
initialization process will spawn a shell shortly. The command line is similar
to those of Unix systems and many common Unix commands are available.

Keyboard Layout
---------------

The kernel translates the raw keyboard scancodes into unicode codepoints using
the current keyboard layout table associated with the terminal. Such layout
tables are compiled from a textual description into compressed binary tables
using the `kblayout-compiler` program. The kernel has a default US keyboard
layout integrated into it at compile time.

You can change the current keyboard layout using the `chkblayout` program. It
loads a keyboard layouts with the requested name from the `/share/kblayout`
directory, or a user-supplied file if the path contains a slash. The system
comes with a few existing keyboard layouts and more can be added if desired.

* `dk'
* `us'

You can select a particular keyboard layout by typing something like:

    chkblayout da_DK

Mind that while the user-space programs and the kernel all handle text correctly
as unicode, your console may not be able to render such code points. VGA text
mode consoles, for instance, only has the VGA font available, which contains
just codepage 437 characters. The console driver helpfully translates some
unicode characters that look like codepage 437 characters into their lookalikes
for rendering. The graphical kernel console reuses the VGA font internally, but
has no hardware restrictions. In principle, support could be added such that
user-space can upload font data to the kernel graphical console.

Changing the Video Mode
-----------------------

After boot, the screen resolution will be default be in VGA text mode. This only
allows 80 columns and 25 rows of 16 color ASCII text. Naturally, this is very
limited but works on most systems.

You can change the display resolution using the `chvideomode` program. However,
you need a suitable kernel driver to do the mode setting. The current release
only has a simple BGA driver, which works in virtual machines such as Qemu,
VirtualBox or Bochs.

The BGA driver can support virtually any resolution, but since the interface
fails to report which resolutions are appropriate in the current context, I have
simply hard-coded whatever resolutions I found reasonable when I wrote it.

Powering Off
------------

Unless you are using a harddisk image or your initrd has been specially
configured to mount a permanent root filesystem, then this environment is
entirely temporary. You will loose all modifications when powering off the
current system.

The system automatically reboots when the initial shell exits. You can exit the
current shell by entering `exit`, which if entered from the initial shell will
reboot the entire system (losing any changes if the root filesystem is not
permanent). Additionally, you can press Control-D ('^D') to exit the current
shell, but the initial shell will require you to enter a literal `exit` before
it exits and the system reboots.

The init process will restart the shell in the event that it crashes or dies
from a signal. You can use the `kill $$` command to kill the current shell, and
if it is the initial shell, you will get a new shell.

More Documentation
------------------

The system documentation (at least what has been documented so far) is installed
in the `/share/doc` directory. You can view these documents (well, the ones that
are plain text, there is no `man` program yet) using the `editor` program, which
you exit by pressing Control-Q ('^Q'). It is probably a good idea to change
the resolution using the `chvideomode` program if drivers are available.

If your release contains third party software, then this directory may also
contain documentation for this software, which may or may not be easily
viewable using the editor.

Filesystem Structure
--------------------

This as an incomplete list of the common top level directories in Sortix.

* `/bin` - Programs.
* `/boot` - Kernel images, initrd images.
* `/dev` - Device filesystem.
* `/etc` - System-wide configuration.
* `/home` - User directories.
* `/include` - Header files.
* `/libexec` - Non-user programs.
* `/lib` - Shared libraries.
* `/root` - Directory of the root user.
* `/share` - Architecture independent files.
* `/share/doc` - Documentation.
* `/share/kblayout` - Keyboard layouts.
* `/src` - System source code.
* `/tmp` - Temporary files.

Some of these directories not be present or empty on your installation.

Using the Shell
---------------

Sortix features a homemade shell, which at best can be described as hacky.
Despite that, it does have some key features. Here are the features that are
supported:

* Processes can be started in the usual Unix manner.
* Tab-completion.
* Line editing.
* History.
* Background tasks ('&').
* Standard redirection ('<', '>', '>>').
* Piping stdout from a task to stdin of another ('|').
* Stopping the currently running task (Control-C, '^C').
* Waiting for a task to complete and executing the next (';').
* Setting environmental variables ('FOO=bar').
* Using $FOO to insert an environmental variable.
* Listing all environmental variable ('env').
* Changing the current directory ('cd /home').
* Removing an environmental variable ('unset FOO').
* Clearing all enviromental variables ('clearenv').
* Executing shell scripts ('sh script.sh param1 param2').
* Escaping some special characters ('\').
* Comments ('#').
* Exiting the shell ('exit') ('exit 42').
* Basic wildcards ('foo*.bar').
* Clearing the screen (Control-L, '^L').
* Deleting the last typed word (Control-W, '^W').
* Deleting everything before the cursor (Control-U, '^U').
* Deleting everything after the cursor (Control-K, '^K').
* Go to start of line (Control-A, '^A', Home).
* Go to end of line (Control-E, '^E', End).
* Move between words (Control-arrows).
* Quotes (''') ('"').

These features are missing from the shell:

* Proper recursive shell grammar parsing.
* Heredoc sequences.
* Command substitution ('$(ls)', '`ls`').
* Arithmetic expansion ('$((2 + 2))').
* Aliases.
* Conditions and loops ('if', 'case', 'while', 'until', 'for').
* Functions.
* Some special parameters ('$@', '$*', '$#', '$-', '$!).
* Special shell variables ('$ENV', '$IFS', '$LINENO').
* Shell prompt customization ('$PS1', '$PS2', '$PS4').
* Tilde expansion ('~/foo').
* Any but the simplest variant of Parameter Expansion.
* Proper shell wildcard support ('*/*.foo').
* Ability for variables to not be exported.
* Export keyword ('export FOO', 'export BAR=bar').
* Subshells.
* Proper redirection to/from specific file descriptors.
* And much more; the shell remains hacky.

If a better shell is installed, and the shell is non-interactive, then the sh
program will automatically run the superior shell transparently.

Included Programs
-----------------

Sortix comes with a number of home-made programs. Here is an overview:

* `asteroids` - remake of the classic asteroids game
* `basename` - strip directory from filenames
* `benchctxswitch` - useless benchmark
* `benchsyscall` - slightly less useless benchmark
* `carray` -  convert a binary file to a C array
* `cat` - display file on terminal
* `chkblayout` - change keyboard layout
* `chmod` - change file mode bits
* `chroot` - change the root directory
* `chvideomode` - change display resolution
* `clear` - clear terminal
* `colormake` - colorful version of make (if make is available)
* `column` - format lines nicely in columns
* `cp` - copy file
* `date` - display current time and date
* `df` - report file system disk space usage.
* `dirname` - strip last component from file name
* `dispd` - non-existent display server
* `du` - report file and directory disk usage
* `echo` - print command line arguments
* `editor` - text editor
* `env` - run a program in a modified environment
* `expr` - evaluate expressions
* `extfs` - ext2 filesystem server
* `false` - exit with an error status
* `find` - recursively list files
* `head` - display start of file
* `help` - show list of available programs
* `id` - print real and effective user and group IDs
* `init` - system management deamon
* `initrdfs` - tool for examining initrds
* `install` - installs a program into a system directory
* `kernelinfo` - display kernel information
* `kill` - send signal to process
* `ln` - create filesystem links
* `ls` - list contents of directory
* `mbrfs` - create partitions for master boot record block device
* `memstat` - print memory information
* `mkdir` - create directory
* `mkinitrd` - create an initrd
* `mktemp` - create temporary file or directory
* `mv` - move a file
* `pager` - display file page by page
* `ps` - report a snapshot of the current processes
* `pstree` - display a tree of processes
* `pwd` - print current directory path
* `realpath` - canonicalize filesystem paths
* `regress` - run system tests
* `rm` - remove file
* `rmdir` - remove empty directory
* `sh` - command shell
* `sleep` - delay for a specified amount of time
* `sort` - sort lines of text files
* `stat` - display file status
* `tail` - display end of file
* `time` - measure program running time
* `trianglix` - triangle system
* `tr` - translate, squeeze and/or delete characters
* `true` - exit with a success status
* `type` - type raw characters directly into the terminal
* `uname` - system information
* `uniq` - report or omit repeated lines
* `unmount` - unmount filesystem
* `uptime` - time since initialization
* `wc` - count words and lines
* `which` - find path to command
* `yes` - repeatedly output a line

Third Party Software
--------------------

In addition, a selection of third-party software has been ported and may be
present on your installation. In particular, the following software packages are
known to work at some level of stability:

* binutils
* bison
* bochs
* bzip2
* dash
* diffutils
* gawk
* gcc
* gettext (GNU)
* git
* grep (GNU)
* groff
* gzip (GNU)
* hello (GNU)
* libassuan
* libatk
* libav
* libcairo
* libdaala
* libdbus
* libexpat
* libffi
* libfontconfig
* libfreetype
* libgcrypt
* libgdk-pixbuf
* libglib
* libGL (Mesa)
* libgmp
* libgnutls
* libgpg-error
* libgtk
* libharfbuzz
* libiconv
* libjpeg
* libksba
* libmpc
* libmpfr
* libnettle
* libogg
* libpango
* libpixman
* libpng
* libSDL
* libssl (LibreSSL)
* libstdc++
* libtheora
* libvorbis
* libwayland
* libxkbcommon
* libz (Sortix)
* m4 (GNU)
* make (GNU)
* patch (GNU)
* python
* quake
* sed (GNU)
* tar (GNU)
* xz
* (and more)

More software will be ported in the future as the system matures. Your release
may not contain all the ported software because of disk space constraints or
the stability/usefulness of the package or the difficulty to build.

Included Games
--------------

The system comes with a number of small casual games. Note that some games
require running in the correct display mode before they will function.

### Asteroids ###

Mine for crystals in an asteroid field! Start the `asteroids` program and if you
have the needed driver support, you can explore space in a gloriously rendered
asteroid field of bitmap graphics. Use the arrow keys to navigate. Avoid the
white asteroids as they will destroy your space ship. Use the `space` key to
fire a laser beam that breaks the asteroid apart, but beware that the fragments
will accelerate. Alternatively, use the `Left Control` key to shoot a fireworks
of laser beams, but it doesn't reach as far and may split a big asteroid into
many small dangerous asteroids. Your goal is to collect as many blue crystals as
possible without dying. You can use the `B` key to spawn a bot that will
cluelessly follow you and shoot whatever asteroids it deems dangerous. You can
use the `A` key to spawn a black hole (or just a big attracting thing).

System Source Code
------------------

The entire system source code (except third party components) is installed into
the `/src` directory. You can use the `editor` program to view and edit the
system source code. If your release contains the GNU compiler collection (gcc),
you can even modify the system and compile custom programs.

Editing Files
-------------

You can use the `editor` program to edit files. The editor itself is fairly
simple to use. It currently supports these keyboard commands:

* `Ctrl-C` - Copy
* `Ctrl-I` - Go to line
* `Ctrl-K` - Cut
* `Ctrl-O` - Open a file
* `Ctrl-Q` - Exit
* `Ctrl-S` - Save a file
* `Ctrl-V` - Paste
* `Ctrl-X` - Cut
* `ESC language <c or c++>` - enable syntax highlighting
* `ESC line-numbering <on or off>` - enable line numbering
* `ESC margin <column>` - add right margin at column
* `ESC popen <command>` - open command output
* `ESC tabsize <desired-tab-size>` - change tab size

It is not currently possible to port third party editors because the terminal
implementation is not standards-compliant enough and is seriously lacking.

Partitions
----------

If the initialization code didn't automatically create block devices for your
partitions, but the harddisk block device itself is supported, you can use the
`mbrfs` program to create block devices for the partitions.

    mbrfs /dev/ata0

The program will output the names of the new block devices.

Mounting Filesystems
--------------------

The `extfs` filesystem server translates a block device formatted with the
second extended filesystem and mounts it at an empty directory. The filesystem
may need to be carefully configured as not all ext2 features are currently
supported. In particular, only the `large_file` and `filetype` features are
supported. There is no support for formatting or checking filesystems yet, this
will have to be done from an external system. If the device `/dev/ata0p1`
contains an ext2 filesystem, then it can be mounted using:

    mkdir /mnt
    extfs /dev/ata0 /mnt

You can then access the filesystem at `/fs`. You can unmount it safely when you
are done using it:

    unmount /mnt

This will safely shut down the filesystem process and sync to disk. Beware how
init doesn't know about your personal mount points and powering off the system
will cause all processes to be destroyed, and such mount points will not be
shut down gracefully at the moment.

Graphical User Interface
------------------------

The `dispd` display server is still under development. Sortix does not feature
any documented graphical user interface at the moment.

Sortix comes with the orthogonal graphical multi-user revolutionary triangle
system Trianglix, an attempt at making the most foreign yet somehow usable user
interface. You just need to run `trianglix` to start it.

Network
-------

Network support is still under development and is not documented at this point.
Unix sockets are available.

Building Sortix under Sortix
----------------------------

You can build Sortix under itself and make modifications to it. It is not yet
possible to build all the the third party software used to build itself, but you
can use the system to improve it until it is possible to build third party
software under it.

You need a version of Sortix that ships its own source code in `/src` and
contains a copy of the GNU Compiler Collection (gcc). Additionally, you should
be booting from a harddisk so that the new kernel is used after a reboot.
Otherwise you can only update to a new user-land.

You can use the `colormake` program instead of regular `make` if you want
colored output, which eases spotting compile warnings and errors. This program
simply invokes the real `make` and colors its output.

The `/src` directory contains a makefile that eases rebuilding the entire system
automatically. To rebuild the entire system simply run:

    cd /src
    make
    chroot /src/sysroot init   # to test the new user-land
    make sysmerge              # to update to the new user-land

This will recompile the entire operating system and install it into the staging
area `/src/sysroot`. This is a full usable image of the root filesystem for a
Sortix installation and you can chroot into it if desired.

The sysmerge target merges the image in `/src/sysroot` onto the current root
filesystem without changing local files, as opposed to a make install that
indiscriminately overwrites everything.

You will be running a new version of everything upon completion, except the
kernel and existing processes will continue to run the old programs. If the root
filesystem is permanent and the computer has been configured to boot from it,
then you can reboot the system by exiting the initial shell. You will then be
running an entirely new system.

### Building System Components ###

The projects that are part of the core system can be easily built and installed
with the provided makefiles. They can be built with this simple sequence:

    cd /src/${package}
    make
    make install

If you wish to install the package somewhere than the default location, you can
use the `PREFIX` (and, depending on your needs, `EXEC_PREFIX`, `BINDIR`,
`LIBDIR`, ...) environmental variable to specify where the program will be
installed:

    make PREFIX=/local
    make PREFIX=/local install

Additionally, should you require that the package should be installed into a
temporary location before it is fully installed, use the `DESTDIR` environmental
variable:

    make PREFIX=/local
    make PREFIX=/local DESTDIR=/temporary-location

This will effectively install the package into `/temporary-location/local`, but
the package will expect that it is installed into `/local` when it is run. This
is useful for package management purposes to capture a copy of all the installed
files before they are installed for real.

### The C library ###

The C library (libc) implements the common functions, data types and constants
required by the C programming language standard, POSIX, other standards, and
various Sortix-specific extensions. Most programs for Sortix directly or
indirectly rely on this core library for basic functionality and system calls.
The build process builds a regular user-space C library as well as a
freestanding C library designed for inclusion into the Sortix kernel. After
rebuilding the C library, you may wish to rebuild the entire user-land since
those programs have an older libc version statically linked into them.

### The Math Library ###

The Math Library (libm) provides useful mathematical functions for manipulating
floating-point numbers of various sizes. This implements the <math.h> header as
known from standard C. Like the C library, you will likely want to recompile
large parts of user-land if you update this library.

### The Pthread Library ###

The Pthread Library (libpthread) provides a threading implementation through the
standard header <pthread.h> as known from POSIX. Like the C library, you will
likely want to recompile large parts of user-land if you update this library.

### The Display Daemon ###

The dispd library allows processes to communicate with the dispd server that
handles window management and graphical output. Currently, there is no such
server and the library calls the kernel interface itself. This library allows
games such as asteroids to detect the current resolution and request whatever
resolution they need to function.

### Utility Collection ###

The utility collection contains common programs such as `ls`, `cp`, `mkdir` and
more. These programs allow the basic operation from the command line.

### Benchmarks ###

The system comes with some small and outdated benchmark programs. They don't
currently give any meaningful values, so you should not use them for anything.

### Games ###

The games directory contains the source code for the above mentioned games.
These depend on libdispd.

### mbrfs ###

The `mbrfs` program creates block devices for every partition in the master boot
record in a block device.

### extfs ###

The `extfs` program translates a block device formatted with the second extended
filesystem and mounts it at an empty directory.

### mkinitrd ###

This program produces a Sortix compatible initrd, the file that contains the
initial filesystem used to bootstrap the real root filesystem.

### regress ###

This is a collection of operating system test cases run using the `regress`
driver program.

### carray ###

This program converts a binary file to an C array.

### sh ###

This is the Sortix shell. Currently the `sh` program is a wrapper that looks for
a better non-interactive shell if running a script, otherwise it invokes the
hacky `sortix-sh` program.

### Kernel ###

The Sortix kernel is the core of the Sortix operating system. It provides all
the primitives libc needs to implement a Unix-like environment.

Note that you need to reboot the system to use the new kernel and that you need
a permanent root filesystem or your local changes will be lost.