WORKING WITH THE SUSE AND UNITED LINUX KERNEL SOURCES

Andreas Gruenbacher <agruen@suse.de>, SuSE Labs, October 2003


This document gives an overview how SuSE and United Linux kernels are
created, and describes how tasks such as building individual kernels
or creating external kernel modules are done.

A companion Update Media HOWTO that describes how to build driver update
disks (among other things) is available at:

    ftp://ftp.suse.com/pub/people/hvogel/Update-Media-HOWTO.


TABLE OF CONTENTS

  Overview
  Compiling your own kernel
  Building external modules
  Patch selection mechanism
  How to configure the kernel sources
  Where to find configuration files
  Pre-configured kernel headers
  Module load paths
  Trigger scripts


OVERVIEW

The kernels for SuSE and United Linux are generated from the vanilla
Linux kernel sources found at http://ftp.kernel.org/, on top of which a
set of additional patches is applied. The resulting kernel source tree
is configured and built, resulting in a binary kernel.

Internally, the add-on patches and configuration files are maintained in
a CVS repository. The repository files are packed up for processing by
RPM. RPM then converts these files into a kernel source tree ready to be
compiled: It unpacks the vanilla kernel sources and applies the patches
that are relevant for a given machine architecture. The following figure
shows these three stages. The files of the second stage (labeled
"kernel-source/") are found in the kernel-source.spm package. Up to stage
two, all architecture share exactly the same files. The result in the
third stage is architecture specific.


		      +------------+
		      | Kernel CVS |
		      +-----+------+
		            |
		            |
		    +----------------+
		    | kernel-source/ |
		    +-------+--------+
		            |
		            |
		+------------------------+
		| kernel-source.i586.rpm |
		+------------------------+


In a second step, the kernel sources are compiled, resulting in a
binary kernel package such as k_deflt.$ARCH.rpm. In addition to the
kernel-source.$ARCH.rpm package, additonal packages contain the source
code of external kernel modules. These packages all have a name prefix
of km_ (for example, km_pcmcia). The km_packages usually originate from
base packages with the same name, but without the km_ prefix.


		      +---------+
		      | pcmcia/ |
		      +----+----+
		           |
		           |
		+---------------------+
		|  km_pcmcia.i586.rpm |
		+---------------------+


The k_* binary kernel packages configure and compile the kernel sources
from the kernel-source.$ARCH.rpm as well as from the km_*.$ARCH.rpm
packages. Different configurations for the various kernel variants (or
"flavors") are contained in the CVS repository and the packed-up
sources. The architecture specific kernel source trees only contain the
configurations files relevant for their architetures.


	+------------------------+   +---------------+
	| kernel-source.i586.rpm |   | km_*.i586.rpm |
	+-----------+------------+   +-------+-------+
	            |                        |
	            +----------+      +------+
	                       |      |
	                     +----------+
	                     | k_deflt/ |
	                     +----+-----+
	                          |
	                          |
	                 +------------------+
	                 | k_deflt.i586.rpm |
	                 +------------------+


On the file system, the kernel source tree is installed into the
directory /usr/src/linux-$VERSION-$RELEASE (for example,
/usr/src/linux-2.4.21-99). The symbolic link /usr/src/linux points to
that directory. The km_* packages install their module sources to
directories below /usr/src/kernel-modules. The pre-configured kernel
headers for the different configurations are installed below
/usr/src/linux-$VERSION-$RELEASE-include; the symbolic link
/usr/src/linux-include points to that directory.

Putting external module sources into km_* packages and building them
together with the rest of the kernel sources allows us to include the
binary modules into the binary kernel packages, without having to split
the module sources from the module utilities. (This is the usual way how
modules and their associated utilities are distributed.)


COMPILING YOUR OWN KERNEL

As described above, the kernel sources are found in the
kernel-source.$ARCH.rpm and km_*.$ARCH.rpm packages. It is possible to
build a binary kernel from the kernel-source package alone. The
following steps are needed:

  (1) Install kernel-source.$ARCH.rpm. Change to the /usr/src/linux
      directory.
  (2) Configure the kernel (for example, `make oldconfig' or `make
      cloneconfig, see HOW TO CONFIGURE THE KERNEL SOURCES).
  (3) Run `make dep' to compute dependencies and version information.
  (4) Build the static kernel image (for example, `make bzImage').
  (5) Build the modules found in the kernel source tree by running
      `make modules'.
  (6) Install the kernel and the modules (for example, `make install'
      and `make modules_install').
  (7) If necessary, run mkinitrd for the new kernel (see `mkinitrd -h').
  (8) Add the kernel to the boot manager. If using lilo, run `lilo'
      to update the boot map.

Step (3) differs slightly among different architectures. On Intel x86
and AMD x86_64 systems, the static part of the kernel is built with
`make bzImage'. On ia64, `make compressed' is used. The ppc and ppc64
architectures use `make zImage'. The s390 and s390x architectures use
`make image'. The axp architecture (Alpha) uses `make boot'.

(For the User-Mode-Linux kernel, "ARCH=um" must be appended to
the make command line; the kernel binary is called "linux". The UML
source tree package kernel-source-um.i586.rpm can be generated from
kernel-source.src.rpm; the source tree will be placed under
/usr/src/linux-um.)

For building kernel modules from one or more of the km_* packages,
install the km_* packages. Each km_* package installes the module
source files into a directory below /usr/src/kernel-modules.

Some of the km_* packages look for the kernel sources in /usr/src/linux,
while others follow the symbolic link /lib/modules/$(uname -r)/build.
For consistent results, boot into the kernel for which you want to build
modules, and then make distclean, cloneconfig, and dep in
/usr/src/linux. Then, do the following for each such directory:

  (1) Change into /usr/src/kernel-modules/$DIR
  (2) Build the module(s) by typing `make -f Makefile.module'.
  (3) Install the module(s) by typing `make -f Makefile.module
      install'.

Alternatively to building binary kernels by hand, you can also build
one of the k_* packages using RPM.


BUILDING EXTERNAL MODULES

The kernel modules found in the kernel-source and km_* packages all
become part of the k_* binary kernel packages. It is also possible to
build external modules (that is, modules that are not part of a k_*
package).

Before building such modules, the kernel-source.$ARCH.rpm package must
be installed. This package contains the kernel source code, and
pre-configured kernel headers for the different configurations (or
"flavors") that exist for that specific architecture. External kernel
modules must be built for a specific kernel configuration; modules with
different configurations are not compatible with each other in general.
For building the module, the path to the configured kernel sources must
be specified. This path can be /usr/src/linux. Alternatively, the path
to one of the pre-configured kernel header trees under
/usr/src/linux-include can be specified. These directory trees are
configured for the different SuSE and UnitedLinux configurations
("flavors").

Building with the pre-configured headers is preferred over building with
the kernel sources in /usr/src/linux: This allow to build modules for
more than one configuration without having to reconfigure the kernel
source tree.

NOTE: Pre-configured kernel headers are a very recent addition. You will
      need a 2.4.21 kernel-source package with a release number >= 106.
      The kernel-source package up to and including UnitedLinux Service
      Pack 3 Release Candidate 1 contains a bug. The fix is described
      below; see PRE-CONFIGURED KERNEL HEADERS.

The recommended way of building external kernel modules is to use the
kernel build infrastructure as far as that is possible. The essential
step is to build the actual module as follows (KERNEL_SOURCE being the
directory where the kernel headers lie):

    make -C $KERNEL_SOURCE modules SUBDIRS=`pwd`

Please take a look at the demo modules (installed under
/usr/share/doc/packages/kernel-source) to see how this step can be
embedded in makefiles. The hello.vanilla example can be used against a
vanilla kernel source tree. The hello.suse source tree depends on
Rules.modules. This file is only found in very recent SuSE and
UnitedLinux kernel sources. Please use the approach in hello.suse if
possible; it avoids duplicating information unnecessarily.

BUG NOTE:
	Up to and including UnitedLinux Service Pack 3 Release Candidate
	1, the kernel-source package does not include the Rules.modules
	file used in hello.suse. Please copy the Rules.modules file from
	this tarball to /usr/src/linux/ and to
	/usr/src/linux-include/*/.


PATCH SELECTION MECHANISM

The SuSE / United Linux kernels consist of the vanilla kernel sources,
on top of which a number of patches is applied. While most of these
patches are applied on all architectures, some are only relevant on a
subset of architectures. The file series.conf determines which patches
are applied on which architecture. A script named "guards" converts
series.conf into a plain list of patches to be applied. Guards decides
which patches to include and exclude based on a list of symbols. This
list is produced by the helper script "arch-symbols". A fully patched
kernel source tree can be generated from vanilla sources + patches like
this:

    $ tar xvfz linux-2.4.21.tar.gz
    $ for p in $(./guards $(./arch-symbols) < series.conf) ; do
	patch -d linux-2.4.21 -p1 < $p || break
      done

The configuration script config.conf, which is similar to series.conf
and also processed by guard, is used for configuration file selection.
See Section WHERE TO FIND CONFIGURATION FILES.

The configuration file formats should be obvious from the comments in
series.conf and from the guards(1) manual page. (The guards(1) manual
page can be generated by running pod2man on the guards script.)


HOW TO CONFIGURE THE KERNEL SOURCES

Before a binary kernel can be built, or an additional loadable module
for an existing kernel can be created, the kernel must be configured.
A configuration consists of several thousand settings. In order for a
loadable module to work with an existing kernel, it must be created with
a configuration that is identical (or at least very close) to the
kernel's configuration. Each configuration is contained in a single
file. (The next Section describes where the standard SuSE and
UnitedLinux kernel configurations can be found. It will seldom be
necessary to use a different configuration.)

Configuring the kernel sources for a specific configuration is
straightfoward:

  - Locate the configuration file you want to use. (See WHERE TO FIND
    CONFIGURATION FILES).

  - Copy the configuration to the file .config in the kernel source
    tree. The kernel-source package installs its source tree in
    /usr/src/linux.

  - Run the following commands in sequence to apply the configuration,
    generate version information files, etc.:

	make distclean
	make oldconfig
	make dep

    Alternatively to `make oldconfig', you can also use `make
    menuconfig' for a text menu oriented user interface. If the kernel
    sources do not match the configuration file exactly, `make
    oldconfig' will prompt for settings that are undefined.

    For configuring the kernel to match the running kernel, there is a
    shortcut `make cloneconfig' that expands the file /proc/config.gz
    into .config, and then runs `make oldconfig'.

In previous versions, the kernel sources contained in the
kernel-source.$ARCH.rpm package were pre-configured with a default
configuration. Since SuSE Linux 9.0, the kernel sources in
/usr/src/linux-$VER_REL are not configured anymore.

External kernel modules against one of the SuSE and UnitedLinux
configurations can also be built using the pre-configured kernel headers
found in /usr/src/linux-include (which is just a symbolic link to
/usr/src/linux-$VER-include). This is much easier if you want to build
modules for more then one kernel configuration at once. See
PRE-CONFIGURED KERNEL HEADERS.


WHERE TO FIND CONFIGURATION FILES

Kernel configuration files are stored in the kernel CVS repository. When
packing up the repository, they end up in kernel-source/config.tar.bz.
Then, when kernel-source.$ARCH.rpm is built, the config files that are
applicable for the given architecture are selected. Each selected file
config/$ARCH/$FLAVOR is copied to arch/$ARCH/defconfig.$FLAVOR in the
kernel source tree (for eaxmple, arch/i386/defconfig.default).

Each k_$FLAVOR package is based on arch/$ARCH/defconfig.$FLAVOR (k_deflt
is based on arch/$ARCH/defconfig.default). The k_* packages install
their configuration files as /boot/config-$VER_STR (for example,
boot/config-2.4.21-99-default).  (Previous k_* packages did store
their configuration file as /boot/vmlinuz.config.)

In addition, the running kernel exposes a gzip compressed version of its
configuration file as /proc/config.gz.


PRE-CONFIGURED KERNEL HEADERS

All SuSE and UnitedLinux kernels for an architecture are built from the
same kernel source tree. (Architectures include different patches, so
the kernel source trees among architectures differ slightly.) To
optimize for specific processors, multiple binary kernels are generated
from these sources using different kernel configurations, resulting in
the various k_* packages (for example, k_deflt, k_smp, k_athlon). As
described above, the configuration files for the different
configurations (or "flavors") are found in arch/$ARCH/defconfig.$FLAVOR.
Using these configuration files, these configurations can be reproduced
(see HOW TO CONFIGURE THE KERNEL SOURCES).

Because reconfiguring the kernel sources is time consuming, the
kernel-source package also contains pre-configured kernel headers
(including the kernel build environment) that can be used right away for
building modules for this configuration.  These files are found under
/usr/src/linux-include/$FLAVOR.

BUG NOTE:
	The pre-configured headers shipped with SuSE Linux 9.0 contain
	several minor bugs that could not be fixed in time. For using
	pre-configured headers, please grab a more recent kernel-source
	package from ftp://ftp.suse.com/pub/people/mantel/next/RPM/.

BUG NOTE:
	Up to and including UnitedLinux Service Pack 3 Release Candidate
	1, the pre-configured kernel headers contain a bug. When building
	modules against the pre-configured headers, this result in the
	following error message:

	  touch: cannot touch `include/config/MARKER': Permission denied

	As a workaround, please change into /usr/src/linux-include, and
	update the timestamps of these marker files with:

	  touch */include/config/MARKER


MODULE LOAD PATHS

Previous SuSE and UnitedLinux kernels were keeping module files in
/lib/modules/2.4.21-4GB, /lib/modules/2.4.21-4GB-SMP and similar. The
kernel release number was not part of the kernel release name. Because
of that, it was not possible to install multiple binary kernels on a
system in parallel in most situations. This has been changed; the
release number and the configuration name are now part of the kernel
release name. Modules that belong to a specific kernel release are
installed in /lib/modules/2.4.21-99-smp and similar.

This would be a problem for external kernel modules; with each minor
kernel update the module path would change, and the external modules
would get lost. Therefore, a path that spans multiple kernel releases
was added (in or example, /lib/modules/2.4.21-override-smp). The module
utilities are searching for modules in this path before the more
specific path. Modules in the *-override-* directory take precedence
over modules in the *-$RELEASE-* directory.

NOTE: Modules in the override path are not removed automatically. This
      means that broken or outdated modules in that path will fail to
      load, or may even cause the kernel to crash, even if the more
      specific module path contains a correct module. It is the
      administrator's responsibility to remove obsolete modules from the
      override path.

NOTE: Starting with SP3 the SuSE and UnitedLinux kernels are using
      symbol version information (CONFIG_MODVERSIONS). This allows to
      load modules built for different kernel versions or releases as
      long as the interface betwen the kernel and the module does not
      change. The CONFIG_MODVERSIONS mechanism is only able to detect
      some mis-uses of modules though; unless a module has been built
      for precisely the same kernel and comfiguration, modules can
      still fail.


TRIGGER SCRIPTS

Trigger scripts are a mechanism for triggering actions on installation,
update or removal of k_* packages. This mechanism is currently used by
the NVidia kernel module, which is linked at kernel install time.

When a k_* package is installed, all executable files in
/lib/modules/scripts are invoked from the %pre, %preun, %post, and
%postun RPM scripts. These scripts are passed the following parameters:

  $1 = "--pre", "--post", "--preun", or "--postun", the calling RPM
       script.
  $2 = The kernel release name (for example, 2.4.21-99-smp).
  $3 = The number of instances of the k_* package that are currently
       installed.

The RPM scripts are called in the following order:

  - If a package is being installed or updated:

    (1) pre-install script of the package being installed.
        $3 == 1  -->  installation
	$3 == 2  -->  update

    (2) post-install script of the package being installed
        $3 == 1  -->  installation
	$3 == 2  -->  update

  - If a package is being updated or uninstalled:

    (3) pre-uninstall script of the package being removed
	$3 == 0  -->  uninstall
        $3 == 1  -->  update

    (4) post-uninstall script of the package being removed
	$3 == 0  -->  uninstall
        $3 == 1  -->  update


REFERENCES

General

  Linux Documentation Project, http://www.tldp.org/

  Linux Weekly News, http://lwn.net

  Rusty's Remarkably Unreliable Guides (Kernel Hacking
    and Kernel Locking guides),
    http://www.netfilter.org/unreliable-guides/

  Kernel newbies, http://www.kernelnewbies.org/


Loadable Kernel Modules

  Peter Jay Salzman and Ori Pomerantz: Linux Kernel Module
    Programming Guide, Version 2.4, April 2003,
    http://www.tldp.org/guides.html

  Linux Weekly News: Driver porting: compiling external modules,
    http://lwn.net/Articles/21823/

