459 lines
14 KiB
Groff
459 lines
14 KiB
Groff
.Dd December 29, 2015
|
|
.Dt DEVELOPMENT 7
|
|
.Os
|
|
.Sh NAME
|
|
.Nm development
|
|
.Nd operating system development instructions
|
|
.Sh SYNOPSIS
|
|
.Pa /src
|
|
.Sh DESCRIPTION
|
|
Releases come with the system source code in
|
|
.Pa /src
|
|
as a
|
|
.Xr git 1
|
|
repository.
|
|
It can be modified, compiled and installed on the current system.
|
|
The source code is built with a
|
|
.Xr make 1
|
|
build system.
|
|
The source code can be located in any location, if so, simply substitute
|
|
.Pa /src
|
|
with the real location.
|
|
These instructions only apply to building the operating system from within
|
|
itself, those building it from another operating system needs to follow
|
|
.Xr cross-development 7
|
|
instead.
|
|
.Pp
|
|
If you are building a new version of the operation system where build tools have
|
|
been added or changed, you first need to install the new tools.
|
|
This is not needed when building the matching release.
|
|
To do so, run as root:
|
|
.Bd -literal
|
|
cd /src
|
|
make distclean # fully clean build directory
|
|
make install-build-tools # install new tools
|
|
make clean-build-tools # clean for real build below
|
|
.Ed
|
|
.Pp
|
|
To build the base operating system and upgrade the current
|
|
system, run as root:
|
|
.Bd -literal
|
|
cd /src
|
|
make # build new operating system in /src/sysroot
|
|
make sysmerge # upgrade current operating system with /src/sysroot
|
|
.Ed
|
|
.Pp
|
|
The build system creates a minimal root filesystem structure in the
|
|
.Pa /src/sysroot
|
|
and builds each operating system component in turn, installing them into the
|
|
sysroot.
|
|
The ports in the
|
|
.Pa /src/ports
|
|
are built automatically by downloading the upstream releases and applying the
|
|
appropriate patches.
|
|
The result is a system image that can be turned into working system by adding
|
|
important configuration such as
|
|
.Xr passwd 5 .
|
|
.Pp
|
|
The
|
|
.Sy sysmerge
|
|
make target ensures a system is built in
|
|
.Pa /src/sysroot
|
|
and then runs the
|
|
.Xr sysmerge 8
|
|
program which installs the new system files onto the existing system.
|
|
It updates the system manifest as well all ports installed in the sysroot.
|
|
The
|
|
.Xr initrd 7
|
|
is automatically regenerated using
|
|
.Xr update-initrd 8 .
|
|
The bootloader, if enabled in
|
|
.Xr upgrade.conf 5 ,
|
|
is reinstalled and configured as necessary.
|
|
The new user-space is running on completion, though existing processes will be
|
|
running the old programs.
|
|
A reboot is needed to run the new kernel.
|
|
If the ABI changed and the current kernel isn't able to run the new programs,
|
|
then the upgrade is delayed and will be automatically completed on the next
|
|
boot.
|
|
The
|
|
.Sy sysmerge-wait
|
|
make target forces waiting until the next boot.
|
|
.Ss Root Makefile
|
|
The
|
|
.Pa /src/Makefile
|
|
handles the high level build of the operating system.
|
|
The important targets are:
|
|
.Bl -tag -width "12345678"
|
|
.It Sy all
|
|
(default) Build each component in turn and install them into the sysroot.
|
|
.It Sy available-ports
|
|
Search for newer available versions of ports.
|
|
.It Sy build-tools
|
|
Make all build tools.
|
|
.It Sy clean
|
|
Clean the component directories and the port source code.
|
|
.Sy ( clean-core , clean-ports )
|
|
.It Sy clean-build-tools
|
|
Clean the directories of all build tools.
|
|
.It Sy clean-cross-compiler
|
|
Clean the directories for the cross-compiler.
|
|
.It Sy clean-cross-toolchain
|
|
Clean the directories for the build tools and cross-compiler.
|
|
.Sy ( clean-build-tools , clean-cross-compiler )
|
|
.It Sy distclean
|
|
Run every clean target such that the source code is ready for distribution.
|
|
.Sy ( clean-builds , clean-core , clean-mirror , clean-release , clean-repository , clean-sysroot , distclean-ports, clean-cross-compiler )
|
|
.It Sy distclean-ports
|
|
Remove the port source code extractions.
|
|
.It Sy extract-ports
|
|
Extract the upstream release for each
|
|
.Xr port 5
|
|
in
|
|
.Pa /src/ports
|
|
and apply the appropriate patches.
|
|
.It Sy install-build-tools
|
|
Install all build tools after making them.
|
|
.It Sy install-cross-compiler
|
|
Install the cross-compiler after making it.
|
|
.It Sy install-cross-toolchain
|
|
Install the build tools and cross-compiler after making them.
|
|
.Sy ( install-build-tools , install-cross-compiler )
|
|
.It Sy iso
|
|
Create a release iso in the
|
|
.Pa /src/builds
|
|
directory after making
|
|
.Sy all .
|
|
.It Sy mirror
|
|
Download the upstream release for each
|
|
.Xr port 5
|
|
in
|
|
.Pa /src/ports
|
|
from
|
|
.Ev SORTIX_PORTS_MIRROR
|
|
into the
|
|
.Pa /src/mirror
|
|
directory.
|
|
The operating system can be built without network connectivity once the mirror
|
|
is populated.
|
|
The local mirror has the same structure as the remote mirror and can be used as
|
|
a remote mirror.
|
|
.It Sy mostlyclean
|
|
Clean everything except binary packages and the mirror of upstream releases.
|
|
.Sy ( clean-builds , clean-ports , clean-release , clean-sysroot , distclean-ports, clean-cross-compiler )
|
|
.It Sy presubmit
|
|
Verify the port configuration
|
|
.Sy ( verify-ports ) ,
|
|
the coding style is followed
|
|
.Sy ( verify-coding-style ) ,
|
|
the manual pages does not have lints
|
|
.Sy ( verify-manual ) ,
|
|
the build tools compile
|
|
.Sy ( verify-build-tools ) ,
|
|
that everything compiles without warnings on all architectures
|
|
.Sy ( verify-build ) ,
|
|
and the system headers works in all supported configurations
|
|
.Sy ( verify-headers ) .
|
|
.It Sy release
|
|
Make
|
|
.Sy iso
|
|
and construct release directory tree in
|
|
.Pa /src/release
|
|
suitable for online publishing.
|
|
.It Sy sortix.iso
|
|
Make
|
|
.Sy iso
|
|
and place it in the current directory as
|
|
.Pa sortix.iso .
|
|
.It Sy sysmerge
|
|
Upgrade the current operating system using the sysroot after making the
|
|
.Sy all
|
|
target.
|
|
.It Sy sysmerge-full
|
|
Like
|
|
.Sy sysmerge
|
|
but do a full operating system upgrade that uninstalls ports not present in the
|
|
sysroot using
|
|
.Fl \-full .
|
|
.It Sy sysmerge-full-wait
|
|
The combination of
|
|
.Sy sysmerge-full
|
|
and
|
|
.Sy sysmerge-full-wait .
|
|
.It Sy sysmerge-wait
|
|
Like
|
|
.Sy sysmerge
|
|
but delay the upgrade until the next boot using
|
|
.Fl \-wait .
|
|
.It Sy sysroot-base-headers
|
|
Create the sysroot and install only the headers of the standard library and
|
|
kernel into it.
|
|
This is useful when bootstrapping the runtime libraries of the compiler that
|
|
need to know about libc prior to building libc.
|
|
.It Sy upgrade-ports
|
|
Search for newer available versions of ports and update the
|
|
.Sy VERSION
|
|
variable in the
|
|
.Xr port 5
|
|
and switch it into development mode.
|
|
.El
|
|
.Pp
|
|
The important environment variables influencing the Makefile are:
|
|
.Bl -tag -width "12345678"
|
|
.It Ev BUILD
|
|
The platform of the current operating system.
|
|
This defaults to the current machine and operating system.
|
|
.It Ev HOST
|
|
Specifies platform on which the compiled code will run.
|
|
This defaults to the current machine and operating system.
|
|
This is used when cross-compiling the operating system.
|
|
When cross-compiling the operating system, it must be set to one of
|
|
.Sy i686-sortix
|
|
and
|
|
.Sy x86_64-sortix .
|
|
This must be unset when building the build tools as they run on the current
|
|
operating system.
|
|
The compiler tools are prefixed with this variable if it does not match
|
|
.Ev BUILD.
|
|
.It Ev OPTLEVEL
|
|
Specifies compiler optimization options that gets added to
|
|
.Ev CFLAGS
|
|
and
|
|
.Ev CXXFLAGS .
|
|
.It Ev PACKAGES
|
|
Specifies which ports to build, or all ports if unset.
|
|
Suffixing a port with a
|
|
.Sq "!"
|
|
includes its mandatory dependencies, and suffixing it with
|
|
.Sq !!
|
|
includes the optional dependencies as well.
|
|
The sets of ports defined in
|
|
.Pa /src/build-aux/ports.conf
|
|
can be specified as well.
|
|
For instance
|
|
.Sy PACKAGES=minimal!
|
|
builds all the minimal ports with no optional dependencies.
|
|
.It Ev SORTIX_INCLUDE_SOURCE
|
|
Specifies whether the source code is included in the sysroot.
|
|
This must be one of
|
|
.Sy no , yes
|
|
or
|
|
.Sy git
|
|
and defaults to
|
|
.Sy git
|
|
if
|
|
.Xr git 1
|
|
is installed and
|
|
.Sy yes
|
|
otherwise.
|
|
.It Ev SORTIX_ISO_COMPRESSION
|
|
Specifies the compression algorithm used in iso files.
|
|
This must be one of
|
|
.Sy none , gzip
|
|
or
|
|
.Sy xz
|
|
and defaults to
|
|
.Sy xz .
|
|
.It Ev SORTIX_PORTS_MIRROR
|
|
Upstream releases of ports are downloaded from this mirror, defaulting to the
|
|
official mirror, and falling back to the upstream release site if the mirror
|
|
failed.
|
|
.El
|
|
.Ss Components
|
|
The operating systems components, such as libc and the kernel, each have their
|
|
own directory by that name.
|
|
It contains a makefile that can build and install that component.
|
|
This allows building and installing only that component onto the current
|
|
operating system.
|
|
.Pp
|
|
For instance, to build and install libc, run as root:
|
|
.Bd -literal
|
|
cd /src/libc
|
|
make
|
|
make install
|
|
.Ed
|
|
.Pp
|
|
Note the individual makefiles only install the new system files and leak any
|
|
files that don't exist anymore; and they also don't run any upgrade hooks to
|
|
migrate the current system.
|
|
This mechanism isn't supported unless you are building the same source code as
|
|
the current operating system.
|
|
The global
|
|
.Sy sysmerge
|
|
makefile targets should be used instead as the supported mechanism for operating
|
|
system upgrades.
|
|
.Pp
|
|
System libraries are statically linked and you will have to relink programs with
|
|
the new library for changes to take effect.
|
|
Building the whole operating system from the root makefile ensures components
|
|
are built in the right order such that all programs use fresh libraries.
|
|
The root makefile invokes component makefiles with
|
|
.Ev SYSROOT
|
|
set to
|
|
.Pa /src/sysroot
|
|
to force the compiler to locate files there.
|
|
Likewise when installing, it sets
|
|
.Ev DESTDIR
|
|
to
|
|
.Pa /src/sysroot
|
|
to make it install files there.
|
|
.Ss Directories
|
|
In addition to the directories for each operating system component, there are
|
|
these special directories:
|
|
.Bl -tag -width "12345678"
|
|
.It Pa /src/builds
|
|
The build artifacts produced when building the operating system.
|
|
.It Pa /src/mirror
|
|
The upstream releases of ports are downloaded on the first use and cached inside
|
|
this mirror directory.
|
|
.It Pa /src/ports
|
|
Each subdirectory contains a
|
|
.Xr port 5
|
|
which is automatically built along with the operating system per the
|
|
.Ev PACKAGES
|
|
environment variable.
|
|
.It Pa /src/release
|
|
The
|
|
.Sy release
|
|
root makefile target creates this directory and populates it with a directory
|
|
structure suitable for online publishing of a release.
|
|
.It Pa /src/repository
|
|
This directory stores the binary packages and is created when they are built.
|
|
This works as a cache so ports don't have to be rebuilt every time the operating
|
|
system is.
|
|
Packages are also copied from here rather than the sysroot when making releases.
|
|
.It Pa /src/sysroot
|
|
This directory is made when building the operating system and the freshly made
|
|
files are installed here.
|
|
The build system uses this as the system root which forces the compiler to look
|
|
here for headers and libraries.
|
|
This ensures a clean bootstrap where files from the current operating system do
|
|
not leak into the new system.
|
|
.It Pa /src/sysroot-overlay
|
|
If this directory exists, it is added to the initrd of the produced iso and can
|
|
contain additional system files.
|
|
.El
|
|
.Ss Build Tools
|
|
Some components are used to build the source code and must match the versions in
|
|
the source code being built.
|
|
These are currently:
|
|
.Pp
|
|
.Bl -bullet -compact
|
|
.It
|
|
carray
|
|
.It
|
|
kblayout-compiler
|
|
.It
|
|
mkinitrd
|
|
.It
|
|
sf
|
|
.It
|
|
tix
|
|
.El
|
|
.Pp
|
|
If the currently installed versions of those tools are older than the ones in
|
|
the source code, you must update them.
|
|
The
|
|
.Sy clean-build-tools
|
|
root makefile target cleans the applicable directories, the
|
|
.Sy build-tools
|
|
root makefile target builds them from the source code, and the
|
|
.Sy install-build-tools
|
|
root makefile target installs the new version.
|
|
You must clean the compiled files from the source code afterwards because the
|
|
compiled tools are intended to run on the current system, and have not been
|
|
built properly using
|
|
.Pa /src/sysroot .
|
|
.Ss Ports
|
|
Each
|
|
.Xr port 5
|
|
in the
|
|
.Pa /src/ports
|
|
directory will get built automatically when and installed into the sysroot when
|
|
building the whole operating system.
|
|
Installable binary packages are created in the
|
|
.Pa /src/repository/$HOST
|
|
directory using
|
|
.Xr tix-port 8
|
|
and can be installed with
|
|
.Xr tix-install 8 .
|
|
If an existing binary package with the right version exists in the repository,
|
|
it is used instead of the building the port again.
|
|
.Pp
|
|
The ports system workflow is described in
|
|
.Xr porting 7 .
|
|
.Ss Patches
|
|
The source code is managed as a
|
|
.Xr git 1
|
|
repository and you can make your own changes and commit them.
|
|
A good approach is to set up your own local development branch and work there:
|
|
.Bd -literal
|
|
git checkout -b local
|
|
git add utils/hello.c
|
|
git commit -m 'Add hello(1).'
|
|
.Ed
|
|
.Pp
|
|
The
|
|
.Sy presubmit
|
|
makefile target can be used to verify your work needs some of the development
|
|
conventions.
|
|
.Pp
|
|
Prepare a set of patches suitable for upstream submission and submit a merge
|
|
request to the upstream project.
|
|
.Pp
|
|
If your installation does not have network connectivity, you will need to
|
|
submit the changes from another system.
|
|
If you are dual booting and have another operating system with network
|
|
connectivity, you can boot into the other operating system and mount the
|
|
appropriate filesystem from there.
|
|
If you have a serial line, you can produce a set of .patch files containing
|
|
your changes with
|
|
.Bd -literal
|
|
git format-patch master..local
|
|
.Ed
|
|
.Pp
|
|
and then transfer them over the serial connection as described in
|
|
.Xr serial-transfer 7 .
|
|
.Ss Releases
|
|
CD-ROM release of the operating system can be built with the
|
|
.Sy iso
|
|
root makefile target.
|
|
This will build the whole operating system, if not done already, and produce a
|
|
bootable iso for the current architecture in the
|
|
.Pa /src/builds
|
|
directory.
|
|
The
|
|
.Sy sortix.iso
|
|
root makefile target will do the above and place a
|
|
.Pa sortix.iso
|
|
file in the current directory.
|
|
.Pp
|
|
The
|
|
.Sy release
|
|
root makefile target will run the
|
|
.Sy iso
|
|
target and prepare a
|
|
.Pa /src/release
|
|
directory with a directory structure and miscellaneous files suitable for a
|
|
formal online release.
|
|
.Ss Following Development
|
|
The
|
|
.Xr following-development 7
|
|
manual page documents what needs to be done to stay updated with the latest
|
|
developments.
|
|
You will need to read the new version of that document whenever you update the
|
|
source code.
|
|
.Sh SEE ALSO
|
|
.Xr git 1 ,
|
|
.Xr make 1 ,
|
|
.Xr port 5 ,
|
|
.Xr cross-development 7 ,
|
|
.Xr following-development 7 ,
|
|
.Xr installation 7 ,
|
|
.Xr porting 7 ,
|
|
.Xr serial-transfer 7 ,
|
|
.Xr upgrade 7 ,
|
|
.Xr sysinstall 8 ,
|
|
.Xr sysmerge 8 ,
|
|
.Xr update-initrd 8
|