Since kernel.cpp is intended to be an example of the current best coding
practices within the Sortix kernel, and the Maxsi:: API is deprecated and
is being removed, it should rather use the nice C standard library.
This code uses the cr0 task switched bit to disable the FPU upon task
switch, which allows the kernel to delay copying the registers until
another task starts using them. Or better yet, if no other thread actually
uses the registers, then it won't need to do any copying at all!
Pardon the big ass-commit, this took months to develop and debug and the
refactoring got so far that a clean merge became impossible. The good news
is that this commit does quite a bit of cleaning up and generally improves
the kernel quality.
This makes the kernel fully pre-emptive and multithreaded. This was done
by rewriting the interrupt code, the scheduler, introducing new threading
primitives, and rewriting large parts of the kernel. During the past few
commits the kernel has had its device drivers thread secured; this commit
thread secures large parts of the core kernel. There still remains some
parts of the kernel that is _not_ thread secured, but this is not a problem
at this point. Each user-space thread has an associated kernel stack that
it uses when it goes into kernel mode. This stack is by default 8 KiB since
that value works for me and is also used by Linux. Strange things tends to
happen on x86 in case of a stack overflow - there is no ideal way to catch
such a situation right now.
The system call conventions were changed, too. The %edx register is now
used to provide the errno value of the call, instead of the kernel writing
it into a registered global variable. The system call code has also been
updated to better reflect the native calling conventions: not all registers
have to be preserved. This makes system calls faster and simplifies the
assembly. In the kernel, there is no longer the event.h header or the hacky
method of 'resuming system calls' that closely resembles cooperative
multitasking. If a system call wants to block, it should just block.
The signal handling was also improved significantly. At this point, signals
cannot interrupt kernel threads (but can always interrupt user-space threads
if enabled), which introduces some problems with how a SIGINT could
interrupt a blocking read, for instance. This commit introduces and uses a
number of new primitives such as kthread_lock_mutex_signal() that attempts
to get the lock but fails if a signal is pending. In this manner, the kernel
is safer as kernel threads cannot be shut down inconveniently, but in return
for complexity as blocking operations must check they if they should fail.
Process exiting has also been refactored significantly. The _exit(2) system
call sets the exit code and sends SIGKILL to all the threads in the process.
Once all the threads have cleaned themselves up and exited, a worker thread
calls the process's LastPrayer() method that unmaps memory, deletes the
address space, notifies the parent, etc. This provides a very robust way to
terminate processes as even half-constructed processes (during a failing fork
for instance) can be gracefully terminated.
I have introduced a number of kernel threads to help avoid threading problems
and simplify kernel design. For instance, there is now a functional generic
kernel worker thread that any kernel thread can schedule jobs for. Interrupt
handlers run with interrupts off (hence they cannot call kthread_ functions
as it may deadlock the system if another thread holds the lock) therefore
they cannot use the standard kernel worker threads. Instead, they use a
special purpose interrupt worker thread that works much like the generic one
expect that interrupt handlers can safely queue work with interrupts off.
Note that this also means that interrupt handlers cannot allocate memory or
print to the kernel log/screen as such mechanisms uses locks. I'll introduce
a lock free algorithm for such cases later on.
The boot process has also changed. The original kernel init thread in
kernel.cpp creates a new bootstrap thread and becomes the system idle thread.
Note that pid=0 now means the kernel, as there is no longer a system idle
process. The bootstrap thread launches all the kernel worker threads and then
creates a new process and loads /bin/init into it and then creates a thread
in pid=1, which starts the system. The bootstrap thread then quietly waits
for pid=1 to exit after which it shuts down/reboots/panics the system.
In general, the introduction of race conditions and dead locks have forced me
to revise a lot of the design and make sure it was thread secure. Since early
parts of the kernel was quite hacky, I had to refactor such code. So it seems
that the risk of dead locks forces me to write better code.
Note that a real preemptive multithreaded kernel simplifies the construction
of blocking system calls. My hope is that this will trigger a clean up of
the filesystem code that current is almost beyond repair.
Almost all of the kernel was modified during this refactoring. To the extent
possible, these changes have been backported to older non-multithreaded
kernel, but many changes were tightly coupled and went into this commit.
Of interest is the implementation of the kthread_ api based on the design
of pthreads; this library allows easy synchronization mechanisms and
includes C++-style scoped locks. This commit also introduces new worker
threads and tested mechanisms for interrupt handlers to schedule work in a
kernel worker thread.
A lot of code have been rewritten from scratch and has become a lot more
stable and correct.
Share and enjoy!
Unfortunately this area is not standardized by POSIX. Linux uses an ioctl
which is not that bad, but I'd like to have a designated function. I'm not
sure if this facility is powerful enough and whether it should be improved.
Also note that I use a struct winsize as on Linux, but I use size_ts instead
for the heck of it. Perhaps I should use another name for the struct.
The console renderer now renders to a text buffer, which can be implemented
on any device, whether it's the VGA text buffer or a bitmap graphics device
with font rendering. This replaces the older code that could only render to
a VGA framebuffer and where the input parsing was tightly coupled with the
device rendering phase.
This enables useful features such as directories, CRC32 checksums, and other
useful features. The initrdfs in the kernel is now hooked up against the new
API, although the kernel's current limited FS support is a problem for now.
To work around that, directories are not supported at runtime, although the
internal API understands them wonderfully. This will be fixed when the
kernel gets a real VFS.
Any detected COM ports available as /dev/comN.
It currently utilizes AGAIN to do polling in user-space. This prevents it
from locking up the whole system and makes it respond to the SIGINT hack.
There is also a more reliable and faster polling-blocking mode, but it locks
up the entire system.
The main interrupt mode is broken, perhaps by a bug in VirtualBox.
This will allow development of a better terminal providing stdin.
Added new system calls settermmode(2) and gettermmode(2) declared in
<sys/termmode.h>. They allow querying and changing the current mode of
terminals (enabling raw keyboard data, signal handling, line buffering,
UTF-8 encoding stdin, and more). However, all that is unsupported by the
current terminal device driver.
Added KBKEY_ENCODE and KBKEY_DECODE macros to <sys/keycodes.h> which allows
encoding the kbkey format in UTF-32 characters.
Read operations are enabled by default, but you must set DISKWRITE=1 in
makeflags before write operations are permitted. This protects against
accidentally corrupting the existing filesystems on the system.
When compiled with gcc 4.6.1, 32-bit Sortix would triple fault during
early boot: When the TLB is being flushed, somehow a garbage value had
sneaked into Sortix::Memory::currentdir, and a non-page aligned (and
garbage) page directory is loaded. (Triple fault, here we come!)
However, adding a volatile addr_t foo after the currentdir variable
actually caused the system to boot correctly - the garbage was written
into that variable instead. To debug the problem, I set the foo value
to 0: as long as !foo (hence the name nofoo) everything was alright.
After closer examination I found that the initrd open code wrote to a
pointer supplied by kernel.cpp. The element pointed to was on the
stack. Worse, its address was the same as currentdir (now foo).
Indeed, the stack had gone into the kernel's data segment!
Turns out that this gcc configuration stores variables in the data
segment in the reverse order they are defined in, whereas previous
compilers did the opposite. The hack used to set up the stack during
early boot relied on this (now obviously incorrect) fact.
In effect, the stack was initialized to the end of the stack, not
the start of it: completely ignoring all the nice stack space
allocated in kernel.cpp.
I did not see that one coming.
This fixes issues where it did not fit into the first few MiB,
or that GRUB loaded it someplace weird.
The kernel heap is now also protected against growing into the
ramdisk and the kernel stack.
Fixed 64-bit-ness bug in BSR() and BSF().
Added 64-bit system call stubs in libmaxsi.
Added a Elf64 program loader.
Fixed uninitialized memory bug in the scheduler.
x64/boot.s now takes care of user-space memory permissions.
Fixed bug in x64/syscall.s
That's right. The system now boots in 64-bit mode.
It is horribly unstable, though.
This commit got completely out of control.
Added the fork(), getpid(), getppid(), sleep(), usleep() system calls, and
aliases in the Maxsi:: namespace.
Fixed a bug where zero-byte allocation would fail.
Worked on the DescriptorTable class which now works and can fork.
Got rid of some massive print-registers statements and replaced them with
the portable InterruptRegisters::LogRegisters() function.
Removed the SysExecuteOld function and replaced it with Process::Execute().
Rewrote the boot sequence in kernel.cpp such that it now loads the system
idle process 'idle' as PID 0, and the initization process 'init' as PID 1.
Rewrote the SIGINT hack.
Processes now maintain a family-tree structure and keep track of their
threads. PIDs are now allocated using a simple hack. Virtual memory
per-process can now be allocated using a simple hack. Processes can now be
forked. Fixed the Process::Execute function such that it now resets the
stack pointer to where the stack actually is - not just a magic value.
Removed the old and ugly Process::_endcodesection hack.
Rewrote the scheduler into a much cleaner and faster version. Debug code is
now moved to designated functions. The noop kernel-thread has been replaced
by a simple user-space infinite-loop program 'idle'.
The Thread class has been seperated from the Scheduler except in Scheduler-
related code. Thread::{Save,Load}Registers has been improved and has been
moved to $(CPU)/thread.cpp. Threads can now be forked. A new CreateThread
function creates threads properly and portably.
Added a MicrosecondsSinceBoot() function.
Fixed a crucial bug in MemoryManagement::Fork().
Added an 'idle' user-space program that is a noop infinite loop, which is
used by the scheduler when there is nothing to do.
Rewrote the 'init' program such that it now forks off a shell, instead of
becoming the shell.
Added the $$ (current PID) and $PPID (parent PPID) variables to the shell.
1) The PML2 was not initialized to zeroes, thus leaving some bits behind that
caused the fork code to go crazy, forking the unforkable, and mapping addresses
that never, ever, should have been mapped, leaving behind a trail of page faults
and general protection faults on some computers, while other computers worked
because the uninitalized memory just wasn't uninitialized enough. Yep, this was
a schrödinbug!
2) Fixed a time bomb. The kernel heap was accidentally put such that whenever a
few megabytes were allocated, it would begin overwriting the physical page stack
causing unthinkable events to unfold and would probably be even more obscure to
debug than 1).
Oh, and some string errors fixed and removed RunApplication from kernel.cpp,
funny thing that even linked in the first place. Guess, the optimizer actually
did work for once. :)
Jonas 'Sortie' Termansen