The minimum number of system calls any reasonable *nix style OS should have are (IMHO):
- open
- close
- read
- write
- fork
- exec
- waitpid
the first 4 allow you to both provide input to a program and get its output. (Remember on *nix like operating systems stdout is just another file handle as far as the OS is concerned).
The other 3 are the bare minimum needed to start another program and wait for its result. However, it is certain that SanOS already has these since it is already a very functional operating system.
It is entirely possible that the additions you need to make won't need to be done at a kernel level.
EDIT:
As far as what is needed to support a newer JVM, this paragraph from the SanOS site gives a great hint:
You can run the Windows version of Sun
HotSpot JVM under sanos. This is
possible because sanos supports the
standard PE executable format (.EXE
and .DLL files). Wrappers are provided
for the Win32 DLLs like kernel32.dll,
user32.dll, wsock32.dll, etc., as well
as the C runtime library msvcrt.dll. I
have tested sanos with the following
JVMs:
Basically, the JVMs are the standard windows exe files. So you would just need to find out which system calls the referenced dlls make and ensure that they exist and are implemented correctly.