How to diagnose/trace "sendsig: useracc failed." problem in HP-UX

Question

I am trying to compile Ruby 1.9.1-p0 on HP-UX. After a small change to ext/pty.c it compiles successfully, albeit with a lot of warning messages (about 5K). When I run the self-tests using "make test" it crashes and core-dumps with the following error:

sendsig: useracc failed. 0x9fffffffbf7dae00 0x00000000005000

Pid 3044 was killed due to failure in writing the signal context - possible stack overflow.

Illegal instruction

From googling this problem the Illegal instruction is just a signal that the system uses to kill the process, and not related to the problem. It would seem that there is a problem with the re-establishing the context when calling the signal handler. Bringing the core up in gdb doesn't show a particularly deep stack, so I don't think the "possible stack overflow" is right either.

The gdb stack backtrace output looks like this:

#0  0xc00000000033a990:0 in __ksleep+0x30 () from /usr/lib/hpux64/libc.so.1
#1  0xc0000000001280a0:0 in __mxn_sleep+0xae0 ()
    from /usr/lib/hpux64/libpthread.so.1
#2  0xc0000000000c0f90:0 in <unknown_procedure> + 0xc50 ()
    from /usr/lib/hpux64/libpthread.so.1
#3  0xc0000000000c1e30:0 in pthread_cond_timedwait+0x1d0 ()
    from /usr/lib/hpux64/libpthread.so.1

Answer 1

Answering my own question:

The problem was that the stack being allocated was too small. So it really was a stack overflow. The sendsig() function was preparing a context structure to be copied from kernel space to user space. The useracc() function checks that there's enough space at the address specified to do so.

The Ruby 1.9.1-p0 code was using PTHREAD_STACK_MIN to allocate the stack for any threads created. According to HP-UX documentation, on Itanium this is 256KB, but when I checked the header files, it was only 4KB. The error message from useracc() indicated that it was trying to copy 20KB.

So if a thread received a signal, it wouldn't have enough space to receive the signal context on its stack.