ncurses to external shell and back messing with keys

Question

I have this ncurses application that is doing the standard recipe for temporarily dropping out of ncurses, running an external editor/shell/whatever, and then dropping back to ncurses when it's done.

This ~almost works, except that the first few keypresses that ncurses gets afterwards are obviously bogus; ncurses thinks ^[ and A are seen respectively if I press the up arrow twice.

Anyone seen this behavior before and know what the magic incant to fix this is? If it helps any, this is the Ruby ncurses library.

Answer 1

After rootling around a bit, I found a cargo culting solution: explicitly call keypad(1) after getting out the shell on stdscr. I have no idea why this works, but it does. I'll mark someone else's answer as yes if they can explain why. The current working theory is that keypad touches some sort of internal buffer and clears it.

Scratch that:

NCURSES_EXPORT(int)
keypad(WINDOW *win, bool flag)
{
    T((T_CALLED("keypad(%p,%d)"), win, flag));

    if (win) {
        win->_use_keypad = flag;
        returnCode(_nc_keypad(SP, flag));
    } else
        returnCode(ERR);
}

Answer 2

Yeah. Without keypad ncurses won't handle escape codes for you. From the keypad man page:

   The keypad option enables the keypad of the user's  terminal.   If  en-
   abled (bf is TRUE), the user can press a function key (such as an arrow
   key) and wgetch returns a single value representing the  function  key,
   as in KEY_LEFT.  If disabled (bf is FALSE), curses does not treat func-
   tion keys specially and the program has to  interpret  the  escape  se-
   quences  itself.   If the keypad in the terminal can be turned on (made
   to transmit) and off (made to work locally),  turning  on  this  option
   causes  the terminal keypad to be turned on when wgetch is called.  The
   default value for keypad is false.

Often, the first thing I do in an ncurses program is call keypad(stdscr, true) to enable nice keyboard mapping.

Hope that helps.