From the Jargon file, always a good read for IT industry history buffs and those who want to know what we're talking about:
=====
Back in the mid-1970s, several of the system support staff at Motorola
discovered a relatively simple way to crack system security on the
Xerox CP-V timesharing system. Through a simple programming strategy,
it was possible for a user program to trick the system into running a
portion of the program in 'master mode' (supervisor state), in which
memory protection does not apply. The program could then poke a large
value into its 'privilege level' byte (normally write-protected) and
could then proceed to bypass all levels of security within the
file-management system, patch the system monitor, and do numerous
other interesting things. In short, the barn door was wide open.
Motorola quite properly reported this problem to XEROX via an official
'level 1 SIDR' (a bug report with a perceived urgency of 'needs to be
fixed yesterday'). Because the text of each SIDR was entered into a
database that could be viewed by quite a number of people, Motorola
followed the approved procedure: they simply reported the problem as
'Security SIDR', and attached all of the necessary documentation,
ways-to-reproduce, etc. separately.
Xerox sat on their thumbs...they either didn't realize the severity of
the problem, or didn't assign the necessary operating-system-staff
resources to develop and distribute an official patch.
Months passed. The Motorola guys pestered their Xerox field-support
rep, to no avail. Finally they decided to take Direct Action, to
demonstrate to Xerox management just how easily the system could be
cracked, and just how thoroughly the system security systems could be
subverted.
They dug around in the operating-system listings, and devised a
thoroughly devilish set of patches. These patches were then
incorporated into a pair of programs called Robin Hood and Friar Tuck.
Robin Hood and Friar Tuck were designed to run as 'ghost jobs'
(daemons, in Unix terminology); they would use the existing loophole
to subvert system security, install the necessary patches, and then
keep an eye on one another's statuses in order to keep the system
operator (in effect, the superuser) from aborting them.
So...one day, the system operator on the main CP-V software
development system in El Segundo was surprised by a number of unusual
phenomena. These included the following:
- Tape drives would rewind and dismount their tapes in the middle of a
job.
- Disk drives would seek back and forth so rapidly that they'd attempt
to walk across the floor.
- The card-punch output device would occasionally start up of itself
and punch a (every hole punched). These would usually
jam in the punch.
- The console would print snide and insulting messages from Robin Hood
to Friar Tuck, or vice versa.
- The Xerox card reader had two output stackers; it could be
instructed to stack into A, stack into B, or stack into A unless a
card was unreadable, in which case the bad card was placed into
stacker B. One of the patches installed by the ghosts added some
code to the card-reader driver... after reading a card, it would flip
over to the opposite stacker. As a result, card decks would divide
themselves in half when they were read, leaving the operator to
recollate them manually.
There were some other effects produced, as well.
Naturally, the operator called in the operating-system developers. They
found the bandit ghost jobs running, and X'ed them... and were once
again surprised. When Robin Hood was X'ed, the following sequence of
events took place:
!X id1
id1: Friar Tuck... I am under attack! Pray save me!
id1: Off (aborted)
id2: Fear not, friend Robin! I shall rout the Sheriff of
Nottingham's men!
id1: Thank you, my good fellow!
Each ghost-job would detect the fact that the other had been killed,
and would start a new copy of the recently-slain program within a few
milliseconds. The only way to kill both ghosts was to kill them
simultaneously (very difficult) or to deliberately crash the system.
Finally, the system programmers did the latter... only to find that
the bandits appeared once again when the system rebooted! It turned
out that these two programs had patched the boot-time image (the
/vmunix file, in Unix terms) and had added themselves to the list of
programs that were to be started at boot time...
The Robin Hood and Friar Tuck ghosts were finally eradicated when the
system staff rebooted the system from a clean boot-tape and
reinstalled the monitor. Not long thereafter, Xerox released a patch
for this problem.
It is alleged that Xerox filed a complaint with Motorola's management about
the merry-prankster actions of the two employees in question. It is
not recorded that any serious disciplinary action was taken against
either of them.
=====
So, the upshot is to have multiple jobs (as many as you like), monitoring each other and restarting where necessary. You also need to protect against variants of the kill statement that can kill a whole slew of processes in one hit.