How to overcome an incompatibility between the ksh on Linux vs. that installed on AIX/Solaris/HPUX?

Question

I am involved in the process of porting a system containing several hundreds of ksh scripts from AIX, Solaris and HPUX to Linux. I have come across the following difference in the way ksh behaves on the two systems:

#!/bin/ksh
flag=false
echo "a\nb" | while read x
do
    flag=true
done
echo "flag = ${flag}"
exit 0

On AIX, Solaris and HPUX the output is "flag = true" on Linux the output is "flag = false".

My questions are:

• Is there an environment variable that I can set to get Linux's ksh to behave like the other Os's'? Failing that:
• Is there an option on Linux's ksh to get the required behavior? Failing that:
• Is there a ksh implementation available for Linux with the desired behavior?

Other notes:

• On AIX, Solaris and HPUX ksh is a variant of ksh88.
• On Linux, ksh is the public domain ksh (pdksh)
• On AIX, Solaris and HPUX dtksh and ksh93 (where I have them installed) are consistent with ksh
• The Windows NT systems I have access to: Cygwin and MKS NT, are consistent with Linux.
• On AIX, Solaris and Linux, bash is consistent, giving the incorrect (from my perspective) result of "flag = false".

The following table summarizes the systems the problem:

uname -s       uname -r                   which ksh          ksh version                     flag =
========       ========                   =========          ===========                     ======
Linux          2.6.9-55.0.0.0.2.ELsmp     /bin/ksh           PD KSH v5.2.14 99/07/13.2       false
AIX            3                          /bin/ksh           Version M-11/16/88f             true    // AIX 5.3
                                          /bin/ksh93         Version M-12/28/93e             true
SunOS          5.8, 5.9 and 5.10          /bin/ksh           Version M-11/16/88i             true
                                          /usr/dt/bin/dtksh  Version M-12/28/93d             true
HP-UX          B.11.11 and B.11.23        /bin/ksh           Version 11/16/88                true
                                          /usr/dt/bin/dtksh  Version M-12/28/93d             true
CYGWIN_NT-5.1  1.5.25(0.156/4/2)          /bin/ksh           PD KSH v5.2.14 99/07/13.2       false
Windows_NT     5                          .../mksnt/ksh.exe  Version 8.7.0 build 1859...     false    // MKS

Update

After some advice from people in my company we decided to make the following modification to the code. This gives us the same result whether using the "real" ksh's (ksh88, ksh93) or any of the ksh clones (pdksh, MSK ksh). This also works correctly with bash.

#!/bin/ksh
echo "a\nb" > junk
flag=false
while read x
do
    flag=true
done < junk
echo "flag = ${flag}"
exit 0

Thanks to jj33 for the previously accepted answer.

Answer 1

I don't know of any particular option to force ksh to be compatible with a particular older version. That said, perhaps you could install a very old version of ksh on your linux box, and have it behave in a compatible manner?

It might be easier to install a more modern version of amy shell on the AIX/HP-UX boxes, and just migrate your scripts to use sh. I know there are versions of bash available for all platforms.

Answer 2

Instead of using pdksh on linux, use the "real" ksh from kornshell.org. pdksh is a blind re-implementation of ksh. kornshell.org is the original korn shell dating back 25 years or so (the one written by David Korn). AIX and Solaris use versions of the original ksh, so the kornshell.org version is usually feature- and bug- complete. Having cut my teeth with SunOS/Solaris, installing kornshell.org ksh is usually one of the first things I do on a new Linux box...

Answer 3

Your script gives the correct (true) output when zsh is used with the emulate -L ksh option. If all else fails you may wish to try using zsh on Linux.

Answer 4

I installed 'ksh' and 'pdksh' on my local Ubuntu Hardy system.

ii  ksh            93s+20071105-1 The real, AT&T version of the Korn shell
ii  pdksh          5.2.14-21ubunt A public domain version of the Korn shell

ksh has the "correct" behavior that you're expecting while pdksh does not. You might check your local Linux distribution's software repository for a "real" ksh, instead of using pdksh. The "Real Unix" OS's are going to install the AT&T version of Korn shell, rather than pdksh, by default, what with them being based off AT&T Unix (System V) :-).

Answer 5

Do you have to stay within ksh?

Even if you use the same ksh you'll still call all kinds of external commands (grep, ps, cat, etc...) part of them will have different parameters and different output from system to system. Either you'll have to take in account those differences or use the GNU version of each one of them to make things the same.

The Perl programming language originally was designed exactly to overcome this problem. It includes all the features a unix shell programmer would want from he shell program but it is the same on every Unix system. You might not have the latest version on all those systems, but if you need to install something, maybe it is better to install perl.

Answer 6

After some advice from people in my company we decided to make the following modification to the code. This gives us the same result whether using the "real" ksh's (ksh88, ksh93) or any of the ksh clones (pdksh, MSK ksh). This also works correctly with bash.

#!/bin/ksh
echo "a\nb" > junk
flag=false
while read x
do
    flag=true
done < junk
echo "flag = ${flag}"
exit 0

Thanks to jj33 for the previous accepted answer.

Answer 7

The reason for the differences is whether the inside block is executed in the original shell context or in a subshell. You may be able to control this with the () and {} grouping commands. Using a temporary file, as you do in your update, will work most of the time but will run into problems if the script is run twice rapidly, or if it executes without clearing the file, etc.

#!/bin/ksh
flag=false
echo "a\nb" | { while read x
do    
     flag=true
done }
echo "flag = ${flag}"
exit 0

That may help with the problem you were getting on the Linux ksh. If you use parentheses instead of braces, you'll get the Linux behavior on the other ksh implementations.

Answer 8

This question is very much related: http://stackoverflow.com/questions/124167/bash-variable-scope