Why doesn't **sort** sort the same on every machine?

Question

Using the same sort command with the same input produces different results on different machines. How do I fix that?

Answer 1

What platform are you using? Can you share your input data?

My guess is it's something to do with a culture setting (invariant vs en-us, for example).

Answer 2

This can be the result of locale differences:

$ echo 'CO2_
CO_' | env LC_ALL=C sort
CO2_
CO_


$ echo 'CO2_
CO_' | env LC_ALL=en_US sort
CO_
CO2_

Setting the LC_ALL environment variable to the same value should correct the problem.

Answer 3

Is it possible that the differing machines have different version of sort? For instance, GNU sort is likely different from a Solaris sort, or have different default parameters.

Can you make a simple example sort that shows the differences? Also, get the version of each different sort.

-Aam

Answer 4

This is probably due to different settings of the locale environment variables. sort will use these settings to determine how to compare strings. By setting these environment variables the way you want before calling sort, you should be able to force it to behave in one specific way.

Answer 5

The man-page on OS X says:

*** WARNING *** The locale specified by the environment affects sort order. Set LC_ALL=C to get the traditional sort order that uses native byte values.

which might explain things.

If some of your systems have no locale support, they would default to that locale (C), so you wouldn't have to set it on those. If you have some that supports locales and want the same behavior, set LC_ALL=C on those systems. That would be the way to have as many systems as I know do it the same way.

If you don't have any locale-less systems, just making sure they share locale would probably be enough.

For more canonical information, see the The Single UNIX ® Specification, Version 2 description of locale, environment variables, setlocale() and the description of the sort(1) utility.

Answer 6

For more than you ever wanted to know about sort, read the specification of sort in the Single Unix Specification v3. It states

Comparisons [...] shall be performed using the collating sequence of the current locale.

IOW, how sort sorts is dependent on the locale (language) settings of the environment that the script is running under.