4

Here is my simple test file:

# cat test
grp_pt cmg 
grp_pt compsc 
grp_pt ffre 
grp_pt grp_pt 
grp_pt hpcedt 
grp_ptix dsnlp 
grp_ptix exinspect 
grp_ptix ptix 
grp_ptix sciml 
grp_ptix vers 
grp_ptix vsme 
grp_pt mhm 
grp_pt rec 
grp_pt recom 
grp_ptix grp_ptix 
grp_ptix iai 
grp_ptix mv 
grp_ptix pid

# sort -u test
grp_pt cmg 
grp_pt compsc 
grp_pt ffre 
grp_pt grp_pt 
grp_pt hpcedt 
grp_ptix dsnlp 
grp_ptix exinspect 
grp_ptix grp_ptix 
grp_ptix iai 
grp_ptix mv 
grp_ptix pid 
grp_ptix ptix 
grp_ptix sciml 
grp_ptix vers 
grp_ptix vsme 
grp_pt mhm 
grp_pt rec 
grp_pt recom

I expected the last 3 line should be together and right below of the first 5 lines. But it seems not doing that way.

Any suggestion?

Improve this question
3
  • 4
    Are you sure the delimiters everywhere are the same (space or tab)? It work perfectly fine with sort v9 Commented Jun 25 at 14:24
  • 1
    I'm going to guess that the last three lines don't have a normal space character, but have some unicode character, like a NO-BREAK SPACE or EN SPACE. Commented Jun 25 at 15:30
  • The space character is a normal space. The locale is a natural language one such as en_GB.UTF-8. This should be expected behaviour per my suggested duplicate Commented Jun 26 at 7:15

2 Answers 2

Reset to default
8

This looks related to your locale settings and particularly your collation order setting. To figure out if this is correct behaviour we'd need to know what those settings are. It's often said that your chances of finding a new bug are slim, although they are perhaps higher with a lesser used language.

From information you've provided, it seems clear that the result is correctly in alphabetical order ignoring spaces.

That would seem to suggest your locale settings ignore spaces when comparing strings from your current language. At least they do for the lines of text you've compared.

Under the hood this is being driven by either strcoll or strxfrm, and it's likely that strxfrm is simply stripping the white space for some reason.

I suspect that if you ran this with LC_ALL=C you'd get the result you were expecting:

LC_ALL=C sort -u test

If so then you can set the sort order to be different than your main language settings with the environment variable LC_COLLATE.

Beyond that you'd need to investigate the collation rules for your current language (locale) settings.

Improve this answer
1
  • Thank you so much. This works for me Commented Jun 26 at 14:30
3

We have many ways of sorting lines.

Apparently, your current sort configuration (locate, collate, etc) is ignoring spaces. (my configuration is equal to the one presented)

Suggestion: use the more intuitive option of sort -- Version-sort

sort -V file

Version-sort works well with words, numbers

Improve this answer
4
  • 1
    Would you mind adding your locale to your answer (LANG environment variable). It would’ve good to have an example locale that behaves this way. Commented Jun 26 at 0:38
  • @Philip Couling, echo $LANG gives me en_US.UTF-8 (linux mint 22.1)... and locale -a is C C.utf8 en_AG en_AG.utf8 en_AU.utf8 en_BW.utf8 en_CA.utf8 en_DK.utf8 en_GB.utf8 en_HK.utf8 en_IE.utf8 en_IL en_IL.utf8 en_IN en_IN.utf8 en_NG en_NG.utf8 en_NZ.utf8 en_PH.utf8 en_SG.utf8 en_US.utf8 en_ZA.utf8 en_ZM en_ZM.utf8 en_ZW.utf8 portuguese POSIX pt_PT pt_PT.iso88591 pt_PT.utf8 Commented Jun 26 at 13:36
  • @Philip Couling, all my LANG=<lang_variant>.utf8 sort file are return the "ignore space" behaviour; C, C.utf8 are working as expected. Commented Jun 26 at 13:44
  • Thanks. This also works, but it seems sortinggrp_ptix before grp_pt $ sort -V t1 grp_ptix dsnlp grp_ptix exinspect grp_ptix grp_ptix grp_ptix iai grp_ptix mv grp_ptix pid grp_ptix ptix grp_ptix sciml grp_ptix vers grp_ptix vsme grp_pt cmg grp_pt compsc grp_pt ffre grp_pt grp_pt grp_pt hpcedt grp_pt mhm grp_pt recom grp_pt rec Commented Jun 26 at 14:34

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.