On 4/19/23 03:27, Martin Τrautmann wrote:
>
> Hi all,
>
> how do I sort by multiple columns?
>

awk

> Example:
> +++
> Borgentreich;D9386;Lindenstätte;1;;32;520150.696;5709236.354
> Borgentreich;D9444;Auf der Lindenstätte;1;;32;519950.850;5708982.109
> Borgentreich;D9444;Auf der Lindenstätte;2;;32;519926.937;5708966.116
> Borgentreich;D9444;Auf der Lindenstätte;3;;32;520008.619;5709083.464
> Borgentreich;D9444;Auf der Lindenstätte;4;;32;519860.278;5709041.468
> Borgentreich;T2960;Lindenstätte;12;;32;519622.835;5709023.590
> Borgentreich;T2960;Lindenstätte;6;;32;519696.745;5709038.833
> Borgentreich;T2960;Lindenstätte;4;;32;519722.956;5709043.915
> Borgentreich;T2960;Lindenstätte;15;;32;519489.638;5709077.693
> Borgentreich;T2960;Lindenstätte;24;;32;519518.763;5709090.026
> Borgentreich;T2960;Lindenstätte;18;;32;519559.108;5709037.356
> Borgentreich;T2960;Lindenstätte;14;;32;519596.623;5709013.684
> Borgentreich;T2960;Lindenstätte;16;;32;519569.141;5709017.854
> Borgentreich;T2960;Lindenstätte;22;;32;519540.257;5709072.032
> Borgentreich;T2960;Lindenstätte;26;;32;519503.270;5709103.321
> Borgentreich;T2960;Lindenstätte;2;;32;519758.267;5709057.635
> Borgentreich;T2960;Lindenstätte;10;;32;519648.417;5709028.865
> Borgentreich;T2960;Lindenstätte;11;;32;519607.438;5708989.545
> Borgentreich;T2960;Lindenstätte;3;;32;519732.686;5709020.833
> Borgentreich;T2960;Lindenstätte;7;;32;519678.983;5709007.380
> Borgentreich;T2960;Lindenstätte;9;;32;519651.859;5709000.462
> Borgentreich;T2960;Lindenstätte;5;;32;519708.841;5709015.137
> Borgentreich;T2960;Lindenstätte;1;;32;519778.725;5709026.584
> Borgentreich;T2960;Lindenstätte;8;;32;519673.036;5709040.372
> +++
>
> I want to sort
> * first by column 4, numerical,
> * second by column 2
> * third by column 3
>
> So the result should be
> +++
> Borgentreich;D9444;Auf der Lindenstätte;1;;32;519950.850;5708982.109
> Borgentreich;D9444;Auf der Lindenstätte;2;;32;519926.937;5708966.116
> Borgentreich;D9444;Auf der Lindenstätte;3;;32;520008.619;5709083.464
> Borgentreich;D9444;Auf der Lindenstätte;4;;32;519860.278;5709041.468
> Borgentreich;D9386;Lindenstätte;1;;32;520150.696;5709236.354
> Borgentreich;T2960;Lindenstätte;1;;32;519778.725;5709026.584
> Borgentreich;T2960;Lindenstätte;2;;32;519758.267;5709057.635
> Borgentreich;T2960;Lindenstätte;3;;32;519732.686;5709020.833
> Borgentreich;T2960;Lindenstätte;4;;32;519722.956;5709043.915
> Borgentreich;T2960;Lindenstätte;5;;32;519708.841;5709015.137
> Borgentreich;T2960;Lindenstätte;6;;32;519696.745;5709038.833
> Borgentreich;T2960;Lindenstätte;7;;32;519678.983;5709007.380
> Borgentreich;T2960;Lindenstätte;8;;32;519673.036;5709040.372
> Borgentreich;T2960;Lindenstätte;9;;32;519651.859;5709000.462
> Borgentreich;T2960;Lindenstätte;10;;32;519648.417;5709028.865
> Borgentreich;T2960;Lindenstätte;11;;32;519607.438;5708989.545
> Borgentreich;T2960;Lindenstätte;12;;32;519622.835;5709023.590
> Borgentreich;T2960;Lindenstätte;14;;32;519596.623;5709013.684
> Borgentreich;T2960;Lindenstätte;15;;32;519489.638;5709077.693
> Borgentreich;T2960;Lindenstätte;16;;32;519569.141;5709017.854
> Borgentreich;T2960;Lindenstätte;18;;32;519559.108;5709037.356
> Borgentreich;T2960;Lindenstätte;22;;32;519540.257;5709072.032
> Borgentreich;T2960;Lindenstätte;24;;32;519518.763;5709090.026
> Borgentreich;T2960;Lindenstätte;26;;32;519503.270;5709103.321
> +++
>
> I tried both
> sort -k4 -t";" -n | sort -k2,2 -t";" | sort -k3,3 -t";"
> and
> sort -k4 -t";" -n -k2,2 -k3,3
> and some permutations and reverted orders, without success.
> The sort by column 4 just gets lost or resorted.
>
> I'm not sure about the man page
> -k, --key=POS1[,POS2]
> start a key at POS1, end it at POS2 (origin 1)
>
> So I tried relative positions with
> -k3,1
> as well, without success.
>
> How do I apply the sort syntax properly?
>
> Thanks
> Martin

On Sat, 6 May 2023 02:03:24 -0400, Popping Mad wrote:
> On 4/19/23 03:27, Martin Τrautmann wrote:
>>
>> Hi all,
>>
>> how do I sort by multiple columns?
>>
>
> awk

Nope. "awk" alone does not to the job.

On Fri, 5 May 2023 14:53:24 -0000 (UTC), Kenny McCormack wrote:
> Finally, I note that you mentioned Xcode. That's a Mac/Apple thing. Is
> this "Miller" a specifically Apple thing?

Absolutely not. But if I want to install it the easy way I do a
sudo port install miller
which does require more of the xcode installation which I acutally have.

On 2023-05-06, Martin Τrautmann <t-usenet@gmx.net> wrote:
> On Sat, 6 May 2023 02:03:24 -0400, Popping Mad wrote:
>> On 4/19/23 03:27, Martin Τrautmann wrote:
>>>
>>> Hi all,
>>>
>>> how do I sort by multiple columns?
>>>
>>
>> awk
>
> Nope. "awk" alone does not to the job.

You may be able to cob together with GNU Awk, which provides:

- controlling the traversal of associative array to be in sorted orders.

- the asort function for sorting an associative array
(the indices are clobbered to a 1..N enumeration).

- the asorti function which sorts the indices instead: they
become the values, and indices go to 1..N.

A user-defined comparison can be used in asort and asorti,
which receives all four relevant values: left key and value,
right key and value.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

In article <slrnu5c1i5.3rj.t-usenet@ID-685.user.individual.de>,
Martin rautmann <traut@gmx.de> wrote:
>On Sat, 6 May 2023 02:03:24 -0400, Popping Mad wrote:
>> On 4/19/23 03:27, Martin rautmann wrote:
>>>
>>> Hi all,
>>>
>>> how do I sort by multiple columns?
>>>
>>
>> awk
>
>Nope. "awk" alone does not to the job.

Yes. As Kaz explains.

--
"Women should not be enlightened or educated in any way. They should be
segregated because they are the cause of unholy erections in holy men.

-- Saint Augustine (354-430) --

I use Homebrew on my Macs, which has it but you can also pull the
tar.gz from

https://github.com/johnkerl/miller/releases/tag/v6.7.0

and after extracting cd to the directory and run

xattr -cr mlr

greetings, el

On 05/05/2023 16:35, Martin Τrautmann wrote:
[...]
> Error: Port miller requires a full Xcode installation, which was not
> found on your system.
>
> ...and I've not enough space for that, 256 GB SSD only.

Homebrew only needs the Xcode Command Line Tools, but only if you
wish to or have to compile sources. It's mainly binaries only and
works perfectly well without if if you can tolerate the squealing
:-)-O

I don't want to start a war on that but I find Homebrew more complete.

el

On 06/05/2023 01:45, gerg wrote:
[...]
> Homebrew is a thing on MacOS. A thing that seems to include
> miller v6.7.0.
> <https://formulae.brew.sh/formula/miller#default>
>
> (Homebrew only needs the Xcode runtime, not the full install)
>

On 05/05/2023 16:53, Kenny McCormack wrote:
[...]
> (Quoting Arte Johnson) Interesting... But not very.
>
> I did a little research on "Miller" (which I had never heard of
> before today).

Obviously not enough research, but then that's a bliss :-)-O

> I don't see the point of it. It just seems like another AWK (or
> Perl or Ruby or Python or ...). I.e., what I am saying is that
> the only reason to use the "traditional" Unix tools (cut, join,
> comm, sort, sed, grep, etc, etc) is because you just don't want
> to learn anything new (*). If you were going to learn something
> new, just learn AWK - which can do all the things that any/all
> of those "traditional" tools can do - and much more. Why bother
> to learn this "Miller" thing?

Because it is extremely powerful and can do a lot of things awk
can't do or not do as well or as easily.

It has become one of a few of my goto tools to slice, dice and
look at CSV (and variants)

MILLER
QSV
CSVQ
CSVIEW
CSVLENS
(TYPST)

The first three and typst (which is a new but VERY promising
typesetting software) are under active development, the other two
not so much but they are just viewers and stable.

I have written me a large number of bash functions using
combinations of the above which make my life (Gynecologist six
months before retirement, doing my own admin and book keeping)
much easier (and time is money :-)-O)

From reconciling bank statements, over declaring Value Added and
Employees' Tax submitting claims to Funders.

I have been able to retire some long used Perl Scripts which are
much longer and in spite of having commented them very well I
don't understand any more :-)-O

[...]
> Finally, I note that you mentioned Xcode. That's a Mac/Apple
> thing. Is this "Miller" a specifically Apple thing?

See above, my research comment.

That means, in my understanding, that MacPorts wants to compile
from source and for that it apparently wants the full XCode.

el

On Sat, 6 May 2023 09:47:05 -0000 (UTC), Kaz Kylheku wrote:
> On 2023-05-06, Martin Τrautmann <t-usenet@gmx.net> wrote:
>> On Sat, 6 May 2023 02:03:24 -0400, Popping Mad wrote:
>>> On 4/19/23 03:27, Martin Τrautmann wrote:
>>>>
>>>> Hi all,
>>>>
>>>> how do I sort by multiple columns?
>>>>
>>>
>>> awk
>>
>> Nope. "awk" alone does not to the job.
>
> You may be able to cob together with GNU Awk, which provides:

I agree that it might be able. But there's no solution given.
But why look for an awk solution, it there's a reasonable sort solution
which helped me to learn about sort options and principles.

naming just "awk" is not even as helpful as claiming "excel"

In article <slrnu5ep43.3rj.t-usenet@ID-685.user.individual.de>,
Martin rautmann <traut@gmx.de> wrote:
....
>I agree that it might be able. But there's no solution given.
>But why look for an awk solution, it there's a reasonable sort solution
>which helped me to learn about sort options and principles.

Short answer: If the discussion of AWK (and various other tools) doesn't
make sense to you, you should probably just ignore it.

>naming just "awk" is not even as helpful as claiming "excel"

It wasn't really intended to be "helpful", in the sense in which you are
using that term. It was a side discussion, inspired by the introduced
diversion of yet another tool (the one called "Miller").

Finally, I should note that both of the following are true:

1) I never liked the Unix "sort" command. I always found its command
line syntax "weird". So, I've pretty much avoided using it, all
these years.
2) It's not worth learning AWK just to sort a file. But if you already
know AWK, it will seem like a natural way to sort a file.

--
"We are in the beginning of a mass extinction, and all you can talk
about is money and fairy tales of eternal economic growth."

- Greta Thunberg -

On Sun, 7 May 2023 09:52:38 -0000 (UTC), Kenny McCormack wrote:
> In article <slrnu5ep43.3rj.t-usenet@ID-685.user.individual.de>,
> Martin rautmann <traut@gmx.de> wrote:
> ...
>>I agree that it might be able. But there's no solution given.
>>But why look for an awk solution, it there's a reasonable sort solution
>>which helped me to learn about sort options and principles.
>
> Short answer: If the discussion of AWK (and various other tools) doesn't
> make sense to you, you should probably just ignore it.

I'd like to learn more about awk - but I don't like it that much to
actually find out how awk would do this job, since awk can do that much
more than just sort.

>>naming just "awk" is not even as helpful as claiming "excel"
>
> It wasn't really intended to be "helpful", in the sense in which you are
> using that term. It was a side discussion, inspired by the introduced
> diversion of yet another tool (the one called "Miller").

I did not recognize that from where within this thread this mad's answer
came. Miller was not mentionned there anywhere. In fact it looked like a
full quote with some random single word thrown in somewhere.

On 07/05/2023 13:00, Martin Τrautmann wrote:
> On Sun, 7 May 2023 09:52:38 -0000 (UTC), Kenny McCormack wrote:
>> In article <slrnu5ep43.3rj.t-usenet@ID-685.user.individual.de>,
>> Martin rautmann <traut@gmx.de> wrote:
>> ...
>>> I agree that it might be able. But there's no solution given.
>>> But why look for an awk solution, it there's a reasonable sort solution
>>> which helped me to learn about sort options and principles.
>>
>> Short answer: If the discussion of AWK (and various other tools) doesn't
>> make sense to you, you should probably just ignore it.
>
> I'd like to learn more about awk - but I don't like it that much to
> actually find out how awk would do this job, since awk can do that much
> more than just sort.
>

A start:

sed & awk
Second Edition
Dale Dougherty and Arnold Robbins
https://doc.lagout.org/operating%20system%20/linux/Sed%20%26%20Awk.pdf

--
Chris Elvidge
England

On 07.05.2023 15:08, Chris Elvidge wrote:
> On 07/05/2023 13:00, Martin Τrautmann wrote:
>> On Sun, 7 May 2023 09:52:38 -0000 (UTC), Kenny McCormack wrote:
>>> In article <slrnu5ep43.3rj.t-usenet@ID-685.user.individual.de>,
>>> Martin rautmann <traut@gmx.de> wrote:
>>> ...
>>>> I agree that it might be able. But there's no solution given.
>>>> But why look for an awk solution, it there's a reasonable sort solution
>>>> which helped me to learn about sort options and principles.
>>>
>>> Short answer: If the discussion of AWK (and various other tools) doesn't
>>> make sense to you, you should probably just ignore it.
>>
>> I'd like to learn more about awk - but I don't like it that much to
>> actually find out how awk would do this job, since awk can do that much
>> more than just sort.
>>
>
> A start:
>
> sed & awk
> Second Edition
> Dale Dougherty and Arnold Robbins
> https://doc.lagout.org/operating%20system%20/linux/Sed%20%26%20Awk.pdf
>

This is probably not a good suggestion, where Martin wrote "[...] how
awk would do this job."; we were talking about sorting in this thread,
and that book mostly just refers to how to use the _external_ 'sort'
command (that was IMO anyway the right tool to do the job) called from
within awk. There's no mention of 'asort()' (or other sort functions
built-in in GNU awk). A function is described, though, written in awk,
how to write s sorting function; but re-implementation should IMO not
be the approach how to use awk to sort things. Unix'es 'sort' fits
exactly and has powerful flexibility to solve the elementary sorting
tasks efficiently as we've already seen some weeks ago in this thread.

Thanks for the link, BTW. That way I could have a peek into that book
and see which areas it covers and in which depth.

Janis

In article <u38c1p$3dcuv$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
....
>Thanks for the link, BTW. That way I could have a peek into that book
>and see which areas it covers and in which depth.

If I were going to make a book recommendation (for AWK), it would be for
Robbbins's EAP. That covers GAWK specifically - which is the only AWK
anyone should be using nowadays (other than TAWK, of course).

--
The scent of awk programmers is a lot more attractive to women than
the scent of perl programmers.

(Mike Brennan, quoted in the "GAWK" manual)

On 07.05.2023 18:39, Kenny McCormack wrote:
> In article <u38c1p$3dcuv$1@dont-email.me>,
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> ...
>> Thanks for the link, BTW. That way I could have a peek into that book
>> and see which areas it covers and in which depth.
>
> If I were going to make a book recommendation (for AWK), it would be for
> Robbbins's EAP. That covers GAWK specifically - which is the only AWK
> anyone should be using nowadays (other than TAWK, of course).

Effective Awk Programming
It should be noted that it (also) covers functionality that you
won't find in traditional awk, in the versions that you find on
commercial Unix systems. The book marks GNU specific/non-standard
extensions, though.

The Awk Programming Language
For "plain old" Awk I'd still recommend (also) the old book from
the Awk's authors (Aho, Kernighan, Weinberger).

Sed & Awk
The book behind Chris' link isn't bad; besides all the basic Awk
knowledge it shows, for example, how to combine Awk with other
tools through pipes. The emphasis on Sed, though, looks (to me)
quite anachronistic. In the light of Awk the presence of Sed is
of arguable value. We may consider it just two independent topics
(Sed, Awk) in one book. (Only that they save duplication of the
Regexp chapter.) It probably would have made more sense if the
book would show the strength of the specific tools, and what's
the difference between the same functionality in one tool or the
other. As presented it's just one book for two separate tools.

For the mentioned aspect of combining tools I recall that a book
called (something like) "The Unix Tool-Chest" - or was it "The
Unix Programming Environment" (Kernighan, Pike?) - was very good
and, as far as I recall, also more complete and with convincing
examples.

The inherent shortcomings of one-dimensional pipe-processing vs.
(stateful) processing (e.g. in Awk) should also be kept in mind
for contexts where it matters.

Janis

Chris Elvidge <chris@mshome.net> writes:
[...]
> A start:
>
> sed & awk
> Second Edition
> Dale Dougherty and Arnold Robbins
> https://[...]/Sed%20%26%20Awk.pdf

That's a pirated copy. The second edition was published in 1997
and is still under copyright.

https://www.oreilly.com/library/view/sed-awk/1565922255/

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for XCOM Labs
void Void(void) { Void(); } /* The recursive call of the void */

On 2023-05-07, Kenny McCormack <gazelle@shell.xmission.com> wrote:
> In article <u38c1p$3dcuv$1@dont-email.me>,
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> ...
>>Thanks for the link, BTW. That way I could have a peek into that book
>>and see which areas it covers and in which depth.
>
> If I were going to make a book recommendation (for AWK), it would be for
> Robbbins's EAP. That covers GAWK specifically - which is the only AWK
> anyone should be using nowadays (other than TAWK, of course).

I agree that if you want to write in Awk because you see yourself as an
Awk developer, where all your target systems will somehow be running the
implementation that you prefer, then that probably would be GNU Awk.

While that's a nice sentiment, one big reason people write a script
in awk is that nothing has to be installed, and the problem they
are trying to solve is outside of the algorithmic expressiveness
of the shell and (other) utilities.

For that to work out, those people have to work with whatever awk is
installed.

As soon as you have the leeway to add something else to the target
systems's installation, non-Awk choices are likely on the table too.

For instance, on a good many Linux-based embedded systems, you
get BusyBox Awk. (BusyBox puts all utilities into a single executable,
so your /usr/bin/awk is a hard link to busybox.)

Mawk appears in the default installation of some GNU/Linux distros;
if you have a dependency on GNU Awk, you have to manipulate the
system installation, or ask your downstream user to do that.
(If that's okay, again, other tools could be on the table.)

MacOS's awk doesn't appear to be GNU Awk; it's something from BSD.
If you want Awk code to run on every installation of MacOS without
installing anything other than that code, that's the awk it
has to work with.

(There are also other proprietary Unixes. No longer highly relevant, but
still there.)

Awk is a POSIX tool; you can use POSIX's description of Awk as a guide
for writing portable code. (And even then, if you're dealing with
installations of legacy Unixes, be prepared for some "broken old Awk").

(Maybe what you meant is that GNU Awk is the only Awk that system
vendors should be installing in their images so that everyone could then
just write GNU Awk code, worrying only about what version is installed.
There are probably some systems that would be better off that way.
BusyBox Awk is used for some good reasons, like saving flash space in
constrained systems. Either way, what people should be doing and what
they are doing is different.)

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

Kaz Kylheku <864-117-4973@kylheku.com> writes:
[...]
> For instance, on a good many Linux-based embedded systems, you
> get BusyBox Awk. (BusyBox puts all utilities into a single executable,
> so your /usr/bin/awk is a hard link to busybox.)

A minor point: busybox applets, including awk, are typically *symbolic*
links to busybox. (There might be a configuration option to use hard
links; I haven't checked.)

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for XCOM Labs
void Void(void) { Void(); } /* The recursive call of the void */

Martin Τrautmann wrote:

> Hi all,
>
> how do I sort by multiple columns?
>
> [snip
>
> I want to sort
> * first by column 4, numerical,
> * second by column 2
> * third by column 3
>
> So the result should be
> +++
> Borgentreich;D9444;Auf der Lindenstätte;1;;32;519950.850;5708982.109
> Borgentreich;D9444;Auf der Lindenstätte;2;;32;519926.937;5708966.116
> Borgentreich;D9444;Auf der Lindenstätte;3;;32;520008.619;5709083.464
> Borgentreich;D9444;Auf der Lindenstätte;4;;32;519860.278;5709041.468
> Borgentreich;D9386;Lindenstätte;1;;32;520150.696;5709236.354
> Borgentreich;T2960;Lindenstätte;1;;32;519778.725;5709026.584
> [snip]
> +++

Hi Martin,

Are you sure that this example result is correct? The second, third, and
fourth lines have values of 2, 3, and 4 in column 4, but then the fifth line
has a value of 1 in column 4. That doesn't seem to match your description of
the sorting logic, unless I'm missing something.

Assuming your description of the logic is right, though, and assuming you
have access to GNU sort (I'm using "sort (GNU coreutils) 9.1"), I think you
can use

sort -t ';' -k 4g -k 2 -k 3

The result this gives is

> Borgentreich;D9386;Lindenstätte;1;;32;520150.696;5709236.354
> Borgentreich;D9444;Auf der Lindenstätte;1;;32;519950.850;5708982.109
> Borgentreich;T2960;Lindenstätte;1;;32;519778.725;5709026.584
> Borgentreich;D9444;Auf der Lindenstätte;2;;32;519926.937;5708966.116
> Borgentreich;T2960;Lindenstätte;2;;32;519758.267;5709057.635
> Borgentreich;D9444;Auf der Lindenstätte;3;;32;520008.619;5709083.464
> Borgentreich;T2960;Lindenstätte;3;;32;519732.686;5709020.833
> Borgentreich;D9444;Auf der Lindenstätte;4;;32;519860.278;5709041.468
> Borgentreich;T2960;Lindenstätte;4;;32;519722.956;5709043.915
> Borgentreich;T2960;Lindenstätte;5;;32;519708.841;5709015.137
> Borgentreich;T2960;Lindenstätte;6;;32;519696.745;5709038.833
> Borgentreich;T2960;Lindenstätte;7;;32;519678.983;5709007.380
> Borgentreich;T2960;Lindenstätte;8;;32;519673.036;5709040.372
> Borgentreich;T2960;Lindenstätte;9;;32;519651.859;5709000.462
> Borgentreich;T2960;Lindenstätte;10;;32;519648.417;5709028.865
> Borgentreich;T2960;Lindenstätte;11;;32;519607.438;5708989.545
> Borgentreich;T2960;Lindenstätte;12;;32;519622.835;5709023.590
> Borgentreich;T2960;Lindenstätte;14;;32;519596.623;5709013.684
> Borgentreich;T2960;Lindenstätte;15;;32;519489.638;5709077.693
> Borgentreich;T2960;Lindenstätte;16;;32;519569.141;5709017.854
> Borgentreich;T2960;Lindenstätte;18;;32;519559.108;5709037.356
> Borgentreich;T2960;Lindenstätte;22;;32;519540.257;5709072.032
> Borgentreich;T2960;Lindenstätte;24;;32;519518.763;5709090.026
> Borgentreich;T2960;Lindenstätte;26;;32;519503.270;5709103.321

Hope this helps!

Benjamin

On Mon, 08 May 2023 11:46:35 -0400, Benjamin Esham wrote:
> Martin Τrautmann wrote:
>
>> Hi all,
>>
>> how do I sort by multiple columns?
>>
>> [snip
>>
>> I want to sort
>> * first by column 4, numerical,
>> * second by column 2
>> * third by column 3
>>
>> So the result should be
>> +++
>> Borgentreich;D9444;Auf der Lindenstätte;1;;32;519950.850;5708982.109
>> Borgentreich;D9444;Auf der Lindenstätte;2;;32;519926.937;5708966.116
>> Borgentreich;D9444;Auf der Lindenstätte;3;;32;520008.619;5709083.464
>> Borgentreich;D9444;Auf der Lindenstätte;4;;32;519860.278;5709041.468
>> Borgentreich;D9386;Lindenstätte;1;;32;520150.696;5709236.354
>> Borgentreich;T2960;Lindenstätte;1;;32;519778.725;5709026.584
>> [snip]
>> +++
>
> Hi Martin,
>
> Are you sure that this example result is correct? The second, third, and
> fourth lines have values of 2, 3, and 4 in column 4, but then the fifth line
> has a value of 1 in column 4. That doesn't seem to match your description of
> the sorting logic, unless I'm missing something.

Maybe my description is wrong, determined what is a "sticky" sort from
other applications.

The first sort is done numerical - resulting in the correct order of 1
to 4, from column 4.

The next order is by column 2, keeping the numerical sort of column 4
and grouping thise together.

But then I want to do the final sort on column 3, which does resort by
those names, but does group "Lindenstätte" together, keeping the sort
oder of the former D9386 vs. T2960

So in "sort" terms, the expected order is not 4>2>3, but 3>2>4,
depending on how the sort actually does proceed. That's why I had given
this example - that's the order I have to apply in spreadsheets or
relational databases for a stepwise pipe.

pipe vs. direkt sort seems to be in reverse orders!?

> Assuming your description of the logic is right, though, and assuming you
> have access to GNU sort (I'm using "sort (GNU coreutils) 9.1"), I think you
> can use
>
> sort -t ';' -k 4g -k 2 -k 3
>
> The result this gives is
>
>> Borgentreich;D9386;Lindenstätte;1;;32;520150.696;5709236.354
>> Borgentreich;D9444;Auf der Lindenstätte;1;;32;519950.850;5708982.109
>> Borgentreich;T2960;Lindenstätte;1;;32;519778.725;5709026.584
>> Borgentreich;D9444;Auf der Lindenstätte;2;;32;519926.937;5708966.116
>> Borgentreich;T2960;Lindenstätte;2;;32;519758.267;5709057.635
>> Borgentreich;D9444;Auf der Lindenstätte;3;;32;520008.619;5709083.464
>> Borgentreich;T2960;Lindenstätte;3;;32;519732.686;5709020.833
>> Borgentreich;D9444;Auf der Lindenstätte;4;;32;519860.278;5709041.468
>> Borgentreich;T2960;Lindenstätte;4;;32;519722.956;5709043.915
>> Borgentreich;T2960;Lindenstätte;5;;32;519708.841;5709015.137
>> Borgentreich;T2960;Lindenstätte;6;;32;519696.745;5709038.833
>> Borgentreich;T2960;Lindenstätte;7;;32;519678.983;5709007.380
>> Borgentreich;T2960;Lindenstätte;8;;32;519673.036;5709040.372
>> Borgentreich;T2960;Lindenstätte;9;;32;519651.859;5709000.462
>> Borgentreich;T2960;Lindenstätte;10;;32;519648.417;5709028.865
>> Borgentreich;T2960;Lindenstätte;11;;32;519607.438;5708989.545
>> Borgentreich;T2960;Lindenstätte;12;;32;519622.835;5709023.590
>> Borgentreich;T2960;Lindenstätte;14;;32;519596.623;5709013.684
>> Borgentreich;T2960;Lindenstätte;15;;32;519489.638;5709077.693
>> Borgentreich;T2960;Lindenstätte;16;;32;519569.141;5709017.854
>> Borgentreich;T2960;Lindenstätte;18;;32;519559.108;5709037.356
>> Borgentreich;T2960;Lindenstätte;22;;32;519540.257;5709072.032
>> Borgentreich;T2960;Lindenstätte;24;;32;519518.763;5709090.026
>> Borgentreich;T2960;Lindenstätte;26;;32;519503.270;5709103.321

No, that's not what I want.

Martin Τrautmann wrote:

> On Mon, 08 May 2023 11:46:35 -0400, Benjamin Esham wrote:
>> Martin Τrautmann wrote:
>>
>>> Hi all,
>>>
>>> how do I sort by multiple columns?
>>>
>>> [snip]
>>>
>>> I want to sort
>>> * first by column 4, numerical,
>>> * second by column 2
>>> * third by column 3
>>>
>>> So the result should be
>>> +++
>>> Borgentreich;D9444;Auf der Lindenstätte;1;;32;519950.850;5708982.109
>>> Borgentreich;D9444;Auf der Lindenstätte;2;;32;519926.937;5708966.116
>>> Borgentreich;D9444;Auf der Lindenstätte;3;;32;520008.619;5709083.464
>>> Borgentreich;D9444;Auf der Lindenstätte;4;;32;519860.278;5709041.468
>>> Borgentreich;D9386;Lindenstätte;1;;32;520150.696;5709236.354
>>> Borgentreich;T2960;Lindenstätte;1;;32;519778.725;5709026.584
>>> [snip]
>>> +++
>>
>> Hi Martin,
>>
>> Are you sure that this example result is correct? The second, third, and
>> fourth lines have values of 2, 3, and 4 in column 4, but then the fifth
>> line has a value of 1 in column 4. That doesn't seem to match your
>> description of the sorting logic, unless I'm missing something.
>
> Maybe my description is wrong, determined what is a "sticky" sort from
> other applications.
>
> The first sort is done numerical - resulting in the correct order of 1
> to 4, from column 4.
>
> The next order is by column 2, keeping the numerical sort of column 4
> and grouping thise together.
>
> But then I want to do the final sort on column 3, which does resort by
> those names, but does group "Lindenstätte" together, keeping the sort
> oder of the former D9386 vs. T2960
>
> So in "sort" terms, the expected order is not 4>2>3, but 3>2>4,
> depending on how the sort actually does proceed. That's why I had given
> this example - that's the order I have to apply in spreadsheets or
> relational databases for a stepwise pipe.

Ah, I think I understand. What you are calling the "first" sort is the
"innermost" sort, i.e., the sort that is applied *last*, and only if it is
necessary to break the tie between two rows that have been considered equal
by all of the previous sorting steps.

If I weren't able to get the results I wanted from sort(1), not even with
the GNU extensions, I would probably jump straight to importing the data
into an in-memory SQLite database and expressing the sort as a SQL query. In
your case, you would need to replace the semicolon separators with commas
and add a header line like

col1,col2,col3,col4,col5,col6,col7,col8

at the top of the file. Then, assuming the input is in a file named "input",
you could run

sqlite3 -noheader :memory: '.mode csv' .output \
".import '|cat -' data"
'select * from data order by col3, col2, cast(col4 as integer)' \
< input > output.csv

This produces a CSV file that obeys the informal standard that empty values,
and values with spaces in them, are double-quoted. If you don't want that,
you could replace the semicolons in the input with tabs, change the commas
to tabs in the header line, and run with ".mode tabs" instead of ".mode
csv". This would give you tab-separated output with no extra double quotes.

Of course, you allude to having this data in a relational database already.
I don't know the details of your situation, but in general a database seems
much better suited to this kind of manipulation than command-line tools that
can only deal with text.

Hope this helps!

Benjamin

On Tue, 09 May 2023 10:54:09 -0400, Benjamin Esham wrote:
> Ah, I think I understand. What you are calling the "first" sort is the
> "innermost" sort, i.e., the sort that is applied *last*, and only if it is
> necessary to break the tie between two rows that have been considered equal
> by all of the previous sorting steps.

Yeah, that's it. And the proper sort command had been given before:

sort -t\; -k4,4n -k2,2 -k3,3

> If I weren't able to get the results I wanted from sort(1), not even with
> the GNU extensions, I would probably jump straight to importing the data
> into an in-memory SQLite database and expressing the sort as a SQL query.

That's a little bit of an overkill. All I wanted to do is a manual
grep'n'sort in order to look at some special entries, from about 100k lines,
about 20 of them to look at, but best to look at them in a sorted way.

> Of course, you allude to having this data in a relational database already.
> I don't know the details of your situation, but in general a database seems
> much better suited to this kind of manipulation than command-line tools that
> can only deal with text.

I have this data within a relational database already - but the full set
of data (2 G records) is too much for additional handling (1 G records *
10 related records). I need the first and last related record here only,
for a handful of those (20*2 records).

On 09.05.2023 19:41, Martin Τrautmann wrote:
> On Tue, 09 May 2023 10:54:09 -0400, Benjamin Esham wrote:
>> Ah, I think I understand. What you are calling the "first" sort is the
>> "innermost" sort, i.e., the sort that is applied *last*, and only if it is
>> necessary to break the tie between two rows that have been considered equal
>> by all of the previous sorting steps.
>
> Yeah, that's it. And the proper sort command had been given before:
>
> sort -t\; -k4,4n -k2,2 -k3,3

In that light it's interesting how long (in time and number of posts)
this thread got. ;-)

Janis

On Tue, 09 May 2023 14:06:27 -0400, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

> On 09.05.2023 19:41, Martin Τrautmann wrote:
>> On Tue, 09 May 2023 10:54:09 -0400, Benjamin Esham wrote:
>>> Ah, I think I understand. What you are calling the "first" sort is the
>>> "innermost" sort, i.e., the sort that is applied *last*, and only if it is
>>> necessary to break the tie between two rows that have been considered equal
>>> by all of the previous sorting steps.
>>
>> Yeah, that's it. And the proper sort command had been given before:
>>
>> sort -t\; -k4,4n -k2,2 -k3,3
>
> In that light it's interesting how long (in time and number of posts)
> this thread got. ;-)

Easy to check as I use leafnode. :-)

# grep ^'Subject:' /var/spool/news/comp/unix/shell/*|grep 'sort by multiple columns'|wc -l
85

First post ...
Message-ID: <slrnu3v5vd.m2.t-usenet@ID-685.user.individual.de>
From: Martin =?UTF-8?Q?=CE=A4rautmann?= <t-usenet@gmx.net>
Newsgroups: comp.unix.shell
Subject: sort by multiple columns
Date: Wed, 19 Apr 2023 09:27:12 +0200

Regards, Dave Hodgins

On Tue, 09 May 2023 14:41:33 -0400, David W. Hodgins wrote:
> On Tue, 09 May 2023 14:06:27 -0400, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>
>> On 09.05.2023 19:41, Martin Τrautmann wrote:
>>> On Tue, 09 May 2023 10:54:09 -0400, Benjamin Esham wrote:
>>>> Ah, I think I understand. What you are calling the "first" sort is the
>>>> "innermost" sort, i.e., the sort that is applied *last*, and only if it is
>>>> necessary to break the tie between two rows that have been considered equal
>>>> by all of the previous sorting steps.
>>>
>>> Yeah, that's it. And the proper sort command had been given before:
>>>
>>> sort -t\; -k4,4n -k2,2 -k3,3
>>
>> In that light it's interesting how long (in time and number of posts)
>> this thread got. ;-)
>
> Easy to check as I use leafnode. :-)
>
> # grep ^'Subject:' /var/spool/news/comp/unix/shell/*|grep 'sort by multiple columns'|wc -l
> 85

But the result would not have been sorted properly yet. Maybe you could
use sort to sort the digits suitably first, then use awk to compute the
correct checksum?