In article <u23gql$3rkl5$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>On 23.04.2023 16:36, Kenny McCormack wrote:
>> [...]
>>
>> For most programs, this is rarely a concern, since most pipelines write and
>> read more or less simultaneously in real time, but sort is an edge case for
>> the reason you explain above.
>
>Note also that there are quite some sorting operations inherently
>used (e.g. in 'ls', in shells '*' glob/pattern expansion, etc.).
>For example, don't expect find | xargs ls to provide a sorted
>output.
>
>>
>> Something to keep in mind if you ever decide to sort very large files in a
>> pipeline. [...]
>
>In whatever way some instance of sort is implemented (memory, or
>temporary files, or whatever), my expectation is that
> whatever | sort
>will have to produce sorted output .- Isn't that guaranteed?

Actually, I may be wrong about this. May have posted too quickly.

The bad case would be if a program produced a ton of output, but the reader
didn't read any of it. I'll have to think some more as to whether or not
that applies here.

--
The randomly chosen signature file that would have appeared here is more than 4
lines long. As such, it violates one or more Usenet RFCs. In order to remain
in compliance with said RFCs, the actual sig can be found at the following URL:
http://user.xmission.com/~gazelle/Sigs/GodDelusion

On Sun, 23 Apr 2023 15:51:42 -0000 (UTC)
Spiros Bousbouras <spibou@gmail.com> wrote:
> I don't see the problem. If sort is on the left of a pipe then it will
> sort its whole input and then all it will do is write to the pipe. If sort
> is on the right of a pipe then in the beginning it will only do reading
> until it has read everything and then do the sorting. Obviously if you
> have process1 | process2 and one side does reading or writing (whatever
> applies) much slower than the other side then the fast side will block

To be precise , *may* block if the amount of data going through the pipe is
large enough.

> but there's nothing special with sort about that.

On 23.04.2023 17:51, Spiros Bousbouras wrote:
>
> [...] If sort
> is on the right of a pipe then in the beginning it will only do reading
> until it has read everything and then do the sorting. [...]

This is [in principle] not necessarily the case. The sort algorithm
can start to sort subsets of the stream to create runs of already
sorted sequences. Mergesort, for example, is a good candidate for
such a process; it can use (e.g.) Heapsort to create larger runs in
memory and then needs less merge-runs (which are typically costly
if that's done over files). How much data the Heapsort will process
may vary, but a size of magnitude of the pipe-buffer is reasonable.

Disclaimer: I don't know how Unix'es 'sort' is typically implemented,
but I expect some sophisticated implementation, since what I wrote
above is decades old knowledge (at least since the 1980's - when I
implemented some hybrid sorting algorithms -, or maybe even back to
Donald Knuth's work; but I don't recall whether it's covered in his
"Searching and Sorting" book).

Janis

On 23.04.2023 18:21, Felix Palmen wrote:
>
> This won't be a concern here. You need the whole data to sort something,
> so the sort utility must read until EOF anyways before doing its work.

See my recent reply on a different view.

> So, the real concern is whether you'll have enough RAM.

Not if sorting is (alternatively or also) done over files.

Even at times when 640k was considered immense memory by some, much
larger data sets had been sorted even then. (Speaking about real OS
computers, not about toys). In earth-bound computers there usually
was much more disk/drum/tape memory than kernel memory available.
Even if that's "legacy" the principles are still the same. - Unless
responsible folks start putting everything into a global memory
cloud. :-/

Janis

On Sun, 23 Apr 2023 12:21:44 -0400, Felix Palmen <felix@palmen-it.de> wrote:
> This won't be a concern here. You need the whole data to sort something,
> so the sort utility must read until EOF anyways before doing its work.
> So, the real concern is whether you'll have enough RAM.

Yes the sort has to read the entire input file before it can write anything
to the output file as the last record read may have to be the first one
written.

The way that's handled in low ram systems is to use temporary files, where it
sorts chunks into each temporary file and then merges the temporary files to
create the final output file.

By default the temporary files are stored in /tmp, which on most systems is
now a virtual file system kept in ram.

Either ensure /tmp is mounted on a disk files system with enough free space
or instruct sort to use another directory.

See "man sort" for the -T (aka --temporary-directory=DIR) option.

Regards, Dave Hodgins

On 23/04/2023 17:19, Janis Papanagnou wrote:
> On 23.04.2023 17:51, Spiros Bousbouras wrote:
>>
>> [...] If sort
>> is on the right of a pipe then in the beginning it will only do reading
>> until it has read everything and then do the sorting. [...]
>
> This is [in principle] not necessarily the case. The sort algorithm
> can start to sort subsets of the stream to create runs of already
> sorted sequences. Mergesort, for example, is a good candidate for
> such a process; it can use (e.g.) Heapsort to create larger runs in
> memory and then needs less merge-runs (which are typically costly
> if that's done over files). How much data the Heapsort will process
> may vary, but a size of magnitude of the pipe-buffer is reasonable.
>
> Disclaimer: I don't know how Unix'es 'sort' is typically implemented,
> but I expect some sophisticated implementation, since what I wrote
> above is decades old knowledge (at least since the 1980's - when I
> implemented some hybrid sorting algorithms -, or maybe even back to
> Donald Knuth's work; but I don't recall whether it's covered in his
> "Searching and Sorting" book).

My man page says:

--radixsort
Try to use radix sort, if the sort specifications allow.
The radix sort can only be used for trivial locales (C and
POSIX), and it cannot be used for numeric or month sort.
Radix sort is very fast and stable.

--mergesort
Use mergesort. This is a universal algorithm that can
always be used, but it is not always the fastest.

--qsort
Try to use quick sort, if the sort specifications allow.
This sort algorithm cannot be used with -u and -s.

--heapsort
Try to use heap sort, if the sort specifications allow.
This sort algorithm cannot be used with -u and -s.

* Janis Papanagnou <janis_papanagnou+ng@hotmail.com>:
> On 23.04.2023 18:21, Felix Palmen wrote:
>>
>> This won't be a concern here. You need the whole data to sort something,
>> so the sort utility must read until EOF anyways before doing its work.
>
> See my recent reply on a different view.

So, even if it starts working on "chunks", this won't change anything:
the data from the pipe must be read in order to work with it, so the
size of the pipe won't be a problem here.

It seems the idea assuming this was that the whole data to be sorted
must fit into the pipe buffer. But this isn't the case.

>> So, the real concern is whether you'll have enough RAM.
>
> Not if sorting is (alternatively or also) done over files.

Sure this *can* be done, that's why I mentioned the possibility. I
wasn't aware sort utils these days actually do it.

--
Dipl.-Inform. Felix Palmen <felix@palmen-it.de> ,.//..........
{web} http://palmen-it.de {jabber} [see email] ,//palmen-it.de
{pgp public key} http://palmen-it.de/pub.txt // """""""""""
{pgp fingerprint} 6936 13D5 5BBF 4837 B212 3ACC 54AD E006 9879 F231

On 23.04.2023 18:58, Felix Palmen wrote:
> * Janis Papanagnou <janis_papanagnou+ng@hotmail.com>:
>> On 23.04.2023 18:21, Felix Palmen wrote:
>>>
>>> This won't be a concern here. You need the whole data to sort something,
>>> so the sort utility must read until EOF anyways before doing its work.

s/doing/finishing/

>> See my recent reply on a different view.
>
> So, even if it starts working on "chunks", this won't change anything:
> the data from the pipe must be read in order to work with it, so the
> size of the pipe won't be a problem here.
>
> It seems the idea assuming this was that the whole data to be sorted
> must fit into the pipe buffer. But this isn't the case.

It boils down to this; sorting can _start_ sorting with fewer data
(something like a pipe-full), it can also _continue_ sorting with
more parts of data, and to _finish_ sorting it naturally must have
had all data available.

Janis

On Sun, 23 Apr 2023 15:42:00 -0400, John-Paul Stewart wrote:

> On 4/23/23 10:36, Kenny McCormack wrote:
>> This actually raises an interesting point. Pipes are not infinite in
>> size,
>> and they could, theoretically block if enough is written on the write
>> end without anything being read from the read end. Though the limits
>> are likely very large nowadays on modern systems, I think the original
>> implementation was only 4096 bytes and the standards today (POSIX) may
>> not guarantee anything more than that (haven't checked).
>
> FWIW, the pipe(7) manpage from Debian GNU/Linux has a "Pipe capacity"
> section that says in part:
>
> Before Linux 2.6.11, the capacity of a pipe was the same as the
> system page size (e.g., 4096 bytes on i386). Since Linux
> 2.6.11, the pipe capacity is 16 pages (i.e., 65,536 bytes in a
> system with a page size of 4096 bytes). Since Linux 2.6.35, the
> default pipe capacity is 16 pages, but the capacity can be queried
> and set using the fcntl(2) F_GETPIPE_SZ and F_SET‐
> PIPE_SZ operations. See fcntl(2) for more information.
>
> So pipes on Linux aren't very large at all. I don't know how other Unix
> systems compare.

Could the actual pipe size perhaps be queried
and set with "ulimit"?

$ ulimit -a
[...]
pipe size (512 bytes, -p) 8
[...]

With: GNU bash, version 5.1.16
("help ulimit" for docs on the shell built-in...)

--
-v (Scott)

* Janis Papanagnou <janis_papanagnou+ng@hotmail.com>:
> s/doing/finishing/

Agreed.

> It boils down to this; sorting can _start_ sorting with fewer data
> (something like a pipe-full), it can also _continue_ sorting with
> more parts of data, and to _finish_ sorting it naturally must have
> had all data available.

All correct, but I really doubt the relevance of the parantheses. The
size of the pipe will never be of much interest (except maybe for
performance), mostly because you can't seek a pipe anyways.

--
Dipl.-Inform. Felix Palmen <felix@palmen-it.de> ,.//..........
{web} http://palmen-it.de {jabber} [see email] ,//palmen-it.de
{pgp public key} http://palmen-it.de/pub.txt // """""""""""
{pgp fingerprint} 6936 13D5 5BBF 4837 B212 3ACC 54AD E006 9879 F231

On 24.04.2023 16:05, vallor wrote:
>
> Could the actual pipe size perhaps be queried
> and set with "ulimit"?
>
> $ ulimit -a
> [...]
> pipe size (512 bytes, -p) 8
> [...]
>
> With: GNU bash, version 5.1.16
> ("help ulimit" for docs on the shell built-in...)

It's quite funny that every shell has its own formats; in bash you
have to do the math (8x512) while in ksh it's 4096. Other quantities
have different scaling, e.g. bytes vs. Kibytes. And some have units
not defined (in ulimit or ulimit --man), like "blocks".

# bash
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
file size (blocks, -f) unlimited

# ksh
pipe buffer size (bytes) (-p) 4096
message queue size (Kibytes) (-q) 800
file size (blocks) (-f) unlimited

And zsh's ulimit "doesn't know" pipe size?

Janis

On 23.04.2023 19:29, Felix Palmen wrote:
> * Janis Papanagnou <janis_papanagnou+ng@hotmail.com>:
>> s/doing/finishing/
>
> Agreed.
>
>> It boils down to this; sorting can _start_ sorting with fewer data
>> (something like a pipe-full), it can also _continue_ sorting with
>> more parts of data, and to _finish_ sorting it naturally must have
>> had all data available.
>
> All correct, but I really doubt the relevance of the parantheses.

It's here just to demonstrate a magnitude, no less, no more.

But I seem to recall - faint memories from 4 decades ago - that
I/O-buffer size (similar to pipe-buffer size) was part of the
rationale about why to use such values and how to dimension it
(for optimum processing speed, yes, for performance as you say
below).

> The
> size of the pipe will never be of much interest (except maybe for
> performance), mostly because you can't seek a pipe anyways.

Seeking on the pipe isn't necessary since the pipe is just the
transfer medium, unstructured per se, with data likely even
truncated at the front or rear (because of octet-transmission,
not data-record processing). You'll anyway have it transferred
into a structured memory structure.

Janis

On 23.04.2023 13:28, Martin Τrautmann wrote:
> On Sun, 23 Apr 2023 03:33:47 +0200, Helmut Waitzmann wrote:
>>> If I want to pre-sort by 3 first, then sub-sort by column 2,
>>> that's fine. But when I pipe one sort to the other, the second
>>> sort will destroy the sort before. That's why i had my sort
>>> order in reverted order, using a pipe example.
>>
>> That won't help, either: A sorting pipe using (a standard)
>> "sort" won't solve the problem, because one cannot tell (a
>> standard) "sort" to do a sort on the given key option only. Each
>> sort in the pipe will be total (according to its sort criteria)
>> of its own.
>
> That was my problem - I expected that a pipe through several sorts would
> keep the order. I don't know why it doesn't.

Because sorting on one criterion generally doesn't impose any
restrictions on other criteria. By that sorting can be made a
very efficient implementation. But that's what stable sorting
is for; to make some provisions for specific ordering cases,
how to handle the set of records with equal keys. With Unix'es
'sort' implementation being and able to specify multiple keys
there's of course less need to separate sorting with pipes to
several distinct processes.

Janis

On 23.04.2023 19:46, David W. Hodgins wrote:
>
> If I'm reading it right, it always uses temporary files doing a sort/merge.
> Given that it started in 1988, it's not surprising that it's designed to
> work in a low ram environment.

Some test-run[*] finished here...

The data created and fed into 'sort' is larger than my free RAM.

$ time seq 1000000000 -1 1 | sort -n | N=1 is-sorted
0

real 58m8.18s
user 54m18.16s
sys 1m49.34s

Janis

[*] 'is-sorted' is an awk script, and "0" means it's okay (=sorted),

On 2023-04-23, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
> On Sun, 23 Apr 2023 15:42:00 -0400, John-Paul Stewart <jpstewart@personalprojects.net> wrote:
>> So pipes on Linux aren't very large at all. I don't know how other Unix
>> systems compare.
>
> The pipe only has to store a minimum of one buffer of data. If the process

In fact, I suspect, a pipe doesn't have to store anything. It can be a
pure rendezvous. The write() call can block until the reader performs a
read(), or vice versa, at which time MIN(read_size, write_size) bytes
can be transferred directly between their respective buffers, that value
then being returned from the read and write.

* Kaz Kylheku <864-117-4973@kylheku.com>:
> In fact, I suspect, a pipe doesn't have to store anything. It can be a
> pure rendezvous. The write() call can block until the reader performs a
> read(), or vice versa, at which time MIN(read_size, write_size) bytes
> can be transferred directly between their respective buffers, that value
> then being returned from the read and write.

Yes. IIRC, L4 uses some similar mechanism for IPC. It needs support from
the scheduler of course. And to make it most efficient, the size should
be agreed upon on both sides, so that won't work with typical pipe
semantics.

--
Dipl.-Inform. Felix Palmen <felix@palmen-it.de> ,.//..........
{web} http://palmen-it.de {jabber} [see email] ,//palmen-it.de
{pgp public key} http://palmen-it.de/pub.txt // """""""""""
{pgp fingerprint} 6936 13D5 5BBF 4837 B212 3ACC 54AD E006 9879 F231

Janis Papanagnou wrote:

> On 24.04.2023 16:05, vallor wrote:
>>
>> Could the actual pipe size perhaps be queried
>> and set with "ulimit"?
>>
>> $ ulimit -a
>> [...]
>> pipe size (512 bytes, -p) 8
>> [...]
>>
>> With: GNU bash, version 5.1.16
>> ("help ulimit" for docs on the shell built-in...)
>
> It's quite funny that every shell has its own formats; in bash you
> have to do the math (8x512) while in ksh it's 4096.

I believe the value ulimit is giving here is PIPE_BUF, not the
capacity of the pipe.

On my Linux system, much more than 4096 bytes can be written to
a pipe without anything being read from it:

$ dd if=/dev/zero | sleep 10
^C129+0 records in
128+0 records out
65536 bytes (66 kB, 64 KiB) copied, 2.04325 s, 32.1 kB/s

(I used Ctrl-C to send dd a SIGINT.)

$ ulimit -a | grep pipe
pipe size (512 bytes, -p) 8
$ getconf PIPE_BUF .
4096

In any case, on some systems "pipe capacity" is not a simple concept.
SVR4's STREAMS-based pipes have separate high-water and low-water
thresholds. (The writer blocks when high-water is reached but
doesn't unblock until enough has been read to take the level below
low-water.)

--
Geoff Clare <netnews@gclare.org.uk>

In article <6tukhj-bl1.ln1@ID-313840.user.individual.net>,
Geoff Clare <netnews@gclare.org.uk> wrote:
....
>On my Linux system, much more than 4096 bytes can be written to
>a pipe without anything being read from it:
>
>$ dd if=/dev/zero | sleep 10
>^C129+0 records in
>128+0 records out
>65536 bytes (66 kB, 64 KiB) copied, 2.04325 s, 32.1 kB/s
>
>(I used Ctrl-C to send dd a SIGINT.)

Didn't somebody say upthread that the default limit on Linux is 64K?
So, kinda funny that you chose exactly 64K for your demonstration.

Anyway, you can (according to those same people) bump it up to 1M. if
needed.

--
People who want to share their religious views with you
almost never want you to share yours with them. -- Dave Barry

On Tue, 25 Apr 2023 09:29:48 -0400, Kenny McCormack <gazelle@shell.xmission.com> wrote:

> In article <6tukhj-bl1.ln1@ID-313840.user.individual.net>,
> Geoff Clare <netnews@gclare.org.uk> wrote:
> ...
>> On my Linux system, much more than 4096 bytes can be written to
>> a pipe without anything being read from it:
>>
>> $ dd if=/dev/zero | sleep 10
>> ^C129+0 records in
>> 128+0 records out
>> 65536 bytes (66 kB, 64 KiB) copied, 2.04325 s, 32.1 kB/s
>>
>> (I used Ctrl-C to send dd a SIGINT.)
>
> Didn't somebody say upthread that the default limit on Linux is 64K?
> So, kinda funny that you chose exactly 64K for your demonstration.
>
> Anyway, you can (according to those same people) bump it up to 1M. if
> needed.

It stopped after filling the output buffer, not the pipe. That data was still
waiting to be written to the pipe when the dd command was terminated.

Regards, Dave Hodgins

David W. Hodgins wrote:

> On Tue, 25 Apr 2023 09:29:48 -0400, Kenny McCormack <gazelle@shell.xmission.com> wrote:
>
>> In article <6tukhj-bl1.ln1@ID-313840.user.individual.net>,
>> Geoff Clare <netnews@gclare.org.uk> wrote:
>> ...
>>> On my Linux system, much more than 4096 bytes can be written to
>>> a pipe without anything being read from it:
>>>
>>> $ dd if=/dev/zero | sleep 10
>>> ^C129+0 records in
>>> 128+0 records out
>>> 65536 bytes (66 kB, 64 KiB) copied, 2.04325 s, 32.1 kB/s
>>>
>>> (I used Ctrl-C to send dd a SIGINT.)
>>
>> Didn't somebody say upthread that the default limit on Linux is 64K?
>> So, kinda funny that you chose exactly 64K for your demonstration.

I didn't actively choose 64K. I haven't ever changed the pipe size on
a Linux system, so the size used was whatever is the default.

>> Anyway, you can (according to those same people) bump it up to 1M. if
>> needed.
>
> It stopped after filling the output buffer, not the pipe. That data was still
> waiting to be written to the pipe when the dd command was terminated.

Only one block (of 512 bytes) was waiting to be written. A feature
of dd is that it reads and writes exactly the block sizes you tell it
to (or 512 bytes by default). The dd output:

128+0 records out

means it had successfully written 128 blocks (of 512 bytes) to the pipe
when it exited. The "129+0 records in" is what shows it had read one
extra block that was waiting to be written.

If I tell dd to read and write one byte at a time, it does exactly that:

$ dd bs=1 if=/dev/zero | sleep 10
^C65537+0 records in
65536+0 records out
65536 bytes (66 kB, 64 KiB) copied, 4.88295 s, 13.4 kB/s

--
Geoff Clare <netnews@gclare.org.uk>

with <u265ou$csqa$1@dont-email.me> Janis Papanagnou wrote:
> On 24.04.2023 16:05, vallor wrote:

>> Could the actual pipe size perhaps be queried and set with "ulimit"?
*SKIP*
> And zsh's ulimit "doesn't know" pipe size?

Funny thing, looking through /usr/include/**/resource.h suggests that size
of pipe has nothing to do with setrlimit(2) or ulimit(3). Weird.

--
Torvalds' goal for Linux is very simple: World Domination
Stallman's goal for GNU is even simpler: Freedom

On Sun, 23 Apr 2023 09:43:06 -0400, David W. Hodgins wrote:
> On Sun, 23 Apr 2023 07:28:22 -0400, Martin Τrautmann <t-usenet@gmx.net> wrote:
>> That was my problem - I expected that a pipe through several sorts would
>> keep the order. I don't know why it doesn't.
>
> It may be easier to understand if you use a temporary files instead of pipes.
>
> Sorting the input file by column 4, numerical creating a first temporary file.
> Sort the first temporary file by column 2 creating a second temporary file.
> Sort the second temporary file by column 3 creating the output.
>
> The last sort doesn't know that the prior two sorts have been done. It just
> looks at the file it's giving and sorts it by column 3.
>
> Using a pipe just takes the output of the first and second sort and uses it
> directly as input for the next sort. All the pipe does is eliminate the
> need for a temporary file.

But if I sort by one column only, then through the pipe by another
column only, the second sort SHOULD respect the previous sort.
Unfortunately, I feel it doesn't.

> Keep in mind. When sorting a file, the last line in the input may end up becoming
> the first line in the output. The sort can not write anything to the pipe or
> output file until it's sorted the entire input. With a pipe, the temporary
> file is in ram rather then being a named file on disk.

So the sort via a file actually should work the same as via the pipe?

On Sun, 23 Apr 2023 22:30:24 +0200, Helmut Waitzmann wrote:
> Helmut Waitzmann <nn.throttle@xoxy.net>:
>> Look at these sample lines:
>>
>>
>> 1;0
>> 1;1
>> 1;2
>> 0;0
>> 0;1
>> 0;2
>> 2;0
>> 2;1
>> 2;2
>>
>>
>> To have this sequence of lines sorted in such a way that the
>> first field is sorted in ascending numeric order while the
>> second is sorted in descending numeric order,
>
> I'm sorry, that is a quite misleading description.  What I wanted
> to say is that the sequence of lines should be sorted to look
> like
>
> 0;2
> 0;1
> 0;0
> 1;2
> 1;1
> 1;0
> 2;2
> 2;1
> 2;0
>
> and to achieve this…
>
>> one could specify the two sort criteria at once:
>>
>> sort -t ';' -k 1nb,1 -k 2nr,2

Would you achieve this via a pipe as well?

On Sat, 29 Apr 2023 12:05:17 +0200, Martin Τrautmann wrote:
> On Sun, 23 Apr 2023 22:30:24 +0200, Helmut Waitzmann wrote:
>> Helmut Waitzmann <nn.throttle@xoxy.net>:
>>> Look at these sample lines:
>>>
>>>
>>> 1;0
>>> 1;1
>>> 1;2
>>> 0;0
>>> 0;1
>>> 0;2
>>> 2;0
>>> 2;1
>>> 2;2
>>>
>>>
>>> To have this sequence of lines sorted in such a way that the
>>> first field is sorted in ascending numeric order while the
>>> second is sorted in descending numeric order,
>>
>> I'm sorry, that is a quite misleading description.  What I wanted
>> to say is that the sequence of lines should be sorted to look
>> like
>>
>> 0;2
>> 0;1
>> 0;0
>> 1;2
>> 1;1
>> 1;0
>> 2;2
>> 2;1
>> 2;0
>>
>> and to achieve this…
>>
>>> one could specify the two sort criteria at once:
>>>
>>> sort -t ';' -k 1nb,1 -k 2nr,2
>
> Would you achieve this via a pipe as well?

When I sort by column 2 first and only, I end up with
0;2
1;2
2;2
0;1
1;1
2;1
0;0
1;0
2;0

Why that? I would expect
1;2
0;2
2;2
1;1
0;1
2;1
1;0
0;0
2;0

So why does it resort by first column as well? Since it does that, both
a pipe and a second sort from a temporary file still fail, since they
also ignore the temporary sort of the other column.

On 29/04/2023 11:01, Martin Τrautmann wrote:
> On Sun, 23 Apr 2023 09:43:06 -0400, David W. Hodgins wrote:
>> On Sun, 23 Apr 2023 07:28:22 -0400, Martin Τrautmann <t-usenet@gmx.net> wrote:
>>> That was my problem - I expected that a pipe through several sorts would
>>> keep the order. I don't know why it doesn't.
>>
>> It may be easier to understand if you use a temporary files instead of pipes.
>>
>> Sorting the input file by column 4, numerical creating a first temporary file.
>> Sort the first temporary file by column 2 creating a second temporary file.
>> Sort the second temporary file by column 3 creating the output.
>>
>> The last sort doesn't know that the prior two sorts have been done. It just
>> looks at the file it's giving and sorts it by column 3.
>>
>> Using a pipe just takes the output of the first and second sort and uses it
>> directly as input for the next sort. All the pipe does is eliminate the
>> need for a temporary file.
>
> But if I sort by one column only, then through the pipe by another
> column only, the second sort SHOULD respect the previous sort.
> Unfortunately, I feel it doesn't.

Of course it doesn't. How does the second sort know that the first sort
even happened?

>
>> Keep in mind. When sorting a file, the last line in the input may end up becoming
>> the first line in the output. The sort can not write anything to the pipe or
>> output file until it's sorted the entire input. With a pipe, the temporary
>> file is in ram rather then being a named file on disk.
>
> So the sort via a file actually should work the same as via the pipe?
>

--
Chris Elvidge
England