On 2023-10-03, Richard Kettlewell wrote:

> Ed Morton <mortonspam@gmail.com> writes:
>> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>>> Ed Morton <mortonspam@gmail.com> writes:
>>>> So I don't see any of that as justifying adding a bunch of options to
>>>> grep to do something other than it's primary purpose of g/re/p and
>>>> making it different from all other text processing tools in that
>>>> regard.
>>> Why do you think it needs justifying? It’s their code, they can add
>>> whatever they like to it.
>>
>> The GNU people added functionality to grep in a way that is
>> contradictory to the Unix philosophy and makes it inconsistent with
>> all other text processing tools.
>
> Why does that matter?
>
>> So far the only suggestion I've heard for why they did that is so
>> people could do:
>>
>> grep -r 'regexp'
>>
>> instead of:
>>
>> find . -type f -exec grep -H 'regexp' {} +
>>
>> which saves us about 20 simple, common characters over find+grep but
>> does nothing for find+awk, find+sed, etc.
>
> Sounds good to me. Less typing = better. Given the uptake of the GNU
> tools I suspect my opinion is widely shared.

I have to admit that I have been using grep -lr foo/ for so long that
it would never occur to me to try to wrangle find's arguments into
place for the purpose. The philosophical argument against grep -l is
understandable but for my own practical purposes I am grateful for it.

--
....the reason why so many professional artists drink a lot is not
necessarily very much to do with the artistic temperament, etc. It is
simply that they can afford to, because they can normally take a large
part of a day off to deal with the ravages. ---Amis _On Drink_

On 06.10.2023 16:16, Andy Walker wrote:
>
> Well, that's the trouble with feeping creaturism. Once the
> creature has fept, it's impossible to reconsider it; there are always
> some people using it who mustn't be upset. The time to decide is
> /before/ adding it. [...]

Yes.

> [ disproportionally growing sizes of man-pages (since V7) ]

In our (post-V7, post-SysV) Linux era we now find also man pages
of smaller size - carrying only a hint to an info-page hierarchy.

Yes, it can get even worse.

Janis

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> Geoff Clare <geoff@clare.See-My-Signature.invalid> writes:
>> Ben Bacarisse wrote:
>>> But find is the best we have, and it's a mess. It uses mandatory syntax
>>> that needs to be quoted (the {})
>>
>> The {} does not need to be quoted. That's because { is a reserved
>> word in the shell, not an operator like (.
>
> Is this true of all shells? I was going by the find man page. Maybe
> it needs an update?

`echo {}` prints "{}" in every shell I've tried (bash, csh, tcsh, ksh,
zsh, fish, dash, busybox sh).

[...]

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:

> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>> Geoff Clare <geoff@clare.See-My-Signature.invalid> writes:
>>> Ben Bacarisse wrote:
>>>> But find is the best we have, and it's a mess. It uses mandatory syntax
>>>> that needs to be quoted (the {})
>>>
>>> The {} does not need to be quoted. That's because { is a reserved
>>> word in the shell, not an operator like (.
>>
>> Is this true of all shells? I was going by the find man page. Maybe
>> it needs an update?
>
> `echo {}` prints "{}" in every shell I've tried (bash, csh, tcsh, ksh,
> zsh, fish, dash, busybox sh).

The find man page repeats that {} "might" have to be escaped or quoted
several times. The EXAMPLES sections quotes it, and then some text goes
on to explain why!

I accept that it's wrong to say that.

So I'll change my remark back to the one I had intended: find uses
syntax that needs to be quoted (the ;). Annoyingly, I'd flipped from ;
to {} because I didn't think that ; was used in the quoted text!

--
Ben.

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>
>> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>>> Geoff Clare <geoff@clare.See-My-Signature.invalid> writes:
>>>> Ben Bacarisse wrote:
>>>>> But find is the best we have, and it's a mess. It uses mandatory syntax
>>>>> that needs to be quoted (the {})
>>>>
>>>> The {} does not need to be quoted. That's because { is a reserved
>>>> word in the shell, not an operator like (.
>>>
>>> Is this true of all shells? I was going by the find man page. Maybe
>>> it needs an update?
>>
>> `echo {}` prints "{}" in every shell I've tried (bash, csh, tcsh, ksh,
>> zsh, fish, dash, busybox sh).
>
> The find man page repeats that {} "might" have to be escaped or quoted
> several times. The EXAMPLES sections quotes it, and then some text goes
> on to explain why!
>
> I accept that it's wrong to say that.
>
> So I'll change my remark back to the one I had intended: find uses
> syntax that needs to be quoted (the ;). Annoyingly, I'd flipped from ;
> to {} because I didn't think that ; was used in the quoted text!

The find(1) man page (for GNU findutils 4.8.0 and in the latest version
in the git repo) says in one place (under "-exec command ;") that {} and ;
*might* need to be escaped, and in another (under "-exec command {}+")
that {} *needs* to be escaped.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

On 2023-10-06, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
> On Fri, 06 Oct 2023 13:28:43 -0400, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>> `echo {}` prints "{}" in every shell I've tried (bash, csh, tcsh, ksh,
>> zsh, fish, dash, busybox sh).
>
> From "man bash" ...
> { and } are reserved words and must occur where a reserved word is permitted to be recognized.
>
> So if they occur somewhere where a reserved word is not permitted, then they are
> just normal characters.

Same as:

echo while

etc. while is a reserved word but not everywhere.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 10/5/2023 9:25 PM, Ben Bacarisse wrote:
> Ed Morton <mortonspam@gmail.com> writes:
>
>> On 10/5/2023 2:37 PM, Ben Bacarisse wrote:
>>> Ed Morton <mortonspam@gmail.com> writes:
>>>
>>>> On 10/3/2023 2:00 AM, Richard Kettlewell wrote:
>>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>>> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>>>>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>>>>> So I don't see any of that as justifying adding a bunch of options to
>>>>>>>> grep to do something other than it's primary purpose of g/re/p and
>>>>>>>> making it different from all other text processing tools in that
>>>>>>>> regard.
>>>>>>> Why do you think it needs justifying? It’s their code, they can add
>>>>>>> whatever they like to it.
>>>>>>
>>>>>> The GNU people added functionality to grep in a way that is
>>>>>> contradictory to the Unix philosophy and makes it inconsistent with
>>>>>> all other text processing tools.
>>>>> Why does that matter?
>>>>
>>>> For the same reasons that cohesion and consistency matter in any
>>>> system. Google "software" with those 2 terms to find more information on
>>>> how those concepts apply to software.
>>> Those are not the only concepts that matter.
>>
>> Of course, they're just the answer to the question Richard asked.
>
> They are /your/ answer to his question -- these are the concepts that
> make it matter to you. Other concepts (like ease of use) don't
> necessarily give the same answer which is why I mention them.

I said "The GNU people added functionality to grep in a way that is
contradictory to the Unix philosophy and makes it inconsistent with all
other text processing tools." and in response Richard asked "Why does
that matter?"

I don't see how "ease of use" is an answer to Richard's question.

/My/
> answer would be "It doesn't matter much -- ease of use is too important
> in this case".

I'm claiming that X matters and being asked why, so responding that it
doesn't matter wouldn't make sense or be useful.

>
> (I hope you are not being hyper literal here. My "answer" is not an
> answer in the literal sense. If I took "Why does that matter?"
> absolutely literally, I'd have to give your answer as well with maybe
> just "but only a teeny, tiny bit" added. But ripostes like "Why does
> that matter?" are almost always rhetorical, and not literal requests for
> the reasons, no matter how insignificant the respondent might consider
> them to be.)
>
>>>> If Unix were being invented today and the design decision being presented
>>>> in an attempt to answer the question "how do we call a text processing tool
>>>> on every file found under a directory?" was:
>>>>
>>>> IF (tool == grep) && (provider == GNU) THEN
>>>> tool -r '...'
>>>> ELSE
>>>> find . -type f tool '...' {} +
>>>> ENDIF
>>>>
>>>> instead of just:
>>>>
>>>> find . -type f tool '...' {} +
>>> find . -type f -exec tool '...' '{}' +
>>
>> Right, thanks. I typed my response once and my newsreader discarded it so I
>> had to type it again and rushed it.
>>
>>>> the person suggesting it would be ridiculed and sent back to the drawing
>>>> board.
>>> It's not "just" find ... find is a mess. If the file system, the shell
>>> and all the rest were such that adding some command or prefix like
>>> rec grep pattern .
>>> made grep behave pretty much like grep -r then no one would bother to
>>> add options like -r to grep.
>>
>> Agreed, that is the right solution if people are unhappy with `find` and
>> apparently there is an existing tool to do just that, see Janis's posts in
>> this thread.
>
> tw may be better than find, but that's a low bar. My "rec" was intended
> to be hypothetical. I should maybe have said that if it were possible
> to have a rec prefix that magically just "did the right thing" then -r
> would not be needed.
>
> But your remark raises an interesting question. Are you happy with
> find? I use it quite a lot, but I am unhappy with it pretty much every
> time!
>
>>> But find is the best we have, and it's a mess. It uses mandatory syntax
>>> that needs to be quoted (the {}) and does not always preserve the desred
>>> behaviour. For example
>>> grep -rq needle haystack
>>> can succeed (exit 0) where
>>> find haystack -type f -exec grep -q needle '{}' +
>>> can fail (exit non-zero). This is because find can run multiple
>>> commands and does not always combine the exit statuses correctly. I
>>> would be happy to be wrong about this, but I'm sure I tested it some
>>> time ago.
>>
>> Interesting - that's not something I've ever come across but thanks for
>> bringing it up as a possible concern.
>
> It's not a possible concern, it's an actual concern!

It's only an actual concern if it actually happens and you're saying
above that you're not sure if it happens or not, it's just something you
think you saw some time in the past ("I would be happy to be wrong about
this, but I'm sure I tested it some time ago."), hence my saying it's a
possible concern.

If you want to
> know if "pattern" occurs in any file from . down, you /can't/ use
>
> find . -type f -exec grep -q pattern '{}' +
>
> because you will get the wrong answer in some cases.

OK, then don't do that, do it one of the various other simple ways it
can be done, e.g.:

find . -type f -exec grep -m1 pattern '{}' +

and test for any output, but I'm not arguing that `find` is the best
possible answer to the issues, I'm saying that adding `-r` to grep is
the wrong answer for the reasons I've already stated.

>
> But consider the general case... Imagine a collection of tools to test
> if a file has this or that property. The tools can all have multiple
> arguments but what do they do with conflicting answers? For some
> properties it might be useful for the tool to report success (0) if
> /all/ the arguments have that property and for other properties it might
> be desirable to report success if /any/ of the arguments have it. If
> find splits an argument list into two and the tool reports 0 on one half
> and 1 on the second, what should find report? If the tool is an "all"
> tool it should report 1, and if the tool is an "any" tool it should
> report 0.

How does adding "-r" to grep solve that problem for awk and sed?

Ed.

On 10/5/2023 9:25 PM, Ben Bacarisse wrote:
<snip>

I forgot to answer your question below in my preceding post, sorry:

> But your remark raises an interesting question. Are you happy with
> find? I use it quite a lot, but I am unhappy with it pretty much every
> time!

I find it's arguments clunky and arcane and, after 40+ years of using
it, any time I want to do more than `find . -type f \( -name X -o name Y
\) -exec foo {} +` I still have to look up the man page BUT I rarely
have to do more than that with it so that doesn't overly bother me.

That doesn't mean that giving grep (and only grep among all the text
processing tools) the ability to find files was the best possible
solution to that problem, if it needs to be solved.

Ed.

On 10/3/2023 2:26 AM, Kaz Kylheku wrote:
> On 2023-10-02, Ed Morton <mortonspam@gmail.com> wrote:
>> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>>> Ed Morton <mortonspam@gmail.com> writes:
>>>> So I don't see any of that as justifying adding a bunch of options to
>>>> grep to do something other than it's primary purpose of g/re/p and
>>>> making it different from all other text processing tools in that
>>>> regard.
>>>
>>> Why do you think it needs justifying? It’s their code, they can add
>>> whatever they like to it.
>>>
>>
>> The GNU people added functionality to grep in a way that is
>> contradictory to the Unix philosophy and makes it inconsistent with all
>> other text processing tools.
>
> - GNU stands for GNU is Not Unix.
>
> - The GNU coding standards document says:
>
> The GNU Project regards standards published by other organizations
> as suggestions, not orders. We consider those standards, but we do
> not “obey” them. In developing a GNU program, you should implement
> an outside standard’s specifications when that makes the GNU system
> better overall in an objective sense. When it doesn’t, you shouldn’t.

To be clear, I'm not talking about "grep -r" violating any standards,
I'm talking about it violating the Unix philosophy and software
engineering fundamentals of cohesion and consistency.

>
> - Unix programs have recursion built in: from memory I can think of
> ls, rm, cp, mv, chgrp, chmod and chown.

Those all operate on file attributes, not on the contents of files as
text processing tools like grep do. Tools like `rm` or `chmod` being
able to find and perform actions on files is equivalent to text
processing tools like `grep` or `awk` being able to find and perform
actions on text inside files.

>
> - BSD Unixes have a -R option on grep; some, like FreeBSD, have an -r
> option as well as a rgrep utility.

Yes, GNUisms are apparently creeping into BSD tools, usually for the
better though.

Ed.

On 2023-10-07, Ed Morton <mortonspam@gmail.com> wrote:
> To be clear, I'm not talking about "grep -r" violating any standards,
> I'm talking about it violating the Unix philosophy and software
> engineering fundamentals of cohesion and consistency.

Unix doesn't withstand a moment's scrutiny in the light of any of
these lofty words.

Ed Morton <mortonspam@gmail.com> writes:

> On 10/5/2023 9:25 PM, Ben Bacarisse wrote:
>> Ed Morton <mortonspam@gmail.com> writes:
>>
>>> On 10/5/2023 2:37 PM, Ben Bacarisse wrote:
>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>
>>>>> On 10/3/2023 2:00 AM, Richard Kettlewell wrote:
>>>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>>>> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>>>>>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>>>>>> So I don't see any of that as justifying adding a bunch of options to
>>>>>>>>> grep to do something other than it's primary purpose of g/re/p and
>>>>>>>>> making it different from all other text processing tools in that
>>>>>>>>> regard.
>>>>>>>> Why do you think it needs justifying? It’s their code, they can add
>>>>>>>> whatever they like to it.
>>>>>>>
>>>>>>> The GNU people added functionality to grep in a way that is
>>>>>>> contradictory to the Unix philosophy and makes it inconsistent with
>>>>>>> all other text processing tools.
>>>>>> Why does that matter?
>>>>>
>>>>> For the same reasons that cohesion and consistency matter in any
>>>>> system. Google "software" with those 2 terms to find more information on
>>>>> how those concepts apply to software.
>>>> Those are not the only concepts that matter.
>>>
>>> Of course, they're just the answer to the question Richard asked.
>> They are /your/ answer to his question -- these are the concepts that
>> make it matter to you. Other concepts (like ease of use) don't
>> necessarily give the same answer which is why I mention them.
>
> I said "The GNU people added functionality to grep in a way that is
> contradictory to the Unix philosophy and makes it inconsistent with all
> other text processing tools." and in response Richard asked "Why does that
> matter?"
>
> I don't see how "ease of use" is an answer to Richard's question.
>
>> /My/
>> answer would be "It doesn't matter much -- ease of use is too important
>> in this case".
>
> I'm claiming that X matters and being asked why, so responding that it
> doesn't matter wouldn't make sense or be useful.

Yes, I know. I thought that reading was "hyper literal" as explained
here:

>> (I hope you are not being hyper literal here. My "answer" is not an
>> answer in the literal sense. If I took "Why does that matter?"
>> absolutely literally, I'd have to give your answer as well with maybe
>> just "but only a teeny, tiny bit" added. But ripostes like "Why does
>> that matter?" are almost always rhetorical, and not literal requests for
>> the reasons, no matter how insignificant the respondent might consider
>> them to be.)

I think Richard's question was largely rhetorical. But maybe I'm
wrong. Maybe he did not know what inconsistency matters, and he's
learned from your reply.

>>>> But find is the best we have, and it's a mess. It uses mandatory syntax
>>>> that needs to be quoted (the {}) and does not always preserve the desred
>>>> behaviour. For example
>>>> grep -rq needle haystack
>>>> can succeed (exit 0) where
>>>> find haystack -type f -exec grep -q needle '{}' +
>>>> can fail (exit non-zero). This is because find can run multiple
>>>> commands and does not always combine the exit statuses correctly. I
>>>> would be happy to be wrong about this, but I'm sure I tested it some
>>>> time ago.
>>>
>>> Interesting - that's not something I've ever come across but thanks for
>>> bringing it up as a possible concern.
>> It's not a possible concern, it's an actual concern!
>
> It's only an actual concern if it actually happens and you're saying above
> that you're not sure if it happens or not, it's just something you think
> you saw some time in the past ("I would be happy to be wrong about this,
> but I'm sure I tested it some time ago."), hence my saying it's a possible
> concern.

Sure, but I went on to explain that it can't get it right in all cases
even if it happened to get this case right. "It" (relying on find to
combine exit statuses) really is an actual concern.

>> If you want to
>> know if "pattern" occurs in any file from . down, you /can't/ use
>> find . -type f -exec grep -q pattern '{}' +
>> because you will get the wrong answer in some cases.
>
> OK, then don't do that, do it one of the various other simple ways it can
> be done, e.g.:
>
> find . -type f -exec grep -m1 pattern '{}' +
>
> and test for any output, but I'm not arguing that `find` is the best
> possible answer to the issues, I'm saying that adding `-r` to grep is the
> wrong answer for the reasons I've already stated.

Yes, we all know how to do it other ways (and I like grep -r as the way
to do it). My point was that you exaggerated the simplicity of just
using find as an alternative. The consistent solution (no tool to
provide a -r option because that's a job for find) imposes a significant
cognitive load: remember change the command when you want to rely on the
(possibly)combined exit status.

The first time I encountered this problem was in a script that just
stopped working one day. The set of files had got large enough that
find has split them into two executions of grep. Now you can argue that
I should have been more careful. It's obvious (once you think about it)
that find can't get the right answer in all cases, but it's still part
of what makes find complicated.

>> But consider the general case... Imagine a collection of tools to test
>> if a file has this or that property. The tools can all have multiple
>> arguments but what do they do with conflicting answers? For some
>> properties it might be useful for the tool to report success (0) if
>> /all/ the arguments have that property and for other properties it might
>> be desirable to report success if /any/ of the arguments have it. If
>> find splits an argument list into two and the tool reports 0 on one half
>> and 1 on the second, what should find report? If the tool is an "all"
>> tool it should report 1, and if the tool is an "any" tool it should
>> report 0.
>
> How does adding "-r" to grep solve that problem for awk and sed?

Eh?

--
Ben.

On 10/7/2023 5:17 PM, Ben Bacarisse wrote:
> Ed Morton <mortonspam@gmail.com> writes:
>
>> On 10/5/2023 9:25 PM, Ben Bacarisse wrote:
<snip>
>>> But consider the general case... Imagine a collection of tools to test
>>> if a file has this or that property. The tools can all have multiple
>>> arguments but what do they do with conflicting answers? For some
>>> properties it might be useful for the tool to report success (0) if
>>> /all/ the arguments have that property and for other properties it might
>>> be desirable to report success if /any/ of the arguments have it. If
>>> find splits an argument list into two and the tool reports 0 on one half
>>> and 1 on the second, what should find report? If the tool is an "all"
>>> tool it should report 1, and if the tool is an "any" tool it should
>>> report 0.
>>
>> How does adding "-r" to grep solve that problem for awk and sed?
>
> Eh?
>

The problem you describe above isn't unique to grep, it also exists for
sed, awk and other tools. Giving GNU grep a "-r" option to solve the
problem for grep does nothing to solve the problem for any other tool,
but there are alternative solutions that could be implemented to solve
it for all similar tools as discussed elsethread.

Ed.

Ed Morton <mortonspam@gmail.com> writes:

> On 10/7/2023 5:17 PM, Ben Bacarisse wrote:
>> Ed Morton <mortonspam@gmail.com> writes:
>>
>>> On 10/5/2023 9:25 PM, Ben Bacarisse wrote:
> <snip>
>>>> But consider the general case... Imagine a collection of tools to test
>>>> if a file has this or that property. The tools can all have multiple
>>>> arguments but what do they do with conflicting answers? For some
>>>> properties it might be useful for the tool to report success (0) if
>>>> /all/ the arguments have that property and for other properties it might
>>>> be desirable to report success if /any/ of the arguments have it. If
>>>> find splits an argument list into two and the tool reports 0 on one half
>>>> and 1 on the second, what should find report? If the tool is an "all"
>>>> tool it should report 1, and if the tool is an "any" tool it should
>>>> report 0.
>>>
>>> How does adding "-r" to grep solve that problem for awk and sed?
>> Eh?
>>
>
> The problem you describe above isn't unique to grep, it also exists for
> sed, awk and other tools.

Yes. The problem is even described in a tool-agnostic way -- no mention
of grep at all!

> Giving GNU grep a "-r" option to solve the
> problem for grep does nothing to solve the problem for any other tool, but
> there are alternative solutions that could be implemented to solve it for
> all similar tools as discussed elsethread.

Two points. I don't thing there are alternative solutions, but if so,
can you point me to them as I've lost track? I want to find one (since
I've been bitten by this problem) so I would like to check them out.

Secondly, I am pretty sure that any near solution to this problem will
come with complications. Your original complaint was based on adding -r
to grep (a very common use case) when find is the consistent option.
But find's behaviour is /not/ consistent with what grep does as I have
detailed above. I would go so far as to argue that a solution that is
apparently coherent and consistent, right up until it breaks because of
something arbitrary like when find breaks it's argument list is
borderline dangerous. At the very least, it requires more careful
thought than remembering which tools have -r.

A good, consistent, solution would be allow an argument list to be
supplied by a generator. The program could get to work as soon as it
had a file to process, but there would be no limit on the number of
files. But this is not possible with the Unix model of what constitutes
a command's arguments.

--
Ben.

On 10/8/2023 8:43 AM, Ben Bacarisse wrote:
> Ed Morton <mortonspam@gmail.com> writes:
>
>> On 10/7/2023 5:17 PM, Ben Bacarisse wrote:
>>> Ed Morton <mortonspam@gmail.com> writes:
>>>
>>>> On 10/5/2023 9:25 PM, Ben Bacarisse wrote:
>> <snip>
>>>>> But consider the general case... Imagine a collection of tools to test
>>>>> if a file has this or that property. The tools can all have multiple
>>>>> arguments but what do they do with conflicting answers? For some
>>>>> properties it might be useful for the tool to report success (0) if
>>>>> /all/ the arguments have that property and for other properties it might
>>>>> be desirable to report success if /any/ of the arguments have it. If
>>>>> find splits an argument list into two and the tool reports 0 on one half
>>>>> and 1 on the second, what should find report? If the tool is an "all"
>>>>> tool it should report 1, and if the tool is an "any" tool it should
>>>>> report 0.
>>>>
>>>> How does adding "-r" to grep solve that problem for awk and sed?
>>> Eh?
>>>
>>
>> The problem you describe above isn't unique to grep, it also exists for
>> sed, awk and other tools.
>
> Yes. The problem is even described in a tool-agnostic way -- no mention
> of grep at all!
>
>> Giving GNU grep a "-r" option to solve the
>> problem for grep does nothing to solve the problem for any other tool, but
>> there are alternative solutions that could be implemented to solve it for
>> all similar tools as discussed elsethread.
>
> Two points. I don't thing there are alternative solutions, but if so,
> can you point me to them as I've lost track? I want to find one (since
> I've been bitten by this problem) so I would like to check them out.

A tool such as the tree-walk "tw"
(https://github.com/att/ast/tree/master/src/cmd/tw) tool that apparently
can find files and call a tool on those found files as suggested by me
and others in this thread. I'm not saying that THAT tool is the
solution, though, (I'd never heard of it till it was mentioned in this
thread and I don't know any more about it) I'm saying a solution for all
tools could have been created instead of modifying grep as was done.

>
> Secondly, I am pretty sure that any near solution to this problem will
> come with complications. Your original complaint was based on adding -r
> to grep (a very common use case) when find is the consistent option.

My complaint is that adding -r to grep makes it inconsistent with and
doesn't solve the issues for any of the other similar tools.

> But find's behaviour is /not/ consistent with what grep does as I have
> detailed above. I would go so far as to argue that a solution that is
> apparently coherent and consistent, right up until it breaks because of
> something arbitrary like when find breaks it's argument list is
> borderline dangerous. At the very least, it requires more careful
> thought than remembering which tools have -r.

I'm not saying that "find" solves all of the issues, I'm saying that
adding "-r" to grep was the wrong thing to do as it doesn't solve all
the issues and introduces other issues.

Ed.

> A good, consistent, solution would be allow an argument list to be
> supplied by a generator. The program could get to work as soon as it
> had a file to process, but there would be no limit on the number of
> files. But this is not possible with the Unix model of what constitutes
> a command's arguments.
>

On 09.10.2023 15:18, Ed Morton wrote:
> On 10/8/2023 8:43 AM, Ben Bacarisse wrote:
>>
>> Secondly, I am pretty sure that any near solution to this problem will
>> come with complications. Your original complaint was based on adding -r
>> to grep (a very common use case) when find is the consistent option.
>
> My complaint is that adding -r to grep makes it inconsistent with and
> doesn't solve the issues for any of the other similar tools.

I think there's even a bigger issue. It's usually not sufficient to add
only a -r option, more sooner than later you want to control subsets of
the directory tree, thus have to add yet more options - to *every* tool
that is to be enhanced by a [built-in] recursive tree-walk function.
And every tool will choose its own option subset (and usually also its
own syntax) to implement that.

This observation (incl. inconsistencies between tools' implementations)
make it (design-wise) straightforward for a separation (as already
posted) like tw -tw-options... cmd -cmd-options... cmd-args...

Looking into the original post's GNU grep's options example is quite
enlightening, I'd say.

Janis

On 10/9/2023 9:50 AM, Janis Papanagnou wrote:
> On 09.10.2023 15:18, Ed Morton wrote:
>> On 10/8/2023 8:43 AM, Ben Bacarisse wrote:
>>>
>>> Secondly, I am pretty sure that any near solution to this problem will
>>> come with complications. Your original complaint was based on adding -r
>>> to grep (a very common use case) when find is the consistent option.
>>
>> My complaint is that adding -r to grep makes it inconsistent with and
>> doesn't solve the issues for any of the other similar tools.
>
> I think there's even a bigger issue. It's usually not sufficient to add
> only a -r option,

Right, I'm just using the phrase "adding -r" as an abbreviation for
"adding a bunch of options to find files".

> more sooner than later you want to control subsets of
> the directory tree, thus have to add yet more options - to *every* tool
> that is to be enhanced by a [built-in] recursive tree-walk function.
> And every tool will choose its own option subset (and usually also its
> own syntax) to implement that.
>
> This observation (incl. inconsistencies between tools' implementations)
> make it (design-wise) straightforward for a separation (as already
> posted) like tw -tw-options... cmd -cmd-options... cmd-args...
>
> Looking into the original post's GNU grep's options example is quite
> enlightening, I'd say.

Right again. Adding those options to 1 of the text processing tools was
absurd and adding them to every similar tool would also be absurd -
there is no good solution down that path.

Ed.
>
> Janis
>

Ed Morton <mortonspam@gmail.com> writes:

> On 10/8/2023 8:43 AM, Ben Bacarisse wrote:
>> Ed Morton <mortonspam@gmail.com> writes:
>>
>>> On 10/7/2023 5:17 PM, Ben Bacarisse wrote:
>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>
>>>>> On 10/5/2023 9:25 PM, Ben Bacarisse wrote:
>>> <snip>
>>>>>> But consider the general case... Imagine a collection of tools to test
>>>>>> if a file has this or that property. The tools can all have multiple
>>>>>> arguments but what do they do with conflicting answers? For some
>>>>>> properties it might be useful for the tool to report success (0) if
>>>>>> /all/ the arguments have that property and for other properties it might
>>>>>> be desirable to report success if /any/ of the arguments have it. If
>>>>>> find splits an argument list into two and the tool reports 0 on one half
>>>>>> and 1 on the second, what should find report? If the tool is an "all"
>>>>>> tool it should report 1, and if the tool is an "any" tool it should
>>>>>> report 0.
>>>>>
>>>>> How does adding "-r" to grep solve that problem for awk and sed?
>>>> Eh?
>>>>
>>>
>>> The problem you describe above isn't unique to grep, it also exists for
>>> sed, awk and other tools.
>> Yes. The problem is even described in a tool-agnostic way -- no mention
>> of grep at all!
>>
>>> Giving GNU grep a "-r" option to solve the
>>> problem for grep does nothing to solve the problem for any other tool, but
>>> there are alternative solutions that could be implemented to solve it for
>>> all similar tools as discussed elsethread.
>> Two points. I don't thing there are alternative solutions, but if so,
>> can you point me to them as I've lost track? I want to find one (since
>> I've been bitten by this problem) so I would like to check them out.
>
> A tool such as the tree-walk "tw"
> (https://github.com/att/ast/tree/master/src/cmd/tw) tool that apparently
> can find files and call a tool on those found files as suggested by me and
> others in this thread. I'm not saying that THAT tool is the solution,

For the record, it does not even address the problem I outlined above,
much less solve it. The man page does not even say what its return
status is, so it can't be relied on to anything predictable.

> though, (I'd never heard of it till it was mentioned in this thread and I
> don't know any more about it) I'm saying a solution for all tools could
> have been created instead of modifying grep as was done.

Right. But until there is such a tool, it's hard to be definite about
the merits or demerits of grep -r. With a simple easy to use "rec"
tool, I would ditch using -r because a general solution is obviously
better. But, given the limitations of the Unix process model, it would
have to have flags to allow the user to choose how exit statuses are
combined. It's never going to be totally trivial so there will always
to a slight pressure to implement tool-specific solutions.

>> Secondly, I am pretty sure that any near solution to this problem will
>> come with complications. Your original complaint was based on adding -r
>> to grep (a very common use case) when find is the consistent option.
>
> My complaint is that adding -r to grep makes it inconsistent with and
> doesn't solve the issues for any of the other similar tools.

Yes, that's well understood and I am not disputing it.

Adding -R to ls introduced the same inconsistency a long time ago and
it's not alone. chown, chgrp, diff, rm and cp come to mind. Does the
fact that some of these need to operate in a specific order give them a
pass in your opinion? If so, why does grep -r not get the same pass
given that find -exec grep can't work for all of its uses cases either?

And if not (i.e. if you think none of these should have -r/-R options
either) why is your complaint not about find? Surely the problem would
then be that find it not up to the job, thus forcing the issue for some
tools.

>> But find's behaviour is /not/ consistent with what grep does as I have
>> detailed above. I would go so far as to argue that a solution that is
>> apparently coherent and consistent, right up until it breaks because of
>> something arbitrary like when find breaks it's argument list is
>> borderline dangerous. At the very least, it requires more careful
>> thought than remembering which tools have -r.
>
> I'm not saying that "find" solves all of the issues, I'm saying that adding
> "-r" to grep was the wrong thing to do as it doesn't solve all the issues
> and introduces other issues.

Yes, I get that that is your opinion. My opinion is that since find is
not up to the job, it's hard to say that grep -r was the wrong thing to
do. Do you think that ls -R, rm -r and chmod -r were the wrong thing to
do? What about cp -r? The syntax of find would have to be made even
more complex to make that one work.

I agree that a better solution, in some cases, would be a better find,
but given the limitations it has to work under, even then one would have
to weight up the pros and cons of tool-specific recursive options
because getting it right for grep, rm, chmod and so on is going to
involve more that just a simple "tree-walk" command.

--
Ben.

On 10/9/2023 2:22 PM, Ben Bacarisse wrote:
<snip>
> Do you think that ls -R, rm -r and chmod -r were the wrong thing to
> do? What about cp -r?

As I mentioned elsethread, those are tools that operate on file
attributes, not file contents, so it seems as reasonable to me for them
to be able to find files to operate on as it does for tools that operate
on file contents, such as grep and awk, to be able to find that text
within files to operate on.

If someone decided to add arguments to `rm` or `cp` to change the
contents of files then we could have this same discussion about breaking
with Unix philosophy, loose cohesion, etc. about those tools.

Ed.

On 2023-10-10, Ed Morton <mortonspam@gmail.com> wrote:
> On 10/9/2023 2:22 PM, Ben Bacarisse wrote:
><snip>
>> Do you think that ls -R, rm -r and chmod -r were the wrong thing to
>> do? What about cp -r?
>
> As I mentioned elsethread, those are tools that operate on file
> attributes, not file contents, so it seems as reasonable to me for them
> to be able to find files to operate on as it does for tools that operate
> on file contents, such as grep and awk, to be able to find that text
> within files to operate on.
>
> If someone decided to add arguments to `rm` or `cp` to change the
> contents of files then we could have this same discussion about breaking
> with Unix philosophy, loose cohesion, etc. about those tools.

They do change the contents of files.

One makes them disappear (if the link count drops to zero). The other
can change the content of a an existing file B to be the same as that
of existing file A:

cp A B

It is grep that doesn't change file contents, only reporting on them.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Ed Morton <mortonspam@gmail.com> writes:

> On 10/9/2023 2:22 PM, Ben Bacarisse wrote:
> <snip>
>> Do you think that ls -R, rm -r and chmod -r were the wrong thing to
>> do? What about cp -r?
>
> As I mentioned elsethread, those are tools that operate on file attributes,
> not file contents, so it seems as reasonable to me for them to be able to
> find files to operate on as it does for tools that operate on file
> contents, such as grep and awk, to be able to find that text within files
> to operate on.

I was going to suggest that, as we have both made our points well enough
by now, we could call it a day, but you've introduced a whole new
mystery (to me at least) about what kinds of tool can get away with
having -r and which can't.

To my mind, if I were re-designing the way programs get their arguments,
I would want to come up with one solution that worked for as many of the
above as possible.

--
Ben.

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>> Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>>> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>>>> Geoff Clare <geoff@clare.See-My-Signature.invalid> writes:
>>>>> Ben Bacarisse wrote:
>>>>>> But find is the best we have, and it's a mess. It uses mandatory syntax
>>>>>> that needs to be quoted (the {})
>>>>>
>>>>> The {} does not need to be quoted. That's because { is a reserved
>>>>> word in the shell, not an operator like (.
>>>>
>>>> Is this true of all shells? I was going by the find man page. Maybe
>>>> it needs an update?
>>>
>>> `echo {}` prints "{}" in every shell I've tried (bash, csh, tcsh, ksh,
>>> zsh, fish, dash, busybox sh).
>>
>> The find man page repeats that {} "might" have to be escaped or quoted
>> several times. The EXAMPLES sections quotes it, and then some text goes
>> on to explain why!
>>
>> I accept that it's wrong to say that.
>>
>> So I'll change my remark back to the one I had intended: find uses
>> syntax that needs to be quoted (the ;). Annoyingly, I'd flipped from ;
>> to {} because I didn't think that ; was used in the quoted text!
>
> The find(1) man page (for GNU findutils 4.8.0 and in the latest version
> in the git repo) says in one place (under "-exec command ;") that {} and ;
> *might* need to be escaped, and in another (under "-exec command {}+")
> that {} *needs* to be escaped.

I've submitted a bug report. No response so far.

https://lists.gnu.org/archive/html/bug-findutils/2023-10/msg00002.html

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

On 10/11/2023 7:19 PM, Ben Bacarisse wrote:
> Ed Morton <mortonspam@gmail.com> writes:
>
>> On 10/9/2023 2:22 PM, Ben Bacarisse wrote:
>> <snip>
>>> Do you think that ls -R, rm -r and chmod -r were the wrong thing to
>>> do? What about cp -r?
>>
>> As I mentioned elsethread, those are tools that operate on file attributes,
>> not file contents, so it seems as reasonable to me for them to be able to
>> find files to operate on as it does for tools that operate on file
>> contents, such as grep and awk, to be able to find that text within files
>> to operate on.
>
> I was going to suggest that, as we have both made our points well enough
> by now, we could call it a day,

Me too but you asked me a question so I answered it.

> but you've introduced a whole new
> mystery (to me at least) about what kinds of tool can get away with
> having -r and which can't.
>
> To my mind, if I were re-designing the way programs get their arguments,
> I would want to come up with one solution that worked for as many of the
> above as possible.

Also me too but I'm not too bothered by tools that operate on file
properties (rm, mv, ls, etc.) having different ways to find files just
like I'm not too bothered by tools that operate on text file contents
(grep, sed, awk, etc.) having different ways to find the text within
those files.

Ed.

On 2023-10-07, Ed Morton <mortonspam@gmail.com> wrote:
> On 10/3/2023 2:26 AM, Kaz Kylheku wrote:
>> On 2023-10-02, Ed Morton <mortonspam@gmail.com> wrote:
>>> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>> So I don't see any of that as justifying adding a bunch of options to
>>>>> grep to do something other than it's primary purpose of g/re/p and
>>>>> making it different from all other text processing tools in that
>>>>> regard.
>>>>
>>>> Why do you think it needs justifying? It’s their code, they can add
>>>> whatever they like to it.
>>>>
>>>
>>> The GNU people added functionality to grep in a way that is
>>> contradictory to the Unix philosophy and makes it inconsistent with all
>>> other text processing tools.
>>
>> - GNU stands for GNU is Not Unix.
>>
>> - The GNU coding standards document says:
>>
>> The GNU Project regards standards published by other organizations
>> as suggestions, not orders. We consider those standards, but we do
>> not “obey” them. In developing a GNU program, you should implement
>> an outside standard’s specifications when that makes the GNU system
>> better overall in an objective sense. When it doesn’t, you shouldn’t.
>
> To be clear, I'm not talking about "grep -r" violating any standards,
> I'm talking about it violating the Unix philosophy and software
> engineering fundamentals of cohesion and consistency.
>
>>
>> - Unix programs have recursion built in: from memory I can think of
>> ls, rm, cp, mv, chgrp, chmod and chown.
>
> Those all operate on file attributes, not on the contents of files as
> text processing tools like grep do. Tools like `rm` or `chmod` being
> able to find and perform actions on files is equivalent to text
> processing tools like `grep` or `awk` being able to find and perform
> actions on text inside files.
>
>>
>> - BSD Unixes have a -R option on grep; some, like FreeBSD, have an -r
>> option as well as a rgrep utility.
>
> Yes, GNUisms are apparently creeping into BSD tools, usually for the
> better though.
>
> Ed.
>

Couple of thoughts. First grep has been ported far and wide to systems
that bear little resemblance to unix/posix/linux. Some fraction of those
systems and users will not have access to find.

Second some of those same ported systems are not process friendly and
creating a process for find and another for grep might not be a reasonable
solution. I suspect these two things may have had some influence on
the current state.

Finally I find calling a modern day Ford Mustang a travesty since I'm old
enough to remember 1968. Similarly I find very little unix in todays systems
including the bsd's (geneology notwithstanding). Linux has never bought into
"the unix way" and any doubts were eliminated by systemd. GNU has never
made any real attempt to be limited the existing unix conventions or standards.
Considering the above saying gnu grep doesn't exactly act like a true
unix command is like saying "water is wet".

Personally I see grep -r as a gray area between inconsistent and probably
helpful to some users. I'm more concerned with the general quality of
the software I deal with every day to have bigger concerns about the
future of both software developers and users.

Grep may the the most widely know unix command outside of hardcore
unix types, and I don't expect you will get any traction with you
consisteny concerns. I would also be suprised if you get any
relief either. Don't let it keep you from having a happy holidays!

Stan