vallor <vallor@cultnix.org> writes:
[...]
> Meanwhile, saw someone in another group write:
>
> char * something;
> something = strdup("writable string etc.");
> if( something == NULL ) { etc. }
>
> But that won't work if --std=c99, does work for g99 and c2x.
> The (Linux) man page says:
> /* Since glibc 2.12: */ _POSIX_C_SOURCE >= 200809L
>
> I asked Google about it being a POSIX extension
> added at that late date, and it gave me an answer
> about the C standard:
>
> "C9X London meeting update"
> https://groups.google.com/g/comp.std.c/c/pMaEU_8Rb7w
> _ _ _ _ _
> 2. strsep and strdup are not being added. strsep() is out because
> not enough people wanted it to vote it in; strdup() lost on the
> grounds that it would be the *ONLY* function other than *alloc()
> in the entire library whose return could be sanely passed to free(),
> and this is surprising.
> _ _ _ _ _
>
> Also: <https://stackoverflow.com/questions/32944390/what-is-the-rationale-
> for-not-including-strdup-in-the-c-standard>
>
> Anyway, pointed out that they can just use an initializer, something
> about which I was clued in by a friendly person in this very group.

strdup() and strndup() are being added to the C23 standard.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On Sat, 09 Mar 2024 16:37:19 -0800
Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:

> vallor <vallor@cultnix.org> writes:
> [...]
> > Meanwhile, saw someone in another group write:
> >
> > char * something;
> > something = strdup("writable string etc.");
> > if( something == NULL ) { etc. }
> >
> > But that won't work if --std=c99, does work for g99 and c2x.
> > The (Linux) man page says:
> > /* Since glibc 2.12: */ _POSIX_C_SOURCE >= 200809L
> >
> > I asked Google about it being a POSIX extension
> > added at that late date, and it gave me an answer
> > about the C standard:
> >
> > "C9X London meeting update"
> > https://groups.google.com/g/comp.std.c/c/pMaEU_8Rb7w
> > _ _ _ _ _
> > 2. strsep and strdup are not being added. strsep() is out because
> > not enough people wanted it to vote it in; strdup() lost on the
> > grounds that it would be the *ONLY* function other than *alloc()
> > in the entire library whose return could be sanely passed to free(),
> > and this is surprising.
> > _ _ _ _ _
> >
> > Also:
> > <https://stackoverflow.com/questions/32944390/what-is-the-rationale-
> > for-not-including-strdup-in-the-c-standard>
> >
> > Anyway, pointed out that they can just use an initializer, something
> > about which I was clued in by a friendly person in this very group.
> >
>
> strdup() and strndup() are being added to the C23 standard.
>

What is justification?
What strdup() can do better, for any chosen value of better, than
strlen()+malloc()+memcpy() ?

On Sun, 10 Mar 2024 00:13:38 +0000
Ben Bacarisse <ben.usenet@bsb.me.uk> wrote:

> Richard Harnden <richard.nospam@gmail.invalid> writes:
>
> > On 09/03/2024 12:25, Blue-Maned_Hawk wrote:
> >> Ben wrote:
> ...
> >>>>> On Thu, 7 Mar 2024 06:48:19 -0000 (UTC), Blue-Maned_Hawk wrote:
> >>>>>
> >>>> It does not happen immediately; instead, the signature that the
> >>>> software saves gets more and more corrupted each time the
> >>>> software is opened.
> >>>
> >>> Do you mean the problem is not with Pan after all? "The
> >>> software" is deliberately vague and I don't know why Pan would be
> >>> saving the signature file. It just reads it as far as I can
> >>> tell.
> >>
> >> No, the problem is definitely with Pan. I don't know why it would
> >> be saving the signature file either, but it is.
> >
> > I think Ben has shown that it doesn't.
>
> I think BMH is not using a sig file. The bottom line is that pan can
> do what BMH wants, but he's doing something that does not work.
> Unfortunately his explanation is unhelpful because his sig (as posted)
> is not getting "more and more corrupted" so it's not at all clear what
> he's seeing.
>
> I've tried /not/ using a sig file and that also seems to work. The
> posting.xml configuration file /is/ written every time pan closes, but
> it appears not to be getting corrupted.
>

According to my understanding, one of his requirements is automatic
attachment of random rude phrase to the end of the signature.
Does your solution handle that?

On 09/03/2024 13:19, vallor wrote:

>
> I asked Google about it [strdup] being a POSIX extension
> added at that late date, and it gave me an answer
> about the C standard:
>
> "C9X London meeting update"
> https://groups.google.com/g/comp.std.c/c/pMaEU_8Rb7w

That is from 1997

.... And shows just how useful a searchable usenet archive can be.
Pity google threw their toys out of the pram.

Michael S wrote:

> According to my understanding, one of his requirements is automatic
> attachment of random rude phrase to the end of the signature.
> Does your solution handle that?

I do that manually, and i do not mean the phrases to be rude—if one of
them's been, please let me know and i'll prune it from further selection.

--
Blue-Maned_Hawk│shortens to Hawk│/blu.mɛin.dʰak/│he/him/his/himself/Mr.
blue-maned_hawk.srht.site
Ain't nobody's seen nothin' like that!

Michael S wrote:

> What strdup() can do better, for any chosen value of better, than
> strlen()+malloc()+memcpy() ?

It's shorter.

--
Blue-Maned_Hawk│shortens to Hawk│/blu.mɛin.dʰak/│he/him/his/himself/Mr.
blue-maned_hawk.srht.site
The logical conclusion of “don't repeat yourself” is to never say
anything.

Richard Harnden <richard.nospam@gmail.invalid> writes:

> On 09/03/2024 10:40, Ben wrote:
>> [working sig]
>
> Since you have utf8/ipa working, I don't suppose you could say how to
> pronounce your surname? I know I get it wrong. I'm pretty sure I don't
> even get close.

In English: 'bækəriːs, in French more like baka'ris. (IPA looks horrid
in most software because a mixture of fonts is often used.)

--
Ben.

On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
> On Sat, 09 Mar 2024 16:37:19 -0800
> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>> strdup() and strndup() are being added to the C23 standard.
>>
>
> What is justification?

strdup is required by POSIX already and thus widely implemented.
Many programmers who are not into standards already assume it's in C.

For decades, portable programs have been doing things like this:

#if HAVE_STRDUP
#define xstrdup(s) strdup(s)
#else
char *xstrdup(const char *); // own definition
#endif

> What strdup() can do better, for any chosen value of better, than
> strlen()+malloc()+memcpy() ?

Not take up space in every application for a common library routine.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
> According to my understanding, one of his requirements is automatic
> attachment of random rude phrase to the end of the signature.
> Does your solution handle that?

You can run a cron job that updates the signature file with random
content once every so many minutes. Probably daily would be fine.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

Kaz Kylheku <433-929-6894@kylheku.com> writes:
>On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>> On Sat, 09 Mar 2024 16:37:19 -0800
>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>> strdup() and strndup() are being added to the C23 standard.
>>>
>>
>> What is justification?
>
>strdup is required by POSIX already and thus widely implemented.
>Many programmers who are not into standards already assume it's in C.
>
>For decades, portable programs have been doing things like this:
>
>#if HAVE_STRDUP
>#define xstrdup(s) strdup(s)
>#else
>char *xstrdup(const char *); // own definition
>#endif
>
>> What strdup() can do better, for any chosen value of better, than
>> strlen()+malloc()+memcpy() ?
>
>Not take up space in every application for a common library routine.

It's a form of lazy programming. I've seen a lot of open source
code that uses strdup without checking for failure and frequently
"forgetting" to free the result.

On 10/03/2024 14:03, Ben Bacarisse wrote:
> Richard Harnden <richard.nospam@gmail.invalid> writes:
>
>> On 09/03/2024 10:40, Ben wrote:
>>> [working sig]
>>
>> Since you have utf8/ipa working, I don't suppose you could say how to
>> pronounce your surname? I know I get it wrong. I'm pretty sure I don't
>> even get close.
>
> In English: 'bækəriːs, in French more like baka'ris. (IPA looks horrid
> in most software because a mixture of fonts is often used.)
>

Thanks.

On 10/03/2024 18:47, Scott Lurndal wrote:
> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>>> On Sat, 09 Mar 2024 16:37:19 -0800
>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>> strdup() and strndup() are being added to the C23 standard.
>>>>
>>>
>>> What is justification?
>>
>> strdup is required by POSIX already and thus widely implemented.
>> Many programmers who are not into standards already assume it's in C.
>>
>> For decades, portable programs have been doing things like this:
>>
>> #if HAVE_STRDUP
>> #define xstrdup(s) strdup(s)
>> #else
>> char *xstrdup(const char *); // own definition
>> #endif
>>
>>> What strdup() can do better, for any chosen value of better, than
>>> strlen()+malloc()+memcpy() ?
>>
>> Not take up space in every application for a common library routine.
>
> It's a form of lazy programming. I've seen a lot of open source
> code that uses strdup without checking for failure and frequently
> "forgetting" to free the result.

The hidden use of malloc was one of the reasons it was left out of the
standard library.

On 10/03/2024 18:47, Scott Lurndal wrote:
> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>>> On Sat, 09 Mar 2024 16:37:19 -0800
>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>> strdup() and strndup() are being added to the C23 standard.
>>>>
>>>
>>> What is justification?
>>
>> strdup is required by POSIX already and thus widely implemented.
>> Many programmers who are not into standards already assume it's in C.
>>
>> For decades, portable programs have been doing things like this:
>>
>> #if HAVE_STRDUP
>> #define xstrdup(s) strdup(s)
>> #else
>> char *xstrdup(const char *); // own definition
>> #endif
>>
>>> What strdup() can do better, for any chosen value of better, than
>>> strlen()+malloc()+memcpy() ?
>>
>> Not take up space in every application for a common library routine.
>
> It's a form of lazy programming. I've seen a lot of open source
> code that uses strdup without checking for failure and frequently
> "forgetting" to free the result.

And it is probably more likely that machine with many gigabytes of RAM
will develop an electrical fault than that that call for a short string
will be the point where it runs out of memory.
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

On Mon, 11 Mar 2024 16:23:32 +0000
Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:

> On 10/03/2024 18:47, Scott Lurndal wrote:
> > Kaz Kylheku <433-929-6894@kylheku.com> writes:
> >> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
> >>> On Sat, 09 Mar 2024 16:37:19 -0800
> >>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
> >>>> strdup() and strndup() are being added to the C23 standard.
> >>>>
> >>>
> >>> What is justification?
> >>
> >> strdup is required by POSIX already and thus widely implemented.
> >> Many programmers who are not into standards already assume it's in
> >> C.
> >>
> >> For decades, portable programs have been doing things like this:
> >>
> >> #if HAVE_STRDUP
> >> #define xstrdup(s) strdup(s)
> >> #else
> >> char *xstrdup(const char *); // own definition
> >> #endif
> >>
> >>> What strdup() can do better, for any chosen value of better, than
> >>> strlen()+malloc()+memcpy() ?
> >>
> >> Not take up space in every application for a common library
> >> routine.
> >
> > It's a form of lazy programming. I've seen a lot of open source
> > code that uses strdup without checking for failure and frequently
> > "forgetting" to free the result.
>
> And it is probably more likely that machine with many gigabytes of
> RAM will develop an electrical fault than that that call for a short
> string will be the point where it runs out of memory.

Is there any chance at all that on typical Linux machine (i.e. the one
configured to overcommit virtual memory) strdup() returns NULL?
If it was malloc() I'd say - no chance. For strdup() I'm less sure.

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>On 10/03/2024 18:47, Scott Lurndal wrote:
>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>>>> On Sat, 09 Mar 2024 16:37:19 -0800
>>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>>> strdup() and strndup() are being added to the C23 standard.
>>>>>
>>>>
>>>> What is justification?
>>>
>>> strdup is required by POSIX already and thus widely implemented.
>>> Many programmers who are not into standards already assume it's in C.
>>>
>>> For decades, portable programs have been doing things like this:
>>>
>>> #if HAVE_STRDUP
>>> #define xstrdup(s) strdup(s)
>>> #else
>>> char *xstrdup(const char *); // own definition
>>> #endif
>>>
>>>> What strdup() can do better, for any chosen value of better, than
>>>> strlen()+malloc()+memcpy() ?
>>>
>>> Not take up space in every application for a common library routine.
>>
>> It's a form of lazy programming. I've seen a lot of open source
>> code that uses strdup without checking for failure and frequently
>> "forgetting" to free the result.
>
>And it is probably more likely that machine with many gigabytes of RAM

Actually, your assumptions that:
1) strdup is the only allocation function used by an application
2) all strings are "short"

are both flawed.

>will develop an electrical fault than that that call for a short string
>will be the point where it runs out of memory.

Michael S <already5chosen@yahoo.com> writes:
>On Mon, 11 Mar 2024 16:23:32 +0000
>Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:
>
>> On 10/03/2024 18:47, Scott Lurndal wrote:
>> > Kaz Kylheku <433-929-6894@kylheku.com> writes:
>> >> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>> >>> On Sat, 09 Mar 2024 16:37:19 -0800
>> >>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>> >>>> strdup() and strndup() are being added to the C23 standard.
>> >>>>
>> >>>
>> >>> What is justification?
>> >>
>> >> strdup is required by POSIX already and thus widely implemented.
>> >> Many programmers who are not into standards already assume it's in
>> >> C.
>> >>
>> >> For decades, portable programs have been doing things like this:
>> >>
>> >> #if HAVE_STRDUP
>> >> #define xstrdup(s) strdup(s)
>> >> #else
>> >> char *xstrdup(const char *); // own definition
>> >> #endif
>> >>
>> >>> What strdup() can do better, for any chosen value of better, than
>> >>> strlen()+malloc()+memcpy() ?
>> >>
>> >> Not take up space in every application for a common library
>> >> routine.
>> >
>> > It's a form of lazy programming. I've seen a lot of open source
>> > code that uses strdup without checking for failure and frequently
>> > "forgetting" to free the result.
>>
>> And it is probably more likely that machine with many gigabytes of
>> RAM will develop an electrical fault than that that call for a short
>> string will be the point where it runs out of memory.
>
>Is there any chance at all that on typical Linux machine (i.e. the one
>configured to overcommit virtual memory) strdup() returns NULL?
>If it was malloc() I'd say - no chance. For strdup() I'm less sure.
>

An unanswerable question. But, consider that not all allocations
in an application use strdup as the only memory allocator (the majority don't,
and other allocations may be much larger), yet both use the same pool of
address space and host memory.

Consider also that on unix and linux, there are a number of resource limits
intended to prevent a single application from consuming all of
memory, which a call to strdup may encounter even with plenty of RAM
available.

Toy applications may not have an issue with strdup. Real applications
on the other hand might, and an unexpected SIGSEGV is extremely
user-unfriendly and could have catastrophic results.

And on the gripping hand, not testing for a potential catastrophic
failure condition, no matter how unlikely isn't the sign of a good
programmer.

Michael S <already5chosen@yahoo.com> writes:
[...]
> Is there any chance at all that on typical Linux machine (i.e. the one
> configured to overcommit virtual memory) strdup() returns NULL?
> If it was malloc() I'd say - no chance. For strdup() I'm less sure.

strdup() calls malloc(), so strdup() can return NULL if and only if
malloc() can return NULL -- but with the additional constraint that you
first need to have a string argument to strdup() that's long enough to
cause a suffiently large argument to be passed to malloc().

One data point: On my Ubuntu system, malloc(SIZE_MAX) returns NULL for
very large arguments (over about 23 GiB in my quick and dirty test).

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On Mon, 11 Mar 2024 17:05:40 GMT
scott@slp53.sl.home (Scott Lurndal) wrote:

> Michael S <already5chosen@yahoo.com> writes:
> >On Mon, 11 Mar 2024 16:23:32 +0000
> >Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:
> >
> >> On 10/03/2024 18:47, Scott Lurndal wrote:
> >> > Kaz Kylheku <433-929-6894@kylheku.com> writes:
> >> >> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
> >> >>> On Sat, 09 Mar 2024 16:37:19 -0800
> >> >>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
> >> >>>> strdup() and strndup() are being added to the C23 standard.
> >> >>>>
> >> >>>
> >> >>> What is justification?
> >> >>
> >> >> strdup is required by POSIX already and thus widely implemented.
> >> >> Many programmers who are not into standards already assume it's
> >> >> in C.
> >> >>
> >> >> For decades, portable programs have been doing things like this:
> >> >>
> >> >> #if HAVE_STRDUP
> >> >> #define xstrdup(s) strdup(s)
> >> >> #else
> >> >> char *xstrdup(const char *); // own definition
> >> >> #endif
> >> >>
> >> >>> What strdup() can do better, for any chosen value of better,
> >> >>> than strlen()+malloc()+memcpy() ?
> >> >>
> >> >> Not take up space in every application for a common library
> >> >> routine.
> >> >
> >> > It's a form of lazy programming. I've seen a lot of open source
> >> > code that uses strdup without checking for failure and frequently
> >> > "forgetting" to free the result.
> >>
> >> And it is probably more likely that machine with many gigabytes of
> >> RAM will develop an electrical fault than that that call for a
> >> short string will be the point where it runs out of memory.
> >
> >Is there any chance at all that on typical Linux machine (i.e. the
> >one configured to overcommit virtual memory) strdup() returns NULL?
> >If it was malloc() I'd say - no chance. For strdup() I'm less sure.
> >
>
> An unanswerable question. But, consider that not all allocations
> in an application use strdup as the only memory allocator (the
> majority don't, and other allocations may be much larger), yet both
> use the same pool of address space and host memory.
>
> Consider also that on unix and linux, there are a number of resource
> limits intended to prevent a single application from consuming all of
> memory, which a call to strdup may encounter even with plenty of RAM
> available.
>
> Toy applications may not have an issue with strdup. Real
> applications on the other hand might, and an unexpected SIGSEGV is
> extremely user-unfriendly and could have catastrophic results.
>

According to my understanding, on Linux with overcommit enabled,
typical behavior would be that allocation (of non-extravagant size,
say, no more than 100 MB) always succeeds. OOM is called later, on
access. It seems, that most commonly OOM does not hit application that
is allocating right now. Much more likely that it will kill app that
user opened hours ago, then put aside and then tried to use again.

> And on the gripping hand, not testing for a potential catastrophic
> failure condition, no matter how unlikely isn't the sign of a good
> programmer.

Some people would say that writing code (a handler for allocation
returning NULL) that either can't be tested in principle or can be
tested only in principle, but most certainly not tested in
practice, isn't a sign of a good programmer.
Myself, I still tend to code this checks, but
(1) my main targets are not Linux with overcommit, so the
chance of allocation returning NULL could be estimated like "not going
to happen" rather than "can't happen".
(2) I am old full that like his unreasonable old habits

At least, I stopped checking return value of fclose() after being told,
with facts, how stupid it is. So, may be, one day I'd convince myself to
stop checking return value of malloc() as well.

On 2024-03-11, Scott Lurndal <scott@slp53.sl.home> wrote:
> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>On 10/03/2024 18:47, Scott Lurndal wrote:
>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>> Not take up space in every application for a common library routine.
>>>
>>> It's a form of lazy programming. I've seen a lot of open source
>>> code that uses strdup without checking for failure and frequently
>>> "forgetting" to free the result.
>>
>>And it is probably more likely that machine with many gigabytes of RAM
>
> Actually, your assumptions that:
> 1) strdup is the only allocation function used by an application
> 2) all strings are "short"

Even if a string is long enough to need its own mmap request,
that will still return valid memory that later fails to commit.

Under overcommit conditions, you will only reliably get null return out
of a large strdup if you've exhausted your local address space
(of which you have lots under 64 bits).

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 2024-03-11, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
> Michael S <already5chosen@yahoo.com> writes:
> [...]
>> Is there any chance at all that on typical Linux machine (i.e. the one
>> configured to overcommit virtual memory) strdup() returns NULL?
>> If it was malloc() I'd say - no chance. For strdup() I'm less sure.
>
> strdup() calls malloc(), so strdup() can return NULL if and only if
> malloc() can return NULL -- but with the additional constraint that you
> first need to have a string argument to strdup() that's long enough to
> cause a suffiently large argument to be passed to malloc().
>
> One data point: On my Ubuntu system, malloc(SIZE_MAX) returns NULL for
> very large arguments (over about 23 GiB in my quick and dirty test).

I'm guessing that might be only be because of the overcommit_ratio
value; i.e that allowing a 23 GiB increment in the allocated address
space would bring the system over the currently configured overcommit
ratio.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>Michael S <already5chosen@yahoo.com> writes:
>[...]
>> Is there any chance at all that on typical Linux machine (i.e. the one
>> configured to overcommit virtual memory) strdup() returns NULL?
>> If it was malloc() I'd say - no chance. For strdup() I'm less sure.
>
>strdup() calls malloc(), so strdup() can return NULL if and only if
>malloc() can return NULL -- but with the additional constraint that you
>first need to have a string argument to strdup() that's long enough to
>cause a suffiently large argument to be passed to malloc().
>
>One data point: On my Ubuntu system, malloc(SIZE_MAX) returns NULL for
>very large arguments (over about 23 GiB in my quick and dirty test).

That may be due to resource limits (setrlimit/getrlimit/ulimit) on
the size of the address space.

Michael S <already5chosen@yahoo.com> writes:
>On Mon, 11 Mar 2024 17:05:40 GMT
>scott@slp53.sl.home (Scott Lurndal) wrote:
>
>> Michael S <already5chosen@yahoo.com> writes:
>> >On Mon, 11 Mar 2024 16:23:32 +0000
>> >Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:
>> >
>> >> On 10/03/2024 18:47, Scott Lurndal wrote:
>> >> > Kaz Kylheku <433-929-6894@kylheku.com> writes:
>> >> >> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>> >> >>> On Sat, 09 Mar 2024 16:37:19 -0800
>> >> >>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>> >> >>>> strdup() and strndup() are being added to the C23 standard.
>> >> >>>>
>> >> >>>
>> >> >>> What is justification?
>> >> >>
>> >> >> strdup is required by POSIX already and thus widely implemented.
>> >> >> Many programmers who are not into standards already assume it's
>> >> >> in C.
>> >> >>
>> >> >> For decades, portable programs have been doing things like this:
>> >> >>
>> >> >> #if HAVE_STRDUP
>> >> >> #define xstrdup(s) strdup(s)
>> >> >> #else
>> >> >> char *xstrdup(const char *); // own definition
>> >> >> #endif
>> >> >>
>> >> >>> What strdup() can do better, for any chosen value of better,
>> >> >>> than strlen()+malloc()+memcpy() ?
>> >> >>
>> >> >> Not take up space in every application for a common library
>> >> >> routine.
>> >> >
>> >> > It's a form of lazy programming. I've seen a lot of open source
>> >> > code that uses strdup without checking for failure and frequently
>> >> > "forgetting" to free the result.
>> >>
>> >> And it is probably more likely that machine with many gigabytes of
>> >> RAM will develop an electrical fault than that that call for a
>> >> short string will be the point where it runs out of memory.
>> >
>> >Is there any chance at all that on typical Linux machine (i.e. the
>> >one configured to overcommit virtual memory) strdup() returns NULL?
>> >If it was malloc() I'd say - no chance. For strdup() I'm less sure.
>> >
>>
>> An unanswerable question. But, consider that not all allocations
>> in an application use strdup as the only memory allocator (the
>> majority don't, and other allocations may be much larger), yet both
>> use the same pool of address space and host memory.
>>
>> Consider also that on unix and linux, there are a number of resource
>> limits intended to prevent a single application from consuming all of
>> memory, which a call to strdup may encounter even with plenty of RAM
>> available.
>>
>> Toy applications may not have an issue with strdup. Real
>> applications on the other hand might, and an unexpected SIGSEGV is
>> extremely user-unfriendly and could have catastrophic results.
>>
>
>
>According to my understanding, on Linux with overcommit enabled,
>typical behavior would be that allocation (of non-extravagant size,
>say, no more than 100 MB) always succeeds. OOM is called later, on
>access. It seems, that most commonly OOM does not hit application that
>is allocating right now. Much more likely that it will kill app that
>user opened hours ago, then put aside and then tried to use again.
>
>> And on the gripping hand, not testing for a potential catastrophic
>> failure condition, no matter how unlikely isn't the sign of a good
>> programmer.
>
>Some people would say that writing code (a handler for allocation
>returning NULL) that either can't be tested in principle or can be
>tested only in principle, but most certainly not tested in
>practice, isn't a sign of a good programmer.

1) It is certainly possible to test in practice (unit test, or random error-injection).
2) I've never heard anyone say that.

>Myself, I still tend to code this checks, but
>(1) my main targets are not Linux with overcommit, so the
>chance of allocation returning NULL could be estimated like "not going
>to happen" rather than "can't happen".
>(2) I am old full that like his unreasonable old habits

I consider software that can cause a SIGSEGV when run by
a customer to be broken. The programmer(s) should take
every reasonable precaution to ensure that doesn't happen.

If the OOM killer (in any system where overcommit is enabled)
must run, the system is by definition unstable.

>
>At least, I stopped checking return value of fclose() after being told,
>with facts, how stupid it is. So, may be, one day I'd convince myself to
>stop checking return value of malloc() as well.

An application that cares will fsync(2) the file descriptor before
closing it with close(2) and will check the fsync return value and
take appropriate action if it fails.

While I personally eschew fopen/fclose and their ilk due to
performance overhead generally, were I to use it for data which
is required to be valid, I'd always check the results of fflush
before closing with fclose().

It really depends on the application and the importance of the
data (ephemeral data, e.g. terminal output from ps(1), may not require the
same level of scrutiny as a database, for example).

Kaz Kylheku <433-929-6894@kylheku.com> writes:
>On 2024-03-11, Scott Lurndal <scott@slp53.sl.home> wrote:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>On 10/03/2024 18:47, Scott Lurndal wrote:
>>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>>> Not take up space in every application for a common library routine.
>>>>
>>>> It's a form of lazy programming. I've seen a lot of open source
>>>> code that uses strdup without checking for failure and frequently
>>>> "forgetting" to free the result.
>>>
>>>And it is probably more likely that machine with many gigabytes of RAM
>>
>> Actually, your assumptions that:
>> 1) strdup is the only allocation function used by an application
>> 2) all strings are "short"
>
>Even if a string is long enough to need its own mmap request,
>that will still return valid memory that later fails to commit.

Production linux systems generally disable overcommit. Developing
applications that count on overcommit is a flawed idea.

>
>Under overcommit conditions, you will only reliably get null return out
>of a large strdup if you've exhausted your local address space
>(of which you have lots under 64 bits).

Under overcommit, your application will likely be killed by
the operating system OOM killer, or some other process
may be killed. That is unacceptable in a production environment.

The local address space is limited in unix and linux on a
per-process basis. (setrlimit/getrlimit). The administrator
sets system-wide limits, which can be overridden on a per
UID basis via PAM.

Kaz Kylheku <433-929-6894@kylheku.com> writes:
> On 2024-03-11, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>> Michael S <already5chosen@yahoo.com> writes:
>> [...]
>>> Is there any chance at all that on typical Linux machine (i.e. the one
>>> configured to overcommit virtual memory) strdup() returns NULL?
>>> If it was malloc() I'd say - no chance. For strdup() I'm less sure.
>>
>> strdup() calls malloc(), so strdup() can return NULL if and only if
>> malloc() can return NULL -- but with the additional constraint that you
>> first need to have a string argument to strdup() that's long enough to
>> cause a suffiently large argument to be passed to malloc().
>>
>> One data point: On my Ubuntu system, malloc(SIZE_MAX) returns NULL for
>> very large arguments (over about 23 GiB in my quick and dirty test).
>
> I'm guessing that might be only be because of the overcommit_ratio
> value; i.e that allowing a 23 GiB increment in the allocated address
> space would bring the system over the currently configured overcommit
> ratio.

Good guess, but I don't think so.

$ cat /proc/sys/vm/overcommit_memory
0
$ cat /proc/sys/vm/overcommit_ratio
50

(These are the default values.) If I'm reading the proc(5) man page
correctly (which is questionable), 0 means "heuristic overcommit", and
overcommit_ratio is ignored unless the mode is set to 2.

I have 12 GB of physical memory.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

>
> An application that cares will fsync(2) the file descriptor before
> closing it with close(2) and will check the fsync return value and
> take appropriate action if it fails.
>
> While I personally eschew fopen/fclose and their ilk due to
> performance overhead generally, were I to use it for data which
> is required to be valid, I'd always check the results of fflush
> before closing with fclose().

Last I looked at it, and it was several years ago, so can misremember,
fflush() is only slightly better, if at all better, than fclose(). It
also does not guarantee that my data propagated beyond filesystem write
cache.
For fsync() there are problems as well.
(1) - it is not part of C Standard.
(2) - even on systems where it is implemented, like Linux, BSD and
Mac, it is likely to not do the same things.
(3) - if done in UI thread, it's likely to negatively affect
responsiveness of UI. If not done in UI thread, it's too much of
error-prone code to write.

>
> It really depends on the application and the importance of the
> data (ephemeral data, e.g. terminal output from ps(1), may not
> require the same level of scrutiny as a database, for example).