According to Thomas Koenig <tkoenig@netcologne.de>:
>So, an open question: How does one create a pointer to a nested
>function in architectures which disallow rwx pages?

You make pointers two words, one for the code, one that points to the
display for the dynamic data. This has been a standard compiler
technique since the late 1960s, and indeed the GCC page you pointed to
calls them descriptors. In the fine tradition of GCC overimplmenting
everything, they set bits in descriptors to deliberately misalign them
and indirect calls check to see how much indirection to do.

For a particularly painful example of nested functions, see Knuth's Man or Boy test

https://rosettacode.org/wiki/Man_or_boy_test

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

John Levine <johnl@taugh.com> schrieb:
> According to Thomas Koenig <tkoenig@netcologne.de>:
>>So, an open question: How does one create a pointer to a nested
>>function in architectures which disallow rwx pages?
>
> You make pointers two words, one for the code, one that points to the
> display for the dynamic data. This has been a standard compiler
> technique since the late 1960s, and indeed the GCC page you pointed to
> calls them descriptors. In the fine tradition of GCC overimplmenting
> everything, they set bits in descriptors to deliberately misalign them
> and indirect calls check to see how much indirection to do.

.... which is yields ABI problems if the ABI is modeled on C,
and on function pointers being convertible to void* and back),
and needs requires conditional code for _each_ call through a
function pointer, because it needs to check if it is a vanilla
call or a call to a nested function.

> For a particularly painful example of nested functions, see Knuth's Man or Boy test
>
> https://rosettacode.org/wiki/Man_or_boy_test

Yes, that one is quite challenging.

Quadibloc wrote:
> On Sat, 16 Dec 2023 11:54:40 -0500, EricP wrote:
>> Quadibloc wrote:
>
>>> Here, making it so that once access to write plus execute is turned
>>> off,
>>> it can't be turned on again without a reboot seems appropriate. (Or
>>> "potentially on", because forcing the system to boot with this enabled
>>> isn't a good idea either. So there would be three states: RWE
>>> available,
>>> but off; RWE on; RWE turned permanently off until reboot. From the
>>> third state, there is no return to the first two.)
>
>> Did you take a course in customer abusive design or something?
>> Who is it you think you are going to sell this feature to?
>
> Well, Mitch proposes making it completely and absolutely impossible
> for memory to both contain code that can currently be executed, and
> for it to be writeable, at the same time.
>
> I recognized the logic behind this. Yet, there are cases when this
> would be a problem. So I felt that an option might be to dial the
> security just a tiny notch down:
>
> Make it _possible_ to have an operating system that did allow
> read and execute at the same time, but _also_ make it possible for
> an operating system not to allow it - in a way that can't be
> subverted. The only way to change that being disabled is to reboot
> into a different operating system that does allow it.
>
> John Savard
>

I'm objecting to the notion of forcing a processor to reboot
for any reason, let alone in order to change something that is
under control of the operating system. There was a time before PC's
when computers normally ran for a year without reboot and never crashed.

As for Mitch's RW-no-E restriction, I am confident it
will not survive its first encounter with actual customers
(where philosophy collides with income).

EricP <ThatWouldBeTelling@thevillage.com> schrieb:

> I'm objecting to the notion of forcing a processor to reboot
> for any reason, let alone in order to change something that is
> under control of the operating system. There was a time before PC's
> when computers normally ran for a year without reboot and never crashed.

[tkoenig@cfarm135 ~]$ uptime
18:03:12 up 1537 days, 19:33, 1 user, load average: 12,31, 9,59, 8,58
[tkoenig@cfarm135 ~]$ uname -a
Linux cfarm135 4.18.0-80.7.2.el7.ppc64le #1 SMP Thu Sep 12 15:45:05 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux

Thomas Koenig wrote:

> EricP <ThatWouldBeTelling@thevillage.com> schrieb:

>> I'm objecting to the notion of forcing a processor to reboot
>> for any reason, let alone in order to change something that is
>> under control of the operating system. There was a time before PC's
>> when computers normally ran for a year without reboot and never crashed.

When we shut down the server at Ross Technologies, it had been up for
7 years and several months. It had 16× the memory it started with,
25× the disk space, and every CPU had been upgraded multiple times
from single core 33 Mhz SUN to quad core 150 Mhz Ross modules.

On Sun, 17 Dec 2023 11:38:43 -0500, EricP wrote:

> As for Mitch's RW-no-E restriction, I am confident it will not survive
> its first encounter with actual customers (where philosophy collides
> with income).

It's true that since it is Mitch who advanced the assertion that it is
is a useful restriction here, it is he who people should debate its
needfulness with. However, I do not believe that it was _his idea_,
but rather that this restriction is *current accepted practice*, and
indeed is already implemented on a number of current processor
architectures.

That's why I decided that forcing reboot to get out of it was a
reasonable compromise between supporting legacy operation and
still having a fairly basic security feature.

John Savard

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>Quadibloc <quadibloc@servername.invalid> writes:
>>On Sat, 16 Dec 2023 17:33:48 +0000, Anton Ertl wrote:
>>> Quadibloc <quadibloc@servername.invalid> writes:
>>
>>>>Although I tend to agree with you, I also think that allowing write
>>>>permission and execute permission at the same time is highly dangerous.
>>
>>> Security theatre!
>>>
>>> Gforth uses mmap(... RWX ...) for the area where it generates native
>>> code. Now on MacOS on Apple Silicon (MacOS on Intel works fine) that
>>> mmap() fails, which means that currently the development version of
>>> Gforth does not work. When I work around this misfeature of MacOS on
>>> Apple Silicon (by disabling native code generation), Gforth will run
>>> around a factor of 2 slower on MacOS than on Linux.
>>
>>This is an argument you need to have with Mitch, rather than
>>with me.
>
>This is a newsgroup, he can read and react on my posting as well as
>you and others.
>
>>I lack the expertise to really assess how valuable not allowing
>>write and execute at the same time is; it certainly seems potentially
>>dangerous, so I went along with Mitch.
>
>The point I was making is that in the case of Gforth, it does not add
>any danger that is not already there. If you let an untrusted user
>run Gforth, it's like allowing that user to run non-suid binaries that
>that user supplied.

Assuming you're the klapatious who posted the question on stack
overflow, the answer was there - you need to switch on the jit
mode (and have the jit privilege to do what you want).

Thomas Koenig <tkoenig@netcologne.de> writes:
>EricP <ThatWouldBeTelling@thevillage.com> schrieb:
>
>> I'm objecting to the notion of forcing a processor to reboot
>> for any reason, let alone in order to change something that is
>> under control of the operating system. There was a time before PC's
>> when computers normally ran for a year without reboot and never crashed.
>
>[tkoenig@cfarm135 ~]$ uptime
> 18:03:12 up 1537 days, 19:33, 1 user, load average: 12,31, 9,59, 8,58
>[tkoenig@cfarm135 ~]$ uname -a
>Linux cfarm135 4.18.0-80.7.2.el7.ppc64le #1 SMP Thu Sep 12 15:45:05 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux

My best was on a 2000 vintage sony vaio laptop running RH8 hosting sendmail, apache and named
on the internet:

Mon May 10 11:27:56 PDT 2010
11:28am up 1528 days, 6:47, 8 users, load average: 0.23, 0.12, 0.05

Got taken out by a power outage where the 10 year old laptop battery
couldn't cover the outage.

On Thu, 14 Dec 2023 12:44:46 +0000, Quadibloc wrote:

> In any case, this has inspired me to think of a feature to add to
> Concertina II. Add a new kind of header block, which must be the very
> first one, which contains an encrypted checksum of the rest of the block
> - which must be valid for the block to be allowed to execute. A mode
> exists where only blocks with such a checksum can be executed.

But given that a common feature of present-day computers is to
disable writing to blocks which contain executable code, what on
Earth does this feature provide that that one does not provide
already?

I finally came out of my mental fog to see the answer!

Let us have the computer able to put itself into a mode where
while in privileged mode, it can only execute checksummed code.

This will protect the entire kernel against tampering. But what
you really want is to protect the whole operating system, including
the nonprivileged parts, from tampering.

Well, let the MMU have, in addition to "execute" permission to
hand out, an "execute checksummed only" permission to hand out. So
the kernel uses that whenever it allocates blocks containing operating
system code (hmm, is it going to need to do deep object module
inspection to do this?)... so now these features work together.

And then, *duh*, the benefit of these checksums over no write-execute
hits me. Write execute bans protect the copy of code that's in the
memory. Checksums even protect the copy *that's residing on the
computer's disk storage*, which should be of some value.

John Savard

According to Thomas Koenig <tkoenig@netcologne.de>:
>> You make pointers two words, one for the code, one that points to the
>> display for the dynamic data. This has been a standard compiler
>> technique since the late 1960s, ...

>... which is yields ABI problems if the ABI is modeled on C,
>and on function pointers being convertible to void* and back),
>and needs requires conditional code for _each_ call through a
>function pointer, because it needs to check if it is a vanilla
>call or a call to a nested function.

Yup, after almost 50 years C still suffers from "everything is really
an int". You can use C, or you can use nested functions, but you'll be
sad if you try to do both at the same time.

I never found the lack of nested functions in C to be much of a
problem. In the rare cases where I want to do something like that,
it's not hard to pass what would have been in the display as
explicit arguments.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

On Sun, 17 Dec 2023 19:41:45 +0000, Quadibloc wrote:

> And then, *duh*, the benefit of these checksums over no write-execute
> hits me. Write execute bans protect the copy of code that's in the
> memory. Checksums even protect the copy *that's residing on the
> computer's disk storage*, which should be of some value.

And now my vision of the secure computer of the future should be
clear.

On the front of the computer is a little panel that you can slide
open when the computer is turned off. Behind it is a switch.

The switch turns on checksum protection, or turns it off.

One starts by turning it off.

Then the installation program on the DVD-ROM drive can run
as the first thing the computer boots from. And, indeed,
the ROM BIOS can run, to tell the computer how to work the
DVD-ROM drive.

And, indeed, the first DVD-ROM you use came in the box with
your *motherboard*; it is used to install a BIOS on flash
memory which will be checksummed, so it can run when
checksum protection is turned on.

It asks you for a password, which is hashed to produce the
checksum key, stored securely in the CPU.

Then you use the DVD-ROM for installing the operating system,
with checksum protection still turned off. (Even though
you've rebooted after flashing the BIOS. Just because checksum
protection is turned off doesn't mean that checksummed code
won't run; it only means that non-checksummed code can _also_
run.)

The installer program, read in from the DVD-ROM, asks for
a password - which has to be the same one you used for
installing the BIOS - and so it now copies the object code
of the operating system to the hard drive, with appropriate
checksums, unique to _your computer_, in every block of
instructions.

Keep the password, because the operating system won't be
able to update itself without it. (So the favored method
of hacking one of these systems will be to impersonate the
update screen to *get* the key that lets one ask the CPU
to write checksums that work!)

And after the operating system is installed, when the
computer turns itself off, turn on checksum protection,
and turn the computer back on.

This basic technique can of course be augmented by using
more secure and lower-overhead techniques whereby the
computer insists on digital signatures on object modules.

Unlike the digital signature techniques now in use, though,
here the unique key is created by the user, and under the
user's control, so Windows is not privileged above Linux
as has caused controversy around Secure Boot.

John Savard

On Sun, 17 Dec 2023 20:14:37 +0000, Quadibloc wrote:

> The installer program, read in from the DVD-ROM, asks for a password -
> which has to be the same one you used for installing the BIOS -

This assumes that everyone uses the same hash algorithm
to go from a password to a checksum key.

The alternative, where every motherboard maker uses its
own hash algorithm, which then gives back a long string
of hex digits which have to be typed in during operating
system installation does not bear thinking about.

However, this gives hackers a smaller search space in which
to look for bad passwords, which are highly likely when it
is so vitally important not to forget the password which is
rarely used!

Hmm. Perhaps the BIOS installer could also think of a number,
which the user must also write down? And then the computer
might have a unique serial number which can be used also,
although that has its issues too.

John Savard

On 12/17/2023 10:38 AM, EricP wrote:
> Quadibloc wrote:
>> On Sat, 16 Dec 2023 11:54:40 -0500, EricP wrote:
>>> Quadibloc wrote:
>>
>>>> Here, making it so that once access to write plus execute is turned
>>>> off,
>>>> it can't be turned on again without a reboot seems appropriate. (Or
>>>> "potentially on", because forcing the system to boot with this enabled
>>>> isn't a good idea either. So there would be three states: RWE
>>>> available,
>>>> but off; RWE on; RWE turned permanently off until reboot. From the
>>>> third state, there is no return to the first two.)
>>
>>> Did you take a course in customer abusive design or something?
>>> Who is it you think you are going to sell this feature to?
>>
>> Well, Mitch proposes making it completely and absolutely impossible
>> for memory to both contain code that can currently be executed, and
>> for it to be writeable, at the same time.
>>
>> I recognized the logic behind this. Yet, there are cases when this
>> would be a problem. So I felt that an option might be to dial the
>> security just a tiny notch down:
>>
>> Make it _possible_ to have an operating system that did allow
>> read and execute at the same time, but _also_ make it possible for
>> an operating system not to allow it - in a way that can't be
>> subverted. The only way to change that being disabled is to reboot
>> into a different operating system that does allow it.
>>
>> John Savard
>>
>
> I'm objecting to the notion of forcing a processor to reboot
> for any reason, let alone in order to change something that is
> under control of the operating system. There was a time before PC's
> when computers normally ran for a year without reboot and never crashed.
>
> As for Mitch's RW-no-E restriction, I am confident it
> will not survive its first encounter with actual customers
> (where philosophy collides with income).
>

Yeah.
Imposing such a restriction at the ISA level is likely a bit too much of
an ask.

Better IMO to have flags to encode various modes, say:
Read Only
Read/Write
Read/Execute
Read/Write/Execute.

For these, one might have, say, a No-Write and No-Execute flag.
Write-Only and Execute-Only are not really common modes.

However, in my case, the MMU has a No-Read flag as well, so these are
technically possible.

For the main page flags, one might also have, say:
No-User (Supervisor Only)
No-Cache
Shared/Global
Valid/Present
...

If doing ACL checks, it may make sense to have a separate set of
Read/Write/Execute flags that are applied during ACL checking.

I guess, maybe relevant could be to devise a scheme for mapping ACL
checking onto a conventional page-walking style MMU design.

Where, say, one associates pairs of 16-bit Keys/ACLIDs with 16-bit field
in the tables (say: user/group/other read/write/execute flags, and some
mode-control bits for the matching rules).

Though, I guess one possible option could be to Morton-shuffle the two
IDs and then use them as a key into a page-table like structure.

With 16-bit ACLIDs, this would be a 3 level table.

For software handling, one can use a more space-efficient structure
(such as a B-Tree). But, a B-Tree walk or similar in hardware likely
isn't viable.

But, then again, ACL space is likely very sparse (if looking at pairs),
where only certain pairs of keys have anything defined, so a
page-table-like structure is unlikely to be particularly space-efficient
in this case.

....

BGB wrote:
> On 12/17/2023 10:38 AM, EricP wrote:
>>
>> As for Mitch's RW-no-E restriction, I am confident it
>> will not survive its first encounter with actual customers
>> (where philosophy collides with income).
>>
>
> Yeah.
> Imposing such a restriction at the ISA level is likely a bit too much of
> an ask.
>
> Better IMO to have flags to encode various modes, say:
> Read Only
> Read/Write
> Read/Execute
> Read/Write/Execute.
>
> For these, one might have, say, a No-Write and No-Execute flag.
> Write-Only and Execute-Only are not really common modes.
>
> However, in my case, the MMU has a No-Read flag as well, so these are
> technically possible.
>
>
> For the main page flags, one might also have, say:
> No-User (Supervisor Only)
> No-Cache
> Shared/Global
> Valid/Present
> ...

If one has super and User mode, and R,W,E enables, that's 6 PTE bits.
The PTE is very cramped so there is a big incentive to save any bits.
We can get that down to 3 PTE bits by throwing out the silly combinations,
Write-only, Execute-only, Super:NoAccess+User:RWE,...

The P-Present bit suffices for full No-Access pages.

One could hard code 8 reasonable choices from the 64 possible but
my preferred approach now is to use the 3-bit access control field
as an index to an 8-entry table with the full 6 bits for each row,
and let the OS decide which 8 combinations of Super and User accesses
make sense for it.

I'd also use a 3-bit PTE field to index an 8 entry table for cache control,
as x86 does. Though it also has to deal with legacy issues so maybe
2-bits would suffice here (depends on whether one wants to have
write-allocate vs write-noallocate as distinct selectable options).

On 12/18/2023 12:14 PM, EricP wrote:
> BGB wrote:
>> On 12/17/2023 10:38 AM, EricP wrote:
>>>
>>> As for Mitch's RW-no-E restriction, I am confident it
>>> will not survive its first encounter with actual customers
>>> (where philosophy collides with income).
>>>
>>
>> Yeah.
>> Imposing such a restriction at the ISA level is likely a bit too much
>> of an ask.
>>
>> Better IMO to have flags to encode various modes, say:
>>   Read Only
>>   Read/Write
>>   Read/Execute
>>   Read/Write/Execute.
>>
>> For these, one might have, say, a No-Write and No-Execute flag.
>>   Write-Only and Execute-Only are not really common modes.
>>
>> However, in my case, the MMU has a No-Read flag as well, so these are
>> technically possible.
>>
>>
>> For the main page flags, one might also have, say:
>>   No-User (Supervisor Only)
>>   No-Cache
>>   Shared/Global
>>   Valid/Present
>>   ...
>
> If one has super and User mode, and R,W,E enables, that's 6 PTE bits.
> The PTE is very cramped so there is a big incentive to save any bits.
> We can get that down to 3 PTE bits by throwing out the silly combinations,
> Write-only, Execute-only, Super:NoAccess+User:RWE,...
>
> The P-Present bit suffices for full No-Access pages.
>

As long as the total stays under 12 bits or so, probably fine...

Say, my case:
* (11): PR.U3 (User 3, OS / Virtual Memory)
* (10): PR.U2 (User 2, OS / Virtual Memory)
* ( 9): PR.S1 (Size 1) / PR.U1 (User 1, OS / Virtual Memory)
* ( 8): PR.S0 (Size 0) / PR.U0 (User 0, OS / Virtual Memory)
* ( 7): PR.NU (Not User Accessible)
* ( 6): PR.NX (Execute Disable)
* ( 5): PR.NW (Write Disable)
* ( 4): PR.NR (Read Disable)
* ( 3): PR.NC, Page Is Volatile / Non-Cachable
* ( 2): PR.D, (Dirty)
* ( 1): PR.V1, Page Valid Mode / Shared
* ( 0): PR.V0, Page Valid Mode / Present

There are not separate user and supervisor bits in this case.

There are U4 and U5, but these only exist with 16K pages (or U6 and U7
with 64K pages).

The Dirty flag is effectively another User bit in practice, as my MMU
doesn't actually use it for anything (and is more used so the OS
page-management can keep track of if a page may be dirty by effectively
implementing a write barrier).

But, some special cases, such as "User Read Only" may be encoded:
* NC Set, NU Clear:
** NX Clear and NW Set: Secure Execute Mode (*1)
** NX Set and NW Set: User Read Only

*1: These pages allow userland to "temporarily" gain supervisor-mode
instructions, but:
Only if certain magic instructions are invoked on entering and leaving
these pages (trying to execute these instructions elsewhere is a Fault,
trying to leave without using the first exiting the mode, is a fault,
trying to use supervisor instructions without first entering
secure-execute mode via the special instructions is a fault, ...).

For page-table entries:
(63:48) holds the ACLID.

As can be noted, the ACLID and KRR register are used for additional page
access checks. The ACL handling is generally done using a separate
table. If a KRR key is not in the ACL Cache, it will raise an ACL-MISS
fault, and the ISR will load it from the table.

The ACL Cache is currently a very small but fully-associative cache
(needs at least 4-way, 8-way is better, but 8-entries is generally all
that is needed for this cache, as there tend to be only a few ACLs in
use at any given time).

Note the ACLID is not the ASID, where the ASID is held in bits (63:48)
of the TTB register (Page Table Base), with the low-order bits of TTB
mostly used to indicate the page-table layout.

> One could hard code 8 reasonable choices from the 64 possible but
> my preferred approach now is to use the 3-bit access control field
> as an index to an 8-entry table with the full 6 bits for each row,
> and let the OS decide which 8 combinations of Super and User accesses
> make sense for it.
>
> I'd also use a 3-bit PTE field to index an 8 entry table for cache control,
> as x86 does. Though it also has to deal with legacy issues so maybe
> 2-bits would suffice here (depends on whether one wants to have
> write-allocate vs write-noallocate as distinct selectable options).
>

Dunno...

>
>

EricP wrote:

>
> If one has super and User mode, and R,W,E enables, that's 6 PTE bits.
> The PTE is very cramped so there is a big incentive to save any bits.
> We can get that down to 3 PTE bits by throwing out the silly combinations,
> Write-only, Execute-only, Super:NoAccess+User:RWE,...

If one uses a different page table for user and supervisor, you only
need those 3-bits. In my case Guest OS has its own table and its own
ASID completely separate from application. ASID plays the part of G
(global).

On Sun, 17 Dec 2023 19:31:15 +0000, Scott Lurndal wrote:

> Assuming you're the klapatious who posted the question on stack
> overflow,

I presume that's a slightly misspelled reference to Klapaucius,
a robot (along with another, named Trurl) from Stanislaw Lem's _Cyberiad_.

John Savard

John Levine <johnl@taugh.com> schrieb:
> According to Thomas Koenig <tkoenig@netcologne.de>:
>>> You make pointers two words, one for the code, one that points to the
>>> display for the dynamic data. This has been a standard compiler
>>> technique since the late 1960s, ...
>
>>... which is yields ABI problems if the ABI is modeled on C,
>>and on function pointers being convertible to void* and back),
>>and needs requires conditional code for _each_ call through a
>>function pointer, because it needs to check if it is a vanilla
>>call or a call to a nested function.
>
> Yup, after almost 50 years C still suffers from "everything is really
> an int". You can use C, or you can use nested functions, but you'll be
> sad if you try to do both at the same time.

It becomes more interesting if you want to design an ABI (or an
ABI together with an ISA) which caters well to them, or have a
C-based ABI and try to fit them in.

> I never found the lack of nested functions in C to be much of a
> problem. In the rare cases where I want to do something like that,
> it's not hard to pass what would have been in the display as
> explicit arguments.

Some languages have it in their specification (Fortran and, I
believe, Ada), so they need to be supported, and should
be supported well.

According to Thomas Koenig <tkoenig@netcologne.de>:
>> I never found the lack of nested functions in C to be much of a
>> problem. In the rare cases where I want to do something like that,
>> it's not hard to pass what would have been in the display as
>> explicit arguments.
>
>Some languages have it in their specification (Fortran and, I
>believe, Ada), so they need to be supported, and should
>be supported well.

If you want to support nested functions, it's not hard, you make
function pointers two words, the code pointer and the static chain.

As you note, what's hard is trying to retrofit that into a C model
where all pointers are the same. Then you end up with kludges like
trampolines and misaligned descriptor pointers.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

John Levine <johnl@taugh.com> writes:
>According to Thomas Koenig <tkoenig@netcologne.de>:
>>> I never found the lack of nested functions in C to be much of a
>>> problem. In the rare cases where I want to do something like that,
>>> it's not hard to pass what would have been in the display as
>>> explicit arguments.
>>
>>Some languages have it in their specification (Fortran and, I
>>believe, Ada), so they need to be supported, and should
>>be supported well.
>
>If you want to support nested functions, it's not hard, you make
>function pointers two words, the code pointer and the static chain.

Why? Nested functions need access to containing function local _data_,
not code. The actual generated code is independent of the containing
function.

Gcc has an extension that supports nested functions.

https://gcc.gnu.org/onlinedocs/gcc/Nested-Functions.html

On 2023-12-19 16:37, Thomas Koenig wrote:
> John Levine <johnl@taugh.com> schrieb:
>> According to Thomas Koenig <tkoenig@netcologne.de>:
>>>> You make pointers two words, one for the code, one that points to the
>>>> display for the dynamic data. This has been a standard compiler
>>>> technique since the late 1960s, ...
>>
>>> ... which is yields ABI problems if the ABI is modeled on C,
>>> and on function pointers being convertible to void* and back),
>>> and needs requires conditional code for _each_ call through a
>>> function pointer, because it needs to check if it is a vanilla
>>> call or a call to a nested function.
>>
>> Yup, after almost 50 years C still suffers from "everything is really
>> an int". You can use C, or you can use nested functions, but you'll be
>> sad if you try to do both at the same time.
>
> It becomes more interesting if you want to design an ABI (or an
> ABI together with an ISA) which caters well to them, or have a
> C-based ABI and try to fit them in.
>
>> I never found the lack of nested functions in C to be much of a
>> problem. In the rare cases where I want to do something like that,
>> it's not hard to pass what would have been in the display as
>> explicit arguments.
>
> Some languages have it in their specification (Fortran and, I
> believe, Ada), so they need to be supported, and should
> be supported well.

Yes, Ada supports nested subprograms, and also supports nesting for
other kinds of things such as tasks/threads.

But coding standards for high-integrity Ada applications usually forbid
nested subprograms, typically because they are difficult to unit-test,
and early in the history of Ada there were voices suggesting the same
generally, for example the 1980 paper "Nesting in Ada programs is for
the birds", https://dl.acm.org/doi/10.1145/800004.807944. From the
abstract: "Given a data abstraction construct like the Ada package and
in light of current thoughts on programming methodology, we feel that
nesting is an anachronism."

In my own (non-critical) Ada programs, small nested subprograms are not
unusual, but pointers to them are very rare (I don't recall a single
case of it).

Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>In my own (non-critical) Ada programs, small nested subprograms are not
>unusual, but pointers to them are very rare (I don't recall a single
>case of it).

That's interesting, because a major reason for using nested functions
(or, more generally, closures), is if you need to pass a function with
a given interface, but have to pass additional data to that function.

E.g., consider a numerical integration function integrate(f,lo,hi),
which expects a function with one parameter f(x), and you want to
integrate x^y (pow(x,y) in C), where y is, say, passed as parameter.
Then you would do something like:

double intpowy(double y, double lo, double hi)
{ double f(double x)
{
return pow(x,y);
}
return integrate(f,lo,hi);
}

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On 2023-12-19 20:12, Anton Ertl wrote:
> Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>> In my own (non-critical) Ada programs, small nested subprograms are not
>> unusual, but pointers to them are very rare (I don't recall a single
>> case of it).
>
> That's interesting, because a major reason for using nested functions
> (or, more generally, closures), is if you need to pass a function with
> a given interface, but have to pass additional data to that function.

In Ada, instead of passing a (nested) function pointer to some service
that needs such a parameter, one can implement the service as a generic
program unit, where the function to be passed is given as a generic
parameter, and instantiate that generic locally in the nested context,
thus binding the instance with the nested function.

Depending on the way the compiler implements such generics (as macros,
or with shared code) this may or may not be implemented by passing a
function pointer, but even if it is, the compiler can use a "fat
pointer" for this generic case, without being forced to use a fat
pointer for all function pointers.

> E.g., consider a numerical integration function integrate(f,lo,hi),
> which expects a function with one parameter f(x), and you want to
> integrate x^y (pow(x,y) in C), where y is, say, passed as parameter.

That is the canonical example, yes. But it happens I have never needed
to do numerical integration in my Ada programs. I do have some cases
where I have used locally instantiated generics with generic function
parameters in the way I described above. In Ada the integration example
would be declared thusly:

generic
with function F (X : Float) return Float;
function Integrate (Lo, Hi : Float) return Float;

although one would usually not limit it to a specific number type such
as Float, but would make this type also a generic (type) parameter.
IIRC, the ability to define numerical integration as a generic
subprogram, with the function to be integrated as a generic parameter,
was one of the reasons presented for omitting function pointers entirely
from original Ada (Ada 83).

Niklas Holsti <niklas.holsti@tidorum.invalid> schrieb:

>> E.g., consider a numerical integration function integrate(f,lo,hi),
>> which expects a function with one parameter f(x), and you want to
>> integrate x^y (pow(x,y) in C), where y is, say, passed as parameter.
>
>
> That is the canonical example, yes. But it happens I have never needed
> to do numerical integration in my Ada programs. I do have some cases
> where I have used locally instantiated generics with generic function
> parameters in the way I described above. In Ada the integration example
> would be declared thusly:
>
> generic
> with function F (X : Float) return Float;
> function Integrate (Lo, Hi : Float) return Float;
>
> although one would usually not limit it to a specific number type such
> as Float, but would make this type also a generic (type) parameter.
> IIRC, the ability to define numerical integration as a generic
> subprogram, with the function to be integrated as a generic parameter,
> was one of the reasons presented for omitting function pointers entirely
> from original Ada (Ada 83).

Interesting, thanks!

Just one remark: Making numerical routines generic can be a bit
problematic if constants depend on the accuracy of a type; things
like tolerances for checks for convergence, number of coefficients
in a polynomial (or Chebyshev) evaluation, quantities to divide by
for numeric differentiation etc.

Thomas Koenig wrote:

> Niklas Holsti <niklas.holsti@tidorum.invalid> schrieb:

>>> E.g., consider a numerical integration function integrate(f,lo,hi),
>>> which expects a function with one parameter f(x), and you want to
>>> integrate x^y (pow(x,y) in C), where y is, say, passed as parameter.
>>
>>
>> That is the canonical example, yes. But it happens I have never needed
>> to do numerical integration in my Ada programs. I do have some cases
>> where I have used locally instantiated generics with generic function
>> parameters in the way I described above. In Ada the integration example
>> would be declared thusly:
>>
>> generic
>> with function F (X : Float) return Float;
>> function Integrate (Lo, Hi : Float) return Float;
>>
>> although one would usually not limit it to a specific number type such
>> as Float, but would make this type also a generic (type) parameter.
>> IIRC, the ability to define numerical integration as a generic
>> subprogram, with the function to be integrated as a generic parameter,
>> was one of the reasons presented for omitting function pointers entirely
>> from original Ada (Ada 83).

> Interesting, thanks!

> Just one remark: Making numerical routines generic can be a bit
> problematic if constants depend on the accuracy of a type; things
> like tolerances for checks for convergence,

whether compiler constant arithmetic has the same numeric properties as
compiled code instruction sequences,

> number of coefficients
> in a polynomial (or Chebyshev) evaluation, quantities to divide by
> for numeric differentiation etc.