The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".

The expected interpretation semantics for "[: ... ;]" are that this
construct behaves like ":noname ... ;" (at least for the resulting stack
effect), but the system should correctly work regardless whether the
current definition exists (i.e., a definition that is being compiled).
It means, if a definition is being compiled, its compilation shall be
correctly suspended by "[:", and resumed after ";]" under the hood.

Bellow are the test cases for the standardized compilation semantics,
and for the expected interpretation semantics.

Interestingly, some Forth systems fail the t12 and t13 tests for the
standardized compilation semantics.

My implementation passes all these test cases. Its basic factors are as
follows.

The updated "current definition" term:
*current definition*: The definition whose compilation has been started
most recently but not yet ended.

CONCEIVE ( -- ) ( C: -- def-sys )
Start compilation of the new definition.
This word shall correctly work regardless whether the current definition
exists. It does not change STATE.

BIRTH ( -- xt ) ( C: def-sys -- )
End compilation of the current definition and return its xt.

GERM ( -- xt|0 )
If the current definition exists return its xt, otherwise return 0.

===== start of "quotation.test.fth"

\ Test cases for quotations "[: ... ;]"

\ Testing the compilation semantics

\ t11
t{ :noname [: 123 ;] ; execute execute -> 123 }t

: lit, postpone literal ;
: ([:) postpone [: ;
: (;]) postpone ;] ;

\ t12
t{ :noname [ ([:) (;]) ] ; 0<> -> -1 }t
\ t13
t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t

\ Testing the interpretation semantics
\ (the expected behavior)

\ t21
t{ depth [: ;] depth 1- = ?dup nip -> 0 }t
\ t22
t{ [: 123 ;] execute -> 123 }t
\ t23
t{ [: 1 [: 2 ;] 3 ;] execute swap execute -> 1 3 2 }t

\ Testing the interpretation semantics
\ doing compilation of another definition
\ (the expected behavior)

\ t31
t{ [: [ depth [: ;] depth 1- = ?dup nip ] literal ;] execute -> 0 }t
\ t32
t{ [: 1 [ [: 2 ;] ] literal 3 ;] execute swap execute -> 1 3 2 }t

===== end of "quotation.test.fth"

[1] http://www.forth200x.org/quotations-v4.txt

--
Ruvim

On 2024-03-07 16:29, Ruvim wrote:
> The accepted proposal for quotations [1] specifies only compilation
> semantics for the words "[:" and ";]".
>
> The expected interpretation semantics for "[: ... ;]" are that this
> construct behaves like ":noname ... ;" (at least for the resulting stack
> effect), but the system should correctly work regardless whether the
> current definition exists (i.e., a definition that is being compiled).
> It means, if a definition is being compiled, its compilation shall be
> correctly suspended by "[:", and resumed after ";]" under the hood.
>
> Bellow are the test cases for the standardized compilation semantics,
> and for the expected interpretation semantics.
>
> Interestingly, some Forth systems fail the t12 and t13 tests for the
> standardized compilation semantics.
>

[...]

>
>
>
> ===== start of "quotation.test.fth"
>
> \ Test cases for quotations "[: ... ;]"
>
>
>
> \ Testing the compilation semantics
>
> \ t11
> t{ :noname [: 123 ;] ; execute execute -> 123 }t
>
> : lit, postpone literal ;
> : ([:) postpone  [: ;
> : (;]) postpone  ;] ;
>
> \ t12
> t{ :noname  [ ([:) (;]) ]  ;  0<>  ->  -1  }t > \ t13
> t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t
>

t12 roughly tests that the compilation semantics for the words "[:" and
";]" can be correctly performed programmatically.

This test fails in Gforth, VfxForth, minForth.

The problem is that the system enters compilation state after executing
"([:)". But it shall not. This action cannot be a part of compilation
semantics at all (it's not possible to correctly specify this action,
and it is not actually specified for the "[:" compilation semantics).

The reason of the problem is that "postpone" is implemented not quite
correctly in these systems. After including a polyfill for "postpone"
[2], the tests t12 and t13 are passed in Gforth.

VfxForth (5.20 Alpha 1 [build 4065]) and minForth (V3.4.8) provide
neither "find-name" nor correct "find", so "postpone" cannot be
correctly defined by the polyfill.

[2] About POSTPONE semantics in edge cases / Solution
<https://github.com/ForthHub/discussion/discussions/103#solution>

--
Ruvim

Ruvim wrote:

> The accepted proposal for quotations [1] specifies only compilation
> semantics for the words "[:" and ";]".

> The expected interpretation semantics for "[: ... ;]" are ...

Expected by whom?

I'd rather prefer some error message on a stray [: or ;]
(principle of least surprise)

On 2024-03-07 20:18, minforth wrote:
> Ruvim wrote:
>
>> The accepted proposal for quotations [1] specifies only compilation
>> semantics for the words "[:" and ";]".
>
>> The expected interpretation semantics for "[: ... ;]" are ...
>
> Expected by whom?

By me and many other people.

Some Forth systems are actually implement these interpretation semantics
(some of them for the case when the current definition is absent only).

>
> I'd rather prefer some error message on a stray [: or ;] (principle of
> least surprise)

According to the principle of least surprising, interpreting a
definition name should be equivalent to interpreting of the definition body.

And if they are not equivalent — it should be for some clear and
convincing reason. For example, `exit` can be used only in a definition
body.

Let's take the definition:
: foo [: 123 . ;] execute ;

There is no a convincing reason why the following lines should not be
observationally equivalent:
foo
[: 123 . ;] execute

--
Ruvim

Ruvim wrote:
> Let's take the definition:
> : foo [: 123 . ;] execute ;

> There is no a convincing reason why the following lines should not be
> observationally equivalent:
> foo
> [: 123 . ;] execute

Then this should also be equivalent?

: BLA ." 1 " ;

BLA
" 1 "

On 2024-03-07 21:58, minforth wrote:
> Ruvim wrote:
>> Let's take the definition:
>>    : foo [: 123 . ;] execute ;
>
>> There is no a convincing reason why the following lines should not be
>> observationally equivalent:
>>    foo
>>    [: 123 . ;] execute
>
> Then this should also be equivalent?
>
> : BLA ." 1 " ;
>
> BLA
> " 1 "

Yes, `BLA` and `." 1 "`.

Yes,
BLA
and
." 1 "

It's the most expected interpretation semantics for the word `."`

Forth-2012 allows such semantics, but a portable program cannot depend
on that.

--
Ruvim

Ruvim wrote:

> On 2024-03-07 21:58, minforth wrote:
>> Ruvim wrote:
>>> Let's take the definition:
>>>    : foo [: 123 . ;] execute ;
>>
>>> There is no a convincing reason why the following lines should not be
>>> observationally equivalent:
>>>    foo
>>>    [: 123 . ;] execute
>>
>> Then this should also be equivalent?
>>
>> : BLA ." 1 " ;
>>
>> BLA
>> " 1 "

> Yes, `BLA` and `." 1 "`.

> Yes,
> BLA
> and
> ." 1 "

They are not, they do different things: BLA does not parse the input stream.

Simmilarly: [: replacing :noname is not a quotation, it is just some kind
of alias.

Anyhow IMO quotations are crippled as long as they can't access upvalues.
E.g.
: TTT { a } [: a . ;] dup execute 2 to a execute ;
1 TTT -> 1 2

But then interpreting
[: a . ;]
makes no sense.

On 2024-03-08 15:58, minforth wrote:
> Ruvim wrote:
>
>> On 2024-03-07 21:58, minforth wrote:
>>> Ruvim wrote:
>>>> Let's take the definition:
>>>>    : foo [: 123 . ;] execute ;
>>>
>>>> There is no a convincing reason why the following lines should not
>>>> be observationally equivalent:
>>>>    foo
>>>>    [: 123 . ;] execute
>>>
>>> Then this should also be equivalent?
>>>
>>> : BLA ." 1 " ;
>>>
>>> BLA
>>> " 1 "
>
>
>> Yes,
>>    BLA
>> and
>>    ." 1 "
>
> They are not, they do different things: BLA does not parse the input
> stream.

It does not matter, because the effect of the whole phrase is the same.

>
> Simmilarly: [: replacing :noname is not a quotation, it is just some kind
> of alias.

It depends on terminology. Let's assume:
*quotation*: source code for a nameless Forth definition in a special
form that is allowed to be placed inside another definition body as well
as outside of any definition body.

The run-time semantics of a quotation are to place the execution token
of the corresponding Forth definition on the stack. If the start of a
quotation is encountered by the Forth text interpreter in compilation
state, then the run-time semantics of this quotation are appended to the
current definition, otherwise these run-time semantics are performed
when the quotation end is encountered.

So, "[: ... ;]" — is a particular form for quotations. NB: another form
can be introduced via synonyms.

"[: 1 2 3 ;]" — is a quotation, regardless whether it's placed inside
another definition body, or outside.

":noname 1 2 3 ;" is not a quotation, as far as this form cannot be used
inside another definition body.

>
> Anyhow IMO quotations are crippled as long as they can't access upvalues.
> E.g.
> : TTT  { a } [: a . ;] dup execute 2 to a execute ;
> 1 TTT -> 1 2

If they could access local variables of a containing definition — they
should be called "closures", not "quotations".

And since they cannot — they are called differently (i.e. "quotations").

> But then interpreting
> [: a . ;]
> makes no sense.

If you replace a definition's name by its body, you have to use the
whole body (in this case, including the local variables declaration).
But local variables can only be used inside a definition. And this is a
clear and convincing reason for non-equivalence.

--
Ruvim

On 07-03-2024 17:18, minforth wrote:
> Ruvim wrote:
>
>> The accepted proposal for quotations [1] specifies only compilation
>> semantics for the words "[:" and ";]".
>
>> The expected interpretation semantics for "[: ... ;]" are ...
>
> Expected by whom?
>
> I'd rather prefer some error message on a stray [: or ;] (principle of
> least surprise)

In 4tH, you will. Because they are put along with a reference on the
control stack - and balanced. So:

: bla if bladibla ; then

Will trigger an error. BTW, some other Forths will too, but I'm not sure
if the mechanism is the same.

Hans Bezemer

Ruvim wrote:
>>
>> Anyhow IMO quotations are crippled as long as they can't access upvalues.
>> E.g.
>> : TTT  { a } [: a . ;] dup execute 2 to a execute ;
>> 1 TTT -> 1 2

> If they could access local variables of a containing definition — they
> should be called "closures", not "quotations".

Closures are a completely different matter. Closures own their own individual
copy/reference of their lexical environment at the time of their creation.
TTT does not have this copy.

On 2024-03-09 11:18, minforth wrote:
> Ruvim wrote:
>>>
>>> Anyhow IMO quotations are crippled as long as they can't access
>>> upvalues.
>>> E.g.
>>> : TTT  { a } [: a . ;] dup execute 2 to a execute ;
>>> 1 TTT -> 1 2
>
>> If they could access local variables of a containing definition — they
>> should be called "closures", not "quotations".
>
> Closures are a completely different matter. Closures own their own
> individual copy/reference of their lexical environment at the time
> of their creation.

> TTT does not have this copy.

Do you mean that the following will not work in your model?

: TTT2 { a -- xt } [: a dup 1+ to a . ;] dup execute dup execute ;
1 TTT2 execute -> 1 2 3

If this does not work in full, this access to outer locals is confusing
and useless.

--
Ruvim

My quotation "model" with access to upvalues works here.
Useful now and then, but they cannot pass the man-or-boy test.

As I said, pure closures are a different thing altogether.
Read it up, if you are interested.

IIRC a while ago Anton Ertl and/or Bernd Paysan have implemented
closures for gforth:

: A {: w^ k x1 x2 x3 xt: x4 xt: x5 | w^ B :} recursive
k @ 0<= IF x4 x5 f+ ELSE
B k x1 x2 x3 action-of x4 [{: B k x1 x2 x3 x4 :}L
-1 k +!
k @ B @ x1 x2 x3 x4 A ;] dup B !
execute THEN ;

10 [: 1e ;] [: -1e ;] 2dup swap [: 0e ;] A f.

They introduced a special closure syntax, so quotations with
access to locals of the enclosing function must not be confused
with closures.

Additionally for your pleasure they also introduced interpreted [: ;] ;-)

Ruvim <ruvim.pinka@gmail.com> writes:
>On 2024-03-07 16:29, Ruvim wrote:
>> The accepted proposal for quotations [1] specifies only compilation
>> semantics for the words "[:" and ";]".
>>
>> The expected interpretation semantics for "[: ... ;]" are that this
>> construct behaves like ":noname ... ;" (at least for the resulting stack
>> effect), but the system should correctly work regardless whether the
>> current definition exists (i.e., a definition that is being compiled).
>> It means, if a definition is being compiled, its compilation shall be
>> correctly suspended by "[:", and resumed after ";]" under the hood.
>>
>> Bellow are the test cases for the standardized compilation semantics,
>> and for the expected interpretation semantics.
>>
>> Interestingly, some Forth systems fail the t12 and t13 tests for the
>> standardized compilation semantics.
>>
>
>[...]
>
>>
>>
>>
>> ===== start of "quotation.test.fth"
>>
>> \ Test cases for quotations "[: ... ;]"
>>
>>
>>
>> \ Testing the compilation semantics
>>
>> \ t11
>> t{ :noname [: 123 ;] ; execute execute -> 123 }t
>>
>> : lit, postpone literal ;
>> : ([:) postpone [: ;
>> : (;]) postpone ;] ;
>>
>> \ t12
>> t{ :noname [ ([:) (;]) ] ; 0<> -> -1 }t > \ t13
>> t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t
>>
>
>t12 roughly tests that the compilation semantics for the words "[:" and
>";]" can be correctly performed programmatically.
>
>This test fails in Gforth, VfxForth, minForth.

Is there any Forth system where this test succeeds? I.e., is this
supposed failure actually common practice?

>The problem is that the system enters compilation state after executing
>"([:)". But it shall not. This action cannot be a part of compilation
>semantics at all (it's not possible to correctly specify this action,
>and it is not actually specified for the "[:" compilation semantics).

Let's see. <http://www.forth200x.org/documents/forth19-1.pdf> says:

|Suspends compiling to the current definition, starts a new nested
|definition with execution token xt, and compilation continues with
|this nested definition.

The CfV on which it is based <http://www.forth200x.org/quotations.txt>
uses the same wording.

One could argue that "compilation continues" allows switching to
compilation state, so it's not clear that this test should succeed.
But anyway, it is better to improve the wording to make this switch
explicit.

And probably similarly for ";]". We would have to check whether all
systems switch to compilation state after the compilation semantics of
;], or whether some restore the state before "[:".

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2023: https://euro.theforth.net/2023

On 2024-03-09 16:01, Anton Ertl wrote:
> Ruvim <ruvim.pinka@gmail.com> writes:
>> On 2024-03-07 16:29, Ruvim wrote:
>>> The accepted proposal for quotations [1] specifies only compilation
>>> semantics for the words "[:" and ";]".
>>>
>>> The expected interpretation semantics for "[: ... ;]" are that this
>>> construct behaves like ":noname ... ;" (at least for the resulting stack
>>> effect), but the system should correctly work regardless whether the
>>> current definition exists (i.e., a definition that is being compiled).
>>> It means, if a definition is being compiled, its compilation shall be
>>> correctly suspended by "[:", and resumed after ";]" under the hood.
>>>
>>> Bellow are the test cases for the standardized compilation semantics,
>>> and for the expected interpretation semantics.
>>>
>>> Interestingly, some Forth systems fail the t12 and t13 tests for the
>>> standardized compilation semantics.
>>>
>>
>> [...]
>>
>>>
>>>
>>>
>>> ===== start of "quotation.test.fth"
>>>
>>> \ Test cases for quotations "[: ... ;]"
>>>
>>>
>>>
>>> \ Testing the compilation semantics
>>>
>>> \ t11
>>> t{ :noname [: 123 ;] ; execute execute -> 123 }t
>>>
>>> : lit, postpone literal ;
>>> : ([:) postpone [: ;
>>> : (;]) postpone ;] ;
>>>
>>> \ t12
>>> t{ :noname [ ([:) (;]) ] ; 0<> -> -1 }t
>>> \ t13
>>> t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t
>>>
>>
>> t12 roughly tests that the compilation semantics for the words "[:" and
>> ";]" can be correctly performed programmatically.
>>
>> This test fails in Gforth, VfxForth, minForth.
>
> Is there any Forth system where this test succeeds? I.e., is this
> supposed failure actually common practice?

For example, this test succeeds in SwiftForth. Also, as I mentioned, all
these tests succeed in my implementation (I'll publish it later when I
find time).

Anyway, failure of this test shows a defect in POSTPONE in an edge case.
Although it's possible to change the implementation for "[:" in these
systems to workaround this defect.

>> The problem is that the system enters compilation state after executing
>> "([:)". But it shall not. This action cannot be a part of compilation
>> semantics at all (it's not possible to correctly specify this action,
>> and it is not actually specified for the "[:" compilation semantics).
>
> Let's see. <http://www.forth200x.org/documents/forth19-1.pdf> says:
>
> |Suspends compiling to the current definition, starts a new nested
> |definition with execution token xt, and compilation continues with
> |this nested definition.
>
> The CfV on which it is based <http://www.forth200x.org/quotations.txt>
> uses the same wording.
>
> One could argue that "compilation continues" allows switching to
> compilation state,

No.

1. State switching is always specified separately and explicitly.
2. If some behavior/effect is allowed and can be detected by a standard
program — it must be specified. So, if some behavior/effect is not
specified — it is not allowed.
3. The specified use case for "[:" supposes that the system is already
in compilation state (switching cannot be detected). Moreover,
compilation semantics always implies compilation state (of course, when
this fact can be detected by a standard program).

> so it's not clear that this test should succeed.

By my above arguments — it's clear ))

> But anyway, it is better to improve the wording to make this switch
> explicit.

It shall not be specified, since in a standard system this fact must be
impossible to detect by a standard program.

> And probably similarly for ";]". We would have to check whether all
> systems switch to compilation state after the compilation semantics of
> ;], or whether some restore the state before "[:".

Entering interpretation state after ";]" occurs only when a quotation is
used in interpretation state. But this use case is not covered by the
specification yet, so switching should not be specified.

To add this use case we should specify interpretation semantics for "[:".

To formally distinguish and specify this two use cases we can introduce
subtypes:

quotation-sys-comp => quotation-sys
quotation-sys-int => quotation-sys

And then:

"[:"
Compilation: ( C: –– quotation-sys-comp colon-sys )
Interpretation: ( C: –– quotation-sys-int colon-sys )

":]"
Compilation: ( C: quotation-sys colon-sys -- ) ( -- xt | )
If quotation-sys is quotation-sys-comp [ do some actions ]
If quotation-sys is quotation-sys-int [ do other actions, and enter
interpretation state ]

--
Ruvim

On 2024-03-09 15:35, minforth wrote:
> My quotation "model" with access to upvalues works here.
> Useful now and then, but they cannot pass the man-or-boy test.

I still cannot see how your model works in edge cases. If you have an
implementation, could you make it available for testing?

>
> As I said, pure closures are a different thing altogether.

> Read it up, if you are interested.

Thank you, I'm very familiar with the concept of closures and it's
implement options ))

>
> IIRC a while ago Anton Ertl and/or Bernd Paysan have implemented
> closures for gforth:
>
> : A {: w^ k x1 x2 x3 xt: x4 xt: x5 | w^ B :} recursive
>  k @ 0<= IF  x4 x5 f+  ELSE
>    B k x1 x2 x3 action-of x4 [{: B k x1 x2 x3 x4 :}L
>      -1 k +!
>      k @ B @ x1 x2 x3 x4 A ;] dup B !
>      execute  THEN ;
>
> 10 [: 1e ;] [: -1e ;] 2dup swap [: 0e ;] A f.
>
> They introduced a special closure syntax, so quotations with
> access to locals of the enclosing function must not be confused
> with closures.

I disagree to call this mechanism "closures". Since it cannot lexically
capture the local variables of the containing (surrounding) definition,
and it cannot create an own persistent instance of mutable environment.

Let's define the following words:

synonym e execute
: partial1 ( x xt1 -- xt2 )
2>r :noname r> r> lit, compile, postpone ;
;

Now the following lines are observationally equivalent:

:noname 2 [{: x :}d x . x 1+ to x x . ;] ; e dup e e
\ output "2 3 2 3"

:noname 2 [: {: x :} x . x 1+ to x x . ;] partial1 ; e dup e e
\ output "2 3 2 3"

So, what is in Gforth is a kind of partial application that is joint
with local variables and some means to control the live time of a
dynamically created definition.

>
> Additionally for your pleasure they also introduced interpreted [: ;] ;-)

Yes, I know. If you missed it, I pointed out a bug in Gforth, VfxForth
and minForth in POSTPONE when it's applied to "[:".

Gforth and VfxForth succeed in the tests t11, t21 and t22 (but not in
t12, t13, and t3*) from my first message in this thread. minForth — only
in t11. SwiftForth — in t1* tests.

--
Ruvim

minforth@gmx.net (minforth) writes:
>My quotation "model" with access to upvalues works here.
>Useful now and then, but they cannot pass the man-or-boy test.

Why not?

What use did you find for them?

I tried to find uses for the Gforth closures for the paper that could
not be replaced by code without this feature, but failed to find a
concise example where the benefit was convincing (as in: much easier
than the alternative).

So I asked Niklaus Wirth, who has implemented this feature in Pascal,
Modula, Modula-2, Oberon, and Oberon-2; I thought that, with keeping
this feature for so many decades, he must have a good reason. At
first I had trouble reaching him, but he came to Vienna in January
2020, and I asked him in person. He told me that he had removed this
feature from Oberon-07 (based on Oberon) in 2013, and later also
answered in writing.

Anyway, in the meantime we have found a good use for closures: Gforth
implements a variant of the actor model
<https://gforth.org/manual/Message-queues.html>. The messages are
actually xts that are executed by the target task.

Originally, in order to parameterize the passed xts, there was also a
mechanism for passing values that would be pushed on the stack of the
target task, and finally one would pass a word that would consume
these stack items.

Thanks to closures the protocol could be simplified such that only xts
are passed. If you want to parameterize the message, you pass a
closure that includes the parameters. For extra convenience, there
are one-shot heap-allocated closures (defined with :}h1) that FREE
themselves when they run.

>IIRC a while ago Anton Ertl and/or Bernd Paysan have implemented
>closures for gforth:
....
>They introduced a special closure syntax, so quotations with
>access to locals of the enclosing function must not be confused
>with closures.

For Gforth's closures, just say "Gforth closures". There are lots of
other equally valid uses of the word "closure"; Gforth closures are
based on the concept of flat closures [dybvig87,keep+12]. Gforth
closures leave flat-closure conversion to the programmer, and that
turns out to often be more convenient to program than defining outer
locals and capturing them implicitly.

@PhdThesis{dybvig87,
author = "R. Kent Dybvig",
school = "University of North Carolina at Chapel Hill",
title = "Three Implementation Models for Scheme",
year = "1987",
url = "http://agl.cs.unm.edu/~williams/cs491/three-imp.pdf",
brokenURL = "ftp://ftp.cs.indiana.edu/pub/scheme-repository/txt/3imp.ps.Z",
month = apr,
OPTannote = "Introduces flat closures under the name \emph{display
closures} in Section 4.4.2"
}

@InProceedings{keep+12,
title = "Optimizing Closures in {O}(0) Time",
author = "Andrew W. Keep and Alex Hearn and R. Kent Dybvig",
bibdate = "2015-05-09",
bibsource = "DBLP,
http://dblp.uni-trier.de/db/conf/icfp/scheme2012.html#KeepHD12",
booktitle = "Proceedings of the 2012 Annual Workshop on Scheme and
Functional Programming, Scheme 2012, Copenhagen,
Denmark, September 9-15, 2012",
publisher = "ACM",
year = "2012",
xbooktitle = "Scheme@ICFP",
editor = "Olivier Danvy",
ISBN = "978-1-4503-1895-2",
pages = "30--35",
URL = "http://doi.acm.org/10.1145/2661103",
urlwithoutbibliography = "https://www.cs.indiana.edu/~dyb/pubs/closureopt.pdf",
OPTannote = "Describes a number of optimizations for a
flat-closure implementation of Scheme."
}

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2023: https://euro.theforth.net/2023

On 2024-03-09 17:52, Ruvim wrote:
> On 2024-03-09 16:01, Anton Ertl wrote:
[...]
>> And probably similarly for ";]".  We would have to check whether all
>> systems switch to compilation state after the compilation semantics of
>> ;], or whether some restore the state before "[:".
>
> Entering interpretation state after ";]" occurs only when a quotation is
> used in interpretation state. But this use case is not covered by the
> specification yet, so switching should not be specified.
>
>
> To add this use case we should specify interpretation semantics for "[:".
>
> To formally distinguish and specify this two use cases we can introduce
> subtypes:
>
>   quotation-sys-comp => quotation-sys
>   quotation-sys-int  => quotation-sys
>
> And then:
>
> "[:"
> Compilation: ( C: –– quotation-sys-comp colon-sys )
> Interpretation: ( C: –– quotation-sys-int colon-sys )
>
> ":]"
> Compilation: ( C: quotation-sys colon-sys -- ) ( -- xt | )
> If quotation-sys is quotation-sys-comp [ do some actions ]
> If quotation-sys is quotation-sys-int [ do other actions, and enter
> interpretation state ]

An idea. A better way is to specify two sections of compilation semantic
description in the glossary entry, with different stack diagrams.
Similar to pattern matching. It allows to make description easier.

5.6.2.xxxx ":]"

Compilation: ( C: quotation-sys-comp colon-sys -- )
[ do some actions ]

Compilation: ( C: quotation-sys-int colon-sys -- ) ( S: -- xt )
[ do other actions, and enter interpretation state ]

For example, applying this approach to the "SET-ORDER" glossary entry:

16.6.1.2197 SET-ORDER

Execution: ( -1 -- )
Set the search order to the implementation-defined minimum search order.
The minimum search order shall include the words FORTH-WORDLIST and
SET-ORDER.

Execution: ( 0 -- )
Empty the search order.

Execution: ( widn ... wid1 +n -- )
Set the search order to the word lists identified by widn ... wid1.
Subsequently, word list wid1 will be searched first, and word list widn
searched last.

--
Ruvim

Anton Ertl wrote:

> minforth@gmx.net (minforth) writes:
>>My quotation "model" with access to upvalues works here.
>>Useful now and then, but they cannot pass the man-or-boy test.

> Why not?

It falls into the funarg problem category because I use a locals
stack. Reading/writing locals of the enclosing function from within
an enclosed quotation is as easy as reaching deeper down the stack.

So there is no recursion-proof activation record per xt as was
needed for proper closures.

> What use did you find for them?

Calling embedded functions (quotations) without having to pass parameters
or having to use glocbal variables can result in concise readable notation.

Here it is sometimes handy to "not" pass longish matrices around, just fetch
them from the caller's array locals one level up.

OTOH I have absolutely no uses case for closures. Also my gut feeling
tells me they don't fulfill the KISS principle.

In article <2024Mar9.130124@mips.complang.tuwien.ac.at>,
Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
>Ruvim <ruvim.pinka@gmail.com> writes:
>>On 2024-03-07 16:29, Ruvim wrote:
>>> The accepted proposal for quotations [1] specifies only compilation
>>> semantics for the words "[:" and ";]".
>>>
>>> The expected interpretation semantics for "[: ... ;]" are that this
>>> construct behaves like ":noname ... ;" (at least for the resulting stack
>>> effect), but the system should correctly work regardless whether the
>>> current definition exists (i.e., a definition that is being compiled).
>>> It means, if a definition is being compiled, its compilation shall be
>>> correctly suspended by "[:", and resumed after ";]" under the hood.
>>>
>>> Bellow are the test cases for the standardized compilation semantics,
>>> and for the expected interpretation semantics.
>>>
>>> Interestingly, some Forth systems fail the t12 and t13 tests for the
>>> standardized compilation semantics.
>>>
>>
>>[...]
>>
>>>
>>>
>>>
>>> ===== start of "quotation.test.fth"
>>>
>>> \ Test cases for quotations "[: ... ;]"
>>>
>>>
>>>
>>> \ Testing the compilation semantics
>>>
>>> \ t11
>>> t{ :noname [: 123 ;] ; execute execute -> 123 }t
>>>
>>> : lit, postpone literal ;
>>> : ([:) postpone [: ;
>>> : (;]) postpone ;] ;
>>>
>>> \ t12
>>> t{ :noname [ ([:) (;]) ] ; 0<> -> -1 }t > \ t13
>>> t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t
>>>
>>
>>t12 roughly tests that the compilation semantics for the words "[:" and
>>";]" can be correctly performed programmatically.
>>
>>This test fails in Gforth, VfxForth, minForth.
>
>Is there any Forth system where this test succeeds? I.e., is this
>supposed failure actually common practice?
>
>>The problem is that the system enters compilation state after executing
>>"([:)". But it shall not. This action cannot be a part of compilation
>>semantics at all (it's not possible to correctly specify this action,
>>and it is not actually specified for the "[:" compilation semantics).
>
>Let's see. <http://www.forth200x.org/documents/forth19-1.pdf> says:
>
>|Suspends compiling to the current definition, starts a new nested
>|definition with execution token xt, and compilation continues with
>|this nested definition.
>
>The CfV on which it is based <http://www.forth200x.org/quotations.txt>
>uses the same wording.
>
>One could argue that "compilation continues" allows switching to
>compilation state, so it's not clear that this test should succeed.
>But anyway, it is better to improve the wording to make this switch
>explicit.
>
>And probably similarly for ";]". We would have to check whether all
>systems switch to compilation state after the compilation semantics of
>;], or whether some restore the state before "[:".
>
>- anton

I replace both [: ;] and :NONAME ; with { } that leaves an xt, but
} restores the STATE that was present with {.
Note that ;] and ; do the same, as it happens that ;] restores the
compilation state present with [: .
Also ; restore the interpretation state present with :NONAME
Now [ ] and { } can be nested width impunity.

: double [ VARIABLE A ] A ! A @ { DUP + } EXECUTE ;
2 double .
4 OK
I avoid the pathological cases in the above test by restricting
words that parse ahead.

I have reworked shani's lisp using these { } constructs, and
use it also for mal (Make Another Lisp) github project.
So count me a fan of simple quotations.

Groetjes Albert

{ } and [ ] can be constructed with the four nestings.

Let's assume our Forth has the following mechanism available:

({) (}) brackets a piece of code and leaves an execution token, not dea.
Can be used outside a definition instead of :NONAME
As lightweight as possible.
Inside: compilation mode.

(( )) make sure dictionary space can be used between them by compiling an
AHEAD, or equivalent.

([) (]) New context for definitions, maybe in the middle of a word.
- tucks away onto the return stack, any data that could be spoiled by
compiling something, then recovers it again
In particular w.r.t. current wordlist, remove the information
what word is being compiled, then restore it again.
Inside: interpretation mode.

(s s) save and restore STATE.
--
Don't praise the day before the evening. One swallow doesn't make spring.
You must not say "hey" before you have crossed the bridge. Don't sell the
hide of the bear until you shot it. Better one bird in the hand than ten in
the air. First gain is a cat purring. - the Wise from Antrim -

In article <2024Mar9.183021@mips.complang.tuwien.ac.at>,
Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
>
>So I asked Niklaus Wirth, who has implemented this feature in Pascal,
>Modula, Modula-2, Oberon, and Oberon-2; I thought that, with keeping
>this feature for so many decades, he must have a good reason. At
>first I had trouble reaching him, but he came to Vienna in January
>2020, and I asked him in person. He told me that he had removed this
>feature from Oberon-07 (based on Oberon) in 2013, and later also
>answered in writing.

I find this extremely interesting. The Pascal specification is
clear. I'd not thought that - for me - an obscure feature as
closures is present in Pascal, much less that you could remove
this feature.

I always thought that closures are invented by lispers, because
in lisp you cannot write normal programs.

>- anton
--
Don't praise the day before the evening. One swallow doesn't make spring.
You must not say "hey" before you have crossed the bridge. Don't sell the
hide of the bear until you shot it. Better one bird in the hand than ten in
the air. First gain is a cat purring. - the Wise from Antrim -

albert@spenarnc.xs4all.nl writes:
> I always thought that closures are invented by lispers, because
> in lisp you cannot write normal programs.

Lisp didn't get them until fairly late in its evolution, I think. Maybe
old time Lispers here would know. But I think Scheme was the first
dialect that really made use of them. Algol 60 its own version much
earlier, in the form of call-by-name parameters.

albert@spenarnc.xs4all.nl wrote:
> I find this extremely interesting. The Pascal specification is
> clear. I'd not thought that - for me - an obscure feature as
> closures is present in Pascal, much less that you could remove
> this feature.

IIRC Pascal had nested functions from the beginning, unlike C
where they still don't exist. Closures are especially interesting
for functional programming. Since Forth is typeless and treats
execution tokens, addresses and numbers the same, functional
programming is perhaps not as interesting as in other languages.

Paul Rubin <no.email@nospam.invalid> writes:
>albert@spenarnc.xs4all.nl writes:
>> I always thought that closures are invented by lispers, because
>> in lisp you cannot write normal programs.
>
>Lisp didn't get them until fairly late in its evolution, I think. Maybe
>old time Lispers here would know. But I think Scheme was the first
>dialect that really made use of them.

Yes, Scheme was the first implemented Lisp with lexical scoping and it
allows treating closures as first-class values. However, static
scoping was the original intention of McCarthy when he designed Lisp,
and he considered the dynamically scoped implementation to be a bug.
However, Hyrum's law struck, and mainstream Lisp kept dynamic scoping,
with static scoping being added in Common Lisp.

>Algol 60 its own version much
>earlier, in the form of call-by-name parameters.

That, too, but Algol 60 also has nested functions/procedures with
static scoping. Algol 60 does not have first-class closures, though,
and some people reservere the name "closure" to first-class closures.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2023: https://euro.theforth.net/2023

On 2024-03-07 18:44, Ruvim wrote:
> On 2024-03-07 16:29, Ruvim wrote:
>> The accepted proposal for quotations [1] specifies only compilation
>> semantics for the words "[:" and ";]".
>>
[...]
>>
>>
>> ===== start of "quotation.test.fth"
>>
>> \ Test cases for quotations "[: ... ;]"
>>
>>
>>
>> \ Testing the compilation semantics
>>
>> \ t11
>> t{ :noname [: 123 ;] ; execute execute -> 123 }t
>>
>> : lit, postpone literal ;
>> : ([:) postpone  [: ;
>> : (;]) postpone  ;] ;
>>
>> \ t12
>> t{ :noname  [ ([:) (;]) ]  ;  0<>  ->  -1  }t  > \ t13
>> t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t
>>
>
> t12 roughly tests that the compilation semantics for the words "[:" and
> ";]" can be correctly performed programmatically.
>
> This test fails in Gforth, VfxForth, minForth.
>
> The problem is that the system enters compilation state after executing
> "([:)". But it shall not. This action cannot be a part of compilation
> semantics at all (it's not possible to correctly specify this action,
> and it is not actually specified for the "[:" compilation semantics).
>
> The reason of the problem is that "postpone" is implemented not quite
> correctly in these systems. After including a polyfill for "postpone"
> [2], the tests t12 and t13 are passed in Gforth.
>
> VfxForth (5.20 Alpha 1 [build 4065]) and minForth (V3.4.8) provide
> neither "find-name" nor correct "find", so "postpone" cannot be
> correctly defined by the polyfill.

In minForth, "[:" leaves some data on the return stack, and ";]"
consumes this data from the return stack (contrary to the
specification). So, it fails in the following test case:

t{ : [t14] ( -- ) 123 >r ([:) r> lit, (;]) ; immediate -> }t
t{ :noname [t14] ; execute execute -> 123 }t

--
Ruvim

Ruvim wrote:

> In minForth, "[:" leaves some data on the return stack, and ";]"
> consumes this data from the return stack (contrary to the
> specification). So, it fails in the following test case:

> t{ : [t14] ( -- ) 123 >r ([:) r> lit, (;]) ; immediate -> }t
> t{ :noname [t14] ; execute execute -> 123 }t

What specification are you talking about? Your own?

I can't find [t14] in the standard compliance tests.