In discussions about STATE and STATE-smart words, some participants
have wanted to use STATE as an indicator that there is a current colon
definition, and sometimes even claimed that these two concepts are
indistinguishable. The existence of [ and ] means that you can have
interpret state while there is a current definition, and you can have
compile state while there is no current colon definition. E.g.

: foo [ ' + compile, ] ;
] + [

The first line compiles "+" in interpret state while the definition of
FOO is current.

The second line compiles "+" while there is no current colon
definition.

In development Gforth the system now knows whether there is a current
colon definition, and produces warnings accordingly:

: foo [ ' + compile, ] ; ok

] + [
*terminal*:3:3: warning: Compiling outside a definition ok

Gforth still compiles code in the latter case, so you cannot use

create bar ] + - [

instead of

create bar ' + , ' - ,

as in the ITC days (except if you use gforth-itc).

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2023: https://euro.theforth.net/2023

In article <2023Jul31.084315@mips.complang.tuwien.ac.at>
logging-data="3388010"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19U9e2GCi3j0Kqk85RvPmu8",
Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
>In discussions about STATE and STATE-smart words, some participants
>have wanted to use STATE as an indicator that there is a current colon
>definition, and sometimes even claimed that these two concepts are
>indistinguishable.

I am a proponent of diminishing the ubiquitiness of STATE.
There is no reason STATE should be inspected in number, or string
or character or lambda denotations (although I do it at present in
ciforth) for they signal their presence by leaving something
on the stack.
As an alternative [ ] are only used to inform the INTERPRET word to do
interpreting or compiling. As soon as a word (e.g. a prefix & that
leaves a character ) leaves something on the stack in compile state
INTERPRET detects this and compiles it as a literal.
Obviously a word can only leave something on the stack while
compiling, if that word is immediate. The data stack can no
longer be used as a compilation stack, so the compilation
stack has to be separate.

So ] [ are messages to the INTERPRET object. If the word STATE
exists, it is only present in the implementation of these three
words.

Those ideas has inspired ciforth to restrict STATE to
] [ INTERPRET LITERAL DLITERAL

>--
>M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html

Groetjes Albert
--
Don't praise the day before the evening. One swallow doesn't make spring.
You must not say "hey" before you have crossed the bridge. Don't sell the
hide of the bear until you shot it. Better one bird in the hand than ten in
the air. First gain is a cat spinning. - the Wise from Antrim -

none albert schrieb am Montag, 31. Juli 2023 um 16:45:46 UTC+2:
> In article <2023Jul3...@mips.complang.tuwien.ac.at>
> logging-data="3388010"; mail-complaints-to="ab...@eternal-september.org"; posting-account="U2FsdGVkX19U9e2GCi3j0Kqk85RvPmu8",
> Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> >In discussions about STATE and STATE-smart words, some participants
> >have wanted to use STATE as an indicator that there is a current colon
> >definition, and sometimes even claimed that these two concepts are
> >indistinguishable.
> I am a proponent of diminishing the ubiquitiness of STATE.
> There is no reason STATE should be inspected in number, or string
> or character or lambda denotations (although I do it at present in
> ciforth) for they signal their presence by leaving something
> on the stack.
> As an alternative [ ] are only used to inform the INTERPRET word to do
> interpreting or compiling. As soon as a word (e.g. a prefix & that
> leaves a character ) leaves something on the stack in compile state
> INTERPRET detects this and compiles it as a literal.
> Obviously a word can only leave something on the stack while
> compiling, if that word is immediate. The data stack can no
> longer be used as a compilation stack, so the compilation
> stack has to be separate.
>
> So ] [ are messages to the INTERPRET object. If the word STATE
> exists, it is only present in the implementation of these three
> words.
>
> Those ideas has inspired ciforth to restrict STATE to
> ] [ INTERPRET LITERAL DLITERAL

IMO the standard over-restricts STATE to be a binary flag.

It is trivial to extend it to an integer i.e. a multi-value flag
and encode different compilation environments of [ or ] et al

This would be somewhat similar to "messages to the interpreter".

On 2/08/2023 4:42 am, minforth wrote:
> none albert schrieb am Montag, 31. Juli 2023 um 16:45:46 UTC+2:
>> In article <2023Jul3...@mips.complang.tuwien.ac.at>
>> logging-data="3388010"; mail-complaints-to="ab...@eternal-september.org"; posting-account="U2FsdGVkX19U9e2GCi3j0Kqk85RvPmu8",
>> Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> In discussions about STATE and STATE-smart words, some participants
>>> have wanted to use STATE as an indicator that there is a current colon
>>> definition, and sometimes even claimed that these two concepts are
>>> indistinguishable.
>> I am a proponent of diminishing the ubiquitiness of STATE.
>> There is no reason STATE should be inspected in number, or string
>> or character or lambda denotations (although I do it at present in
>> ciforth) for they signal their presence by leaving something
>> on the stack.
>> As an alternative [ ] are only used to inform the INTERPRET word to do
>> interpreting or compiling. As soon as a word (e.g. a prefix & that
>> leaves a character ) leaves something on the stack in compile state
>> INTERPRET detects this and compiles it as a literal.
>> Obviously a word can only leave something on the stack while
>> compiling, if that word is immediate. The data stack can no
>> longer be used as a compilation stack, so the compilation
>> stack has to be separate.
>>
>> So ] [ are messages to the INTERPRET object. If the word STATE
>> exists, it is only present in the implementation of these three
>> words.
>>
>> Those ideas has inspired ciforth to restrict STATE to
>> ] [ INTERPRET LITERAL DLITERAL
>
> IMO the standard over-restricts STATE to be a binary flag.
>
> It is trivial to extend it to an integer i.e. a multi-value flag
> and encode different compilation environments of [ or ] et al
>
> This would be somewhat similar to "messages to the interpreter".

Anyone who has developed a forth will likely have an idea or two how
things could be improved. The TC restricted many things with the aim
of securing general support. ISTM this leaves prospective designers
with two options: risk fragmenting what support still exists for a
standard - or leave for the sake of it and their own sanity.

On Monday, 31 July 2023 at 08:07:10 UTC+1, Anton Ertl wrote:
> In discussions about STATE and STATE-smart words, some participants
> have wanted to use STATE as an indicator that there is a current colon
> definition, and sometimes even claimed that these two concepts are
> indistinguishable. The existence of [ and ] means that you can have
> interpret state while there is a current definition, and you can have
> compile state while there is no current colon definition. E.g.
>
> : foo [ ' + compile, ] ;
> ] + [
>
> The first line compiles "+" in interpret state while the definition of
> FOO is current.
>
> The second line compiles "+" while there is no current colon
> definition.
>
> In development Gforth the system now knows whether there is a current
> colon definition, and produces warnings accordingly:
>
> : foo [ ' + compile, ] ; ok
>
> ] + [
> *terminal*:3:3: warning: Compiling outside a definition ok
>
> Gforth still compiles code in the latter case, so you cannot use
>
> create bar ] + - [
>
> instead of
>
> create bar ' + , ' - ,
>
> as in the ITC days (except if you use gforth-itc).
>
> - anton
> --
> M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: https://forth-standard.org/
> EuroForth 2023: https://euro.theforth.net/2023

In my opinion, state is really for the compiler decide which action to carry out
next ie execute or compile. I think [ and ] are necessary as sometimes you need
to compute values on the fly.

No one foresaw that in the future someone would think ] + [ should produce an
answer. I think such code falls under the umbrella of undefined behaviour anyway.

Should state mean there's a current definition being compiled ?

It seems a reasonable expectation. But given that Anton's asking I feel its a trick question.
So on that basis it might be easier to have a separate variable keeping tabs on whether we are
compiling a definition or not.

In article <uacgle$3udtr$1@dont-email.me>, dxforth <dxforth@gmail.com> wrote:
>On 2/08/2023 4:42 am, minforth wrote:
>> none albert schrieb am Montag, 31. Juli 2023 um 16:45:46 UTC+2:
>>> In article <2023Jul3...@mips.complang.tuwien.ac.at>
>>> logging-data="3388010";
>mail-complaints-to="ab...@eternal-september.org";
>posting-account="U2FsdGVkX19U9e2GCi3j0Kqk85RvPmu8",
>>> Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>> In discussions about STATE and STATE-smart words, some participants
>>>> have wanted to use STATE as an indicator that there is a current colon
>>>> definition, and sometimes even claimed that these two concepts are
>>>> indistinguishable.
>>> I am a proponent of diminishing the ubiquitiness of STATE.
>>> There is no reason STATE should be inspected in number, or string
>>> or character or lambda denotations (although I do it at present in
>>> ciforth) for they signal their presence by leaving something
>>> on the stack.
>>> As an alternative [ ] are only used to inform the INTERPRET word to do
>>> interpreting or compiling. As soon as a word (e.g. a prefix & that
>>> leaves a character ) leaves something on the stack in compile state
>>> INTERPRET detects this and compiles it as a literal.
>>> Obviously a word can only leave something on the stack while
>>> compiling, if that word is immediate. The data stack can no
>>> longer be used as a compilation stack, so the compilation
>>> stack has to be separate.
>>>
>>> So ] [ are messages to the INTERPRET object. If the word STATE
>>> exists, it is only present in the implementation of these three
>>> words.
>>>
>>> Those ideas has inspired ciforth to restrict STATE to
>>> ] [ INTERPRET LITERAL DLITERAL
>>
>> IMO the standard over-restricts STATE to be a binary flag.
>>
>> It is trivial to extend it to an integer i.e. a multi-value flag
>> and encode different compilation environments of [ or ] et al
>>
>> This would be somewhat similar to "messages to the interpreter".
>
>Anyone who has developed a forth will likely have an idea or two how
>things could be improved. The TC restricted many things with the aim
>of securing general support. ISTM this leaves prospective designers
>with two options: risk fragmenting what support still exists for a
>standard - or leave for the sake of it and their own sanity.
>
I hope it is clear that I want to go on the other direction.
In the ciforth model STATE is absolutely standard, but its use
is discouraged. The last thing I want to do is assign other functions
to STATE.
The main reason we want to have STATE is ABORT" ." and some such.
The splitting of ." into ." and .( and CHAR into CHAR and [CHAR]
and CTRL into CTRL and [CTRL] is highly artificial and S" is a pitfall.

The solution is to have strings delimited by double quotes that are
valid in execution and compilation mode, similar to what we are accustomed
to with numbers, and compatible with every sane computer language.
Fortunately they have made it into the 2012 standard.

Groetjes Albert
--
Don't praise the day before the evening. One swallow doesn't make spring.
You must not say "hey" before you have crossed the bridge. Don't sell the
hide of the bear until you shot it. Better one bird in the hand than ten in
the air. First gain is a cat spinning. - the Wise from Antrim -

On 2/08/2023 11:04 pm, albert wrote:
> In article <uacgle$3udtr$1@dont-email.me>, dxforth <dxforth@gmail.com> wrote:
>> On 2/08/2023 4:42 am, minforth wrote:
>>> none albert schrieb am Montag, 31. Juli 2023 um 16:45:46 UTC+2:
>>>> In article <2023Jul3...@mips.complang.tuwien.ac.at>
>>>> logging-data="3388010";
>> mail-complaints-to="ab...@eternal-september.org";
>> posting-account="U2FsdGVkX19U9e2GCi3j0Kqk85RvPmu8",
>>>> Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>>> In discussions about STATE and STATE-smart words, some participants
>>>>> have wanted to use STATE as an indicator that there is a current colon
>>>>> definition, and sometimes even claimed that these two concepts are
>>>>> indistinguishable.
>>>> I am a proponent of diminishing the ubiquitiness of STATE.
>>>> There is no reason STATE should be inspected in number, or string
>>>> or character or lambda denotations (although I do it at present in
>>>> ciforth) for they signal their presence by leaving something
>>>> on the stack.
>>>> As an alternative [ ] are only used to inform the INTERPRET word to do
>>>> interpreting or compiling. As soon as a word (e.g. a prefix & that
>>>> leaves a character ) leaves something on the stack in compile state
>>>> INTERPRET detects this and compiles it as a literal.
>>>> Obviously a word can only leave something on the stack while
>>>> compiling, if that word is immediate. The data stack can no
>>>> longer be used as a compilation stack, so the compilation
>>>> stack has to be separate.
>>>>
>>>> So ] [ are messages to the INTERPRET object. If the word STATE
>>>> exists, it is only present in the implementation of these three
>>>> words.
>>>>
>>>> Those ideas has inspired ciforth to restrict STATE to
>>>> ] [ INTERPRET LITERAL DLITERAL
>>>
>>> IMO the standard over-restricts STATE to be a binary flag.
>>>
>>> It is trivial to extend it to an integer i.e. a multi-value flag
>>> and encode different compilation environments of [ or ] et al
>>>
>>> This would be somewhat similar to "messages to the interpreter".
>>
>> Anyone who has developed a forth will likely have an idea or two how
>> things could be improved. The TC restricted many things with the aim
>> of securing general support. ISTM this leaves prospective designers
>> with two options: risk fragmenting what support still exists for a
>> standard - or leave for the sake of it and their own sanity.
>>
> I hope it is clear that I want to go on the other direction.
> In the ciforth model STATE is absolutely standard, but its use
> is discouraged. The last thing I want to do is assign other functions
> to STATE.
> The main reason we want to have STATE is ABORT" ." and some such.
> The splitting of ." into ." and .(

State-smartness was about convenience. I got tired of typing .( for no
benefit whatsoever.

> and CHAR into CHAR and [CHAR]
> and CTRL into CTRL and [CTRL]

That was existing practice by Forth Inc. Also a fine example of factoring.
Thus far they haven't bugged me.

> S" is a pitfall.

it is now thanks to the differing camps
> The solution is to have strings delimited by double quotes that are
> valid in execution and compilation mode, similar to what we are accustomed
> to with numbers, and compatible with every sane computer language.
> Fortunately they have made it into the 2012 standard.

Prefixed numbers are common outside forth as are C style escapes in strings.
This is no particular reason to use them.

"The fact that an opinion has been widely held is no evidence whatever
that it is not utterly absurd." - Bertrand Russell

NN <november.nihal@gmail.com> writes:
>In my opinion, state is really for the compiler decide which action to carry out
>next ie execute or compile.

In a standard system it actually tells the text interpreter whether to
perform the interpretation or compilation semantics of the
text-interpreted word.

The difference between what I wrote and what you wrote (or some other
positions) has been the source of many discussions over the years.

>No one foresaw that in the future someone would think ] + [ should produce an
>answer.

It's not clear which future you have in mind. The technique

of writing

| create bar ] + - [
|
|instead of
|
| create bar ' + , ' - ,

has been used in indirect-threaded code systems and has been thought
of an explicitly rejected by Forth-94 ff.

>I think such code falls under the umbrella of undefined behaviour anyway.

In standard Forth, "ambiguous condition". But a high-quality Forth
system should still behave sensibly. A recent case, which actually
inspired this thread, uncovered that the following sequence

] drop [ : foo 100 ;

resulted in a corrupt FOO (in gforth-fast):

123 foo .s <1> 100

I.e., the "] drop [" influenced what is compiled in the colon
definition. Despite this (and the case that made us find the problem)
being ambiguous conditions, we still considered Gforth's behaviour a
bug, fixed it, and, in addition, added a warning when compiling code
outside a colon definition.

>Should state mean there's a current definition being compiled ?

No. Admittedly in most cases the compilation semantics of a word
compiles some code, so performing them outside a colon definition is
an ambiguous condition, but:

1) There are also standard words whose compilation semantics do not
compile some code, and you can add more such words. E.g.:

: foo ." foo" ; immediate
] foo [ \ prints "foo"

2) Just because we are in interpretation state does not mean that no
code is being compiled:

: bar postpone drop ;
bar

>But given that Anton's asking I feel its a trick question.

It seems to me that you were asking. I discussed and demonstrated the
difference between the concepts, and your question has inspired me to
some more discussion and demonstrations.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2023: https://euro.theforth.net/2023

On Monday, July 31, 2023 at 12:07:10 AM UTC-7, Anton Ertl wrote:
> In discussions about STATE and STATE-smart words, some participants
> have wanted to use STATE as an indicator that there is a current colon
> definition, and sometimes even claimed that these two concepts are
> indistinguishable. The existence of [ and ] means that you can have
> interpret state while there is a current definition, and you can have
> compile state while there is no current colon definition. E.g.
>
> : foo [ ' + compile, ] ;
> ] + [
>
> The first line compiles "+" in interpret state while the definition of
> FOO is current.
>
> The second line compiles "+" while there is no current colon
> definition.

This topic has already been discussed:
https://groups.google.com/g/comp.lang.forth/c/y96tQf_iOSk/m/WhpmjMTeAgAJ

On Wednesday, 2 August 2023 at 18:20:59 UTC+1, Anton Ertl wrote:
> NN <novembe...@gmail.com> writes:
> >In my opinion, state is really for the compiler decide which action to carry out
> >next ie execute or compile.
> In a standard system it actually tells the text interpreter whether to
> perform the interpretation or compilation semantics of the
> text-interpreted word.
>
> The difference between what I wrote and what you wrote (or some other
> positions) has been the source of many discussions over the years.
> >No one foresaw that in the future someone would think ] + [ should produce an
> >answer.
> It's not clear which future you have in mind. The technique
>
> of writing
> | create bar ] + - [
> |
> |instead of
> |
> | create bar ' + , ' - ,
> has been used in indirect-threaded code systems and has been thought
> of an explicitly rejected by Forth-94 ff.
> >I think such code falls under the umbrella of undefined behaviour anyway.
> In standard Forth, "ambiguous condition". But a high-quality Forth
> system should still behave sensibly. A recent case, which actually
> inspired this thread, uncovered that the following sequence
>
> ] drop [ : foo 100 ;
>
> resulted in a corrupt FOO (in gforth-fast):
>
> 123 foo .s <1> 100
>
> I.e., the "] drop [" influenced what is compiled in the colon
> definition. Despite this (and the case that made us find the problem)
> being ambiguous conditions, we still considered Gforth's behaviour a
> bug, fixed it, and, in addition, added a warning when compiling code
> outside a colon definition.
> >Should state mean there's a current definition being compiled ?
> No. Admittedly in most cases the compilation semantics of a word
> compiles some code, so performing them outside a colon definition is
> an ambiguous condition, but:
>
> 1) There are also standard words whose compilation semantics do not
> compile some code, and you can add more such words. E.g.:
>
> : foo ." foo" ; immediate
> ] foo [ \ prints "foo"
>
> 2) Just because we are in interpretation state does not mean that no
> code is being compiled:
>
> : bar postpone drop ;
> bar
> >But given that Anton's asking I feel its a trick question.
> It seems to me that you were asking. I discussed and demonstrated the
> difference between the concepts, and your question has inspired me to
> some more discussion and demonstrations.
> - anton
> --
> M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: https://forth-standard.org/
> EuroForth 2023: https://euro.theforth.net/2023

Hi Anton,

(1) I should have been more precise in my words. When I used compiler I really
meant text interpreter.

(2)

(a) create bar ' + , ' - ,
(b) create bar ] + - [

If the result of (a) and (b) is equivalent then I can see how they might be accepted.
You haven't said how bar is to be used . You could be using bar as a 2variable
I dont know why forth-94 rejected it but I am sure they had their reasons.
(3)

] drop [ : foo 100 ;

I agree with the fix that compiling outside a colon definition should be flagged as it doesn't make sense.
And given that an error is thrown foo wont compile until the prior drop is fixed.

(4)

: foo ." foo" ; immediate
] foo [ \ prints "foo"

This should be rejected as its compilation outside a definition.( as per 3)

(5)

: bar postpone drop ;
bar

This should be rejected as its compilation outside a definition.( as per 3)
The definition should compile, but at the point of 'bar' it should fail.

I have no examples or counter-examples to offer

I am guessing that in the past forth compilers were simple and caught only the
minimal expected to be caught. But as times have moved on and programmers
demand more, the compilers, are being tuned to catch stuff that would have been
allowed to slip by. This in turn has forced compiler writers to question if something
is acceptable code (or not). But just because something is acceptable doesnt
mean its meaningfull code.

IMO, this whole problem goes away if the input is always compiled, and then only executed if a word was not defined.

That can be determined easily enough if LAST changed or not. If LAST changed, don't execute. Else, execute and reset.

Of course, this approach is probably not "ANSI Compliant", but it works for me.

--- something like this:
void go(const char *cp) {
byte *cH=H, *cL=L;
if (parse(cp) == 0) { H=cH; L=cL; return; } // error
if (cL == L) { CComma(RET); H=cH; run(H); }
} ---

In this world, there is no need for a STATE at all.

(I expect some people will have a problem with this ...)

On Monday, August 28, 2023 at 4:33:44 PM UTC+2, ccur...@gmail.com wrote:
> IMO, this whole problem goes away if the input is always compiled, and then only executed if a word was not defined.
4tH either compiles or executes. It doesn't have a dictionary. A program is like one huge word.

> Of course, this approach is probably not "ANSI Compliant", but it works for me.
I feel you, brother. But you can come pretty close.

> In this world, there is no need for a STATE at all.
Correct. There isn't even a STATE word in 4tH.
> (I expect some people will have a problem with this ...)
... tell me about it <sigh>..

Hans Bezemer