In article <23-03-037@comp.compilers>,
Kaz Kylheku <864-117-4973@kylheku.com> wrote:
>On 2023-03-28, Aharon Robbins <arnold@freefriends.org> wrote:
>> Today, the C and C++ worlds are easier to program in, but it's still
>> not perfect and I don't think I'd want to do without the autotools.
>> Particularly for the less POSIX-y systems, like MinGW and OpenVMS.
>
>Counterpoint: Autotools are a real detriment to GNU project programs.
>
>When a release is cut of a typical GNU program, special steps
>are execute to prepare a tarball which has a compiled configure
>script.
>
>You cannot just do a "git clone" of a GNU program, and then run
>./configure and build. You must run some "make boostrap" nonsense, and
>that requires you to have various Autotools installed, and in specific
>versions!

This is not inherent in the autotools; it's laziness on the part of the
maintainers. For exactly this reason gawk has a very simple bootstrap.sh
program that simply does a touch on various files so that configure will
run without wanting to run the autotools.

>Most Autotools programs will not cleanly cross-compile. Autotools is the
>main reason why distro build systems use QEMU to create a virtual target
>environment with native tools and libraries, and then build the
>"cross-compiled" program as if it were native.

QEMU wasn't around when the Autotools were first designed and
implemented. Most end users don't need to cross compile either, and it
is for them that I (and other GNU maintainers, I suppose) build my
configure scripts.

Yes, the world is different today than when the autotools were
designed. No, the autotools are not perfect. I don't know of a better
alternative though. And don't tell me CMake. CMake is an abomination,
interweaving configuration with building instead of cleanly separating
the jobs. Not to mention its stupid caching which keeps you from
running a simple "make" after you've changed a single file.

>My TXR language project has a hand-written, not generated, ./configure
>script. What you get in a txr-285.tar.gz tarball is exactly what you
>get if you do a "git clone" and "git checkout txr-285", modulo
>the presence of a .git directory and differing timestamps.
>
>You just ./configure and make.

And for gawk it's ./bootstrap.sh && ./configure && make
where bootstrap.sh only takes a few seconds.

>None of my configure-time tests require the execution of a program;
>For some situations, I have developed clever tricks to avoid it.

And why should you, or anyone, be forced to develop such clever tricks?

All of this simply justifies more the approach taken by newer languages,
which is to move all the hard crap into the libraries. The language
developers do all the hard work, instead of the application developers
having to do it. This is great for people who want to just get their
job done, which includes me most of the time. However, and this is a
different discussion, it does lead to a generation of programmers who
have *no clue* as to how to do the hard stuff should they ever need to.

My opinion, of course.

Arnold
--
Aharon (Arnold) Robbins arnold AT skeeve DOT com

Hans-Peter Diettrich <DrDiettrich1@netscape.net> writes:
>My impression was that the FSF favors C and ./configure for "portable"
>code. That's why I understand that any other way is easier for the
>implementation of really portable software, that deserves no extra
>tweaks for each supported target platform, for every single program.

I have not noticed that the FSF has any preference for C, apart from C
being the lingua franca in the late 1980s and the 1990s, and arguably
for certain requirements it still is.

Now C on Unix has to fight with certain portability issues. In early
times C programs contained a config.h that the sysadmin installing a
program had to edit by hand before running make. Then came autoconf,
which generates configure files that run certain checks on the system
and fill in config.h for you; and of course, once the mechanism is
there, stuff in other files is filled in with configure, too.

It's unclear to me what you mean with "any other way is easier". The
way of manually editing config.h certainly was not easier for the
sysadmins. Not sure if it was easier for the maintainer of the
programs.

>Can somebody shed some light on the current practice of writing portable
>C/C++ software, or any other compiled language, that (hopefully) does
>not require additional human work before or after compilation for a
>specific target platform?

There are other tools like Cmake that claim to make autoconf
unnecessary, but when I looked at it, I did not find it useful for my
needs (but I forgot why).

So I'll tell you here some of what autoconf does for Gforth: Gforth is
a Forth system mostly written in Forth, but using a C substrate. Many
system differences are dealt with in the C substrate, often with the
help of autoconf. The configure.ac file describes what autoconf
should do for Gforth; it has grown to 1886 lines.

* It determines the CPU architecture and OS where the configure script
is running at, and uses that to configure some architecture-specific
stuff for Gforth, in particular how to synchronize the data and
instruction caches; later gcc acquired __builtin___clear_cache() to
do that, but at least on some platforms that builtin is broken
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93811>.

* It checks the sizes of the C integer types in order to determine the
C type for Forth's cell and double-cell types.

* It uses the OS information to configure things like the newline
sequence, the directory and path separators.

* It deals with differences between OSs, such as large (>4GB) file
support, an issue relevant in the 1990s.

* It checks for the chcon program, and, if present, uses it to "work
around SELinux brain damage"; if not present, the brain is probably
undamaged.

* It tests which of several ways is accepted by the assembler to skip
code space (needed for implementing Gforth's dynamic
superinstructions).

* It checks for the presence of various programs and library functions
needed for building Gforth, e.g. mmap() (yes, there used to be
systems that do not have mmap()). In some cases it works around the
absence, sometimes with degraded functionality; in other cases it
just reports the absence, so the sysadmin knows what to install.

That's just some of the things I see in configure.ac; there are many
bits and pieces that are too involved and/or too minor to report here.

Our portability stuff does not catch everything. E.g., MacOS on Apple
Silicon has a broken mmap() (broken as far as Gforth is concerned;
looking at POSIX, it's compliant with that, but that does not justify
this breakage; MacOS on Intel works fine, as does Linux on Apple
Silicon), an issue that's new to us; I have not yet devised a
workaround for that, but when I do, a part of the solution may use
autoconf.

Now when you write Forth code in Gforth, it tends to be quite portable
across platforms (despite Forth being a low-level language where, if
you want to see them, it's easy to see differences between 32-bit and
64-bit systems, and between different byte orders). One reason for
that is that Gforth papers over system differences (with the help of
autoconf among other things); another reason is that Gforth does not
expose many of the things where the systems are different, at least
not at the Forth level. You can use the C interface and then access
all the things that C gives access to, many of which are
system-specific, and for which tools like autoconf exist.

The story is probably similar for other languages.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

On Friday, March 31, 2023 at 4:42:14 AM UTC-7, Thomas Koenig wrote:
> gah4 <ga...@u.washington.edu> schrieb:
> > For system like Matlab and Octave, and I believe also for Python,
> > or one of many higher math languages, programs should spend
> > most of the time in the internal compiled library routines.

> They should, but sometimes they don't.

> If you run into things not covered by compiled libraries, but which
> are compute-intensive, then Matlab and (interpreted) Python run
> as slow as molasses, orders of magnitude slower than compiled code.

But then there is dynamic linking.

I have done it in R, but I believe it also works for Matlab and
Python, and is the way many packages are implemented. You write a
small C or Fortran program that does the slow part, and call it from
interpreted code.

And back to my favorite x86 assembler program:

rdtsc: rdtsc
ret

which allows high resolution timing, to find where the program
is spending too much time. Some years ago, I did this on a program
written by someone else, so I mostly didn't know the structure.
Track down which subroutines used too much time, and fix
just those.

In that case, one big time sink is building up a large matrix one
row or one column at a time, which requires a new allocation and
copy for each time. Preallocating to the final (if known) size fixes that.

But then there were some very simple operations that, as you note,
are not included and slow. Small C programs fixed those.
There are complications for memory allocation, which I avoid
by writing mine to assume (require) that all is already allocated.

(snip)

> At the company I work for, I'm told each Python project will only
> use a certain specified version of Python will never be changed for
> fear of incompatibilities - they treat each version as a new
> programming language :-|

> To bring this back a bit towards compilers - a language definition
> is an integral part of compiler writing. If

I have heard about that one.

It seems that there are non-backward compatible changes
from Python 2.x to 3.x. That is, they pretty much are different
languages.

Tradition on updating a language standard is to maintain, as much
as possible, backward compatibility. It isn't always 100%, but often
close enough. You can run Fortran 66 program on new Fortran 2018
compilers without all that much trouble. (Much of the actual problem
comes with extensions used by the old programs.)
[Python's rapid development cycle definitely has its drawbacks. Python 3
is not backward compatible with python 2 (that's why they bumped the major
version number) and they ended support for python 2 way too soon. -John]

Kaz Kylheku <864-117-4973@kylheku.com> writes:
>When a release is cut of a typical GNU program, special steps
>are execute to prepare a tarball which has a compiled configure
>script.
>
>You cannot just do a "git clone" of a GNU program, and then run
>./configure and build. You must run some "make boostrap" nonsense, and
>that requires you to have various Autotools installed, and in specific
>versions!

And the problem is?

The git repo contains only the source code, useful for developers.
The developers have stuff installed that someone who just wants to
install the program does not necessarily want to install. E.g., in
the case of Gforth, you need an older Gforth to build the kernel
images that contain Forth code compiled to an intermediate
representation. Therefore the tarballs contain a number of generated
(or, as you say, "compiled") files, e.g., the configure script, the
kernel images in case of Gforth, or the C files generated by Bison in
case of some other compilers.

If you go for the "git clone" route rather than building from the
tarball, you don't get these amenities, but have to install all the
tools that the developers use, and have to perform an additional step
(usually ./autogen.sh) to produce the configure file. "make
bootstrap" is unlikely to work, because at that stage you don't have a
Makefile.

I remember "make bootstrap" from gcc, where IIRC it compiles gcc first
(stage1) with the pre-installed C compiler, then (stage2) with the
result of stage1, and finally (stage3) again with the result of
stage2; if there is a difference between stage2 and stage3, something
is amiss.

Anyway, tl;dr: If you just want to do "./configure; make", use the
tarball.

>Most Autotools programs will not cleanly cross-compile. Autotools is tha
>main reason why distro build systems use QEMU to create a virtual target
>environment with native tools and libraries, and then build the
>"cross-compiled" program as if it were native.

Clever! Let the machine do the work, rather than having to do manual
work for each package.

>For instance, about a decade and a half ago I helped a company
>replace Windriver cruft with an in-house distribution. Windriver's
>cross-compiled Bash didn't have job control! Ctrl-Z, fg, bg stuff no
>workie. The reason was that it was just cross-compiled straight, on an
>x86 build box. It couldn't run the test to detect job control support,
>and so it defaulted it off, even though the target machine had
>"gnu-linux" in its string. In the in-house distro, my build steps for
>bash exported numerous ac_cv_... internal variables to override the bad
>defaults.

That's the way to do it.

Your idea seems to be that, when the value is not supplied, instead of
a safe default (typically resulting in not using a feature), one
should base the values on the configuration name of the system. I
think the main problem with that is that for those systems most in
need of cross-compiling the authors of the tests don't know good
values for the configuration variables; for linux-gnu systems I
usually configure and compile on the system.

>For some situations, I have developed clever tricks to avoid it. For
>instance, if you want to know the size of a data type:. Here
>is a fragment:

Great! Now we need someone who has enough time to replace the
AC_CHECK_SIZEOF autoconf macro with your technique, and a significant
part of the configuration variables that have to be supplied manually
when cross-configuring Gforth become fully automatic.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

Hans-Peter Diettrich <DrDiettrich1@netscape.net> writes:
>Often I had
>the impression that the author wanted the program not for use on Windows
>machines. Kind of "source open for specific OS only" :-(

Whatever we want, it's also a question of what the OS vendor wants.

For a Unix, there were a few hoops we had to jump through to make
Gforth work: e.g., IRIX 6.5 had a bug in sigaltstack, so we put in a
workaround for that; HP/UX's make dealt with files with the same mtime
differently from other makes, so we put in a workaround for that.
Windows, even with Cygwin, puts up many more hoops to jump through;
Bernd Paysan actually jumped through them for Gforth, but a Windows
build is still quite a bit of work, so he does that only occasionally.

It's no surprise to me that other developers don't jump through these
hoops; maybe if someone payed them for it, but why should they do it
on their own time?

As a recent example of another OS, Apple has intentionally reduced the
functionality of mmap() on MacOS on Apple silicon compared to MacOS on
Intel. As a result, the development version of Gforth does not work
on MacOS on Apple Silicon (it works fine on Linux on Apple Silicon).
I spent a day last summer on the MacOS laptop of a friend (an
extremely unpleasant experience) trying to find the problem and fix
it, and I found the problem, but time ran out before I had a working
fix (it did not help that I had to spend a lot of time on working
around things that I missed in MacOS). Since then this problem has
not reached the top of my ToDo list; and when it does, I will go for
the minimal fix, with the result that Gforth on MacOS will run without
dynamic native-code generation, i.e., slower than on Linux.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

On 4/2/23 12:04 PM, Anton Ertl wrote:

> For a Unix, there were a few hoops we had to jump through to make
> Gforth work: e.g., IRIX 6.5 had a bug in sigaltstack, so we put in a
> workaround for that; HP/UX's make dealt with files with the same mtime
> differently from other makes, so we put in a workaround for that.
> Windows, even with Cygwin, puts up many more hoops to jump through;
> Bernd Paysan actually jumped through them for Gforth, but a Windows
> build is still quite a bit of work, so he does that only occasionally.

Too bad that not all existing OS are POSIX compatible? ;-)

So my impression still is: have a language (plus library) and an
interpreter (VM, browser, compiler...) on each target system. Then
adaptations to a target system have to be made only once, for each
target, not for every single program.

Even for programs with extreme speed requirements the development can be
done from the general implementation, for tests etc., and a version
tweaked for a very specific target system, instead of the single target
version in the first place and problematic ports to many other platforms.

Of course it's up to the software developer or principal to order or
build a software for a (more or less) specific target system only, or a
primarily unbound software.

(G)FORTH IMO is a special case because it's (also) a development system.
Building (bootstrapping) a new FORTH system written in FORTH is quite
complicated, in contrast to languages with stand alone tools like
compiler, linker etc. Some newer (umbilical?) FORTH versions also
compile to native code.

DoDi

Hans-Peter Diettrich <DrDiettrich1@netscape.net> writes:
>On 4/2/23 12:04 PM, Anton Ertl wrote:
>
>> For a Unix, there were a few hoops we had to jump through to make
>> Gforth work: e.g., IRIX 6.5 had a bug in sigaltstack, so we put in a
>> workaround for that; HP/UX's make dealt with files with the same mtime
>> differently from other makes, so we put in a workaround for that.
>> Windows, even with Cygwin, puts up many more hoops to jump through;
>> Bernd Paysan actually jumped through them for Gforth, but a Windows
>> build is still quite a bit of work, so he does that only occasionally.
>
>Too bad that not all existing OS are POSIX compatible? ;-)

Like many standards, POSIX is a subset of the functionality that
programs use. Windows NT used to have a POSIX subsystem in order to
make WNT comply with FIPS 151-2 needed to make WNT eligible for
certain USA government purchases. From what I read, it was useful for
that, but not much else.

>So my impression still is: have a language (plus library) and an
>interpreter (VM, browser, compiler...) on each target system. Then
>adaptations to a target system have to be made only once, for each
>target, not for every single program.

You mean: Write your program in Java, Python, Gforth, or the like?
Sure, they deal with compatibility problems for you, but you may want
to do things (or have performance) that they do not offer, or only
offer through a C interface (and in the latter case you run into the
C-level compatibility again).

>Even for programs with extreme speed requirements the development can be
>done from the general implementation, for tests etc., and a version
>tweaked for a very specific target system, instead of the single target
>version in the first place and problematic ports to many other platforms.

Well, if you go that route, the result can easily be that your program
does not run on Windows. Especially for GNU programs: The primary
goal is that they run on GNU. Any effort spent on a Windows port is
extra effort that not everybody has time for.

>(G)FORTH IMO is a special case because it's (also) a development system.
>Building (bootstrapping) a new FORTH system written in FORTH is quite
>complicated, in contrast to languages with stand alone tools like
>compiler, linker etc.

Not really. Most self-respecting languages have their compiler(s)
implemented in the language itself, resulting in having to bootstrap.
AFAIK the problem Gforth has with Windows is not the bootstrapping;
packaging and installation are different than for Unix.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

On 2023-04-05, Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
> Hans-Peter Diettrich <DrDiettrich1@netscape.net> writes:
>>On 4/2/23 12:04 PM, Anton Ertl wrote:
>>
>>> For a Unix, there were a few hoops we had to jump through to make
>>> Gforth work: e.g., IRIX 6.5 had a bug in sigaltstack, so we put in a
>>> workaround for that; HP/UX's make dealt with files with the same mtime
>>> differently from other makes, so we put in a workaround for that.
>>> Windows, even with Cygwin, puts up many more hoops to jump through;
>>> Bernd Paysan actually jumped through them for Gforth, but a Windows
>>> build is still quite a bit of work, so he does that only occasionally.
>>
>>Too bad that not all existing OS are POSIX compatible? ;-)
>
> Like many standards, POSIX is a subset of the functionality that
> programs use. Windows NT used to have a POSIX subsystem in order to
> make WNT comply with FIPS 151-2 needed to make WNT eligible for
> certain USA government purchases. From what I read, it was useful for
> that, but not much else.

The best POSIX subsystem for Windows is arguably Cygwin. It has
quite a rich POSIX functionality. Not only that, but ANSI terminal
functionality: its I/O system contains a layer which translates
ANSI escape sequences into Windows Console API calls.

Yuo can take a program written on Linux which uses termios to put the
TTY in raw mode, and ANSI escapes to control the screen, and it will
work on Cygwin.

One of its downsides downside is that Cygwin has poor performance
(mainly in the area of file access).

The other downside of Cygwin is that it implements certain conventions
that are at odds with "native" Windows.

In 2016 I started a small project called Cygnal (Cygwin Native
Application Library) to fix problems in this second category,
creating a fork of the Cygwin DLL.

https://www.kylheku.com/cygnal

>>(G)FORTH IMO is a special case because it's (also) a development system.
>>Building (bootstrapping) a new FORTH system written in FORTH is quite
>>complicated, in contrast to languages with stand alone tools like
>>compiler, linker etc.
>
> Not really. Most self-respecting languages have their compiler(s)
> implemented in the language itself, resulting in having to bootstrap.

You can avoid the chicken-and-egg problem that requires boostrapping by
using a host language to implement an interpreter for the target
language. That interpreter can then directly execute the compiler, which
can compile itself and other parts of the run-time, as needed.

It's still a kind of boostrapping, but at no point do you need a
pre-built binary of the target language compiler to build that compiler;
you just need an implementation of a host language.

This works quite well when the host language is good for writing
interpreters, and the target one for compiler work, and also when it's
useful to have an interpreter even when compilation is available.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 4/5/23 6:30 PM, Anton Ertl wrote:
> Hans-Peter Diettrich <DrDiettrich1@netscape.net> writes:

> You mean: Write your program in Java, Python, Gforth, or the like?
> Sure, they deal with compatibility problems for you, but you may want
> to do things (or have performance) that they do not offer, or only
> offer through a C interface (and in the latter case you run into the
> C-level compatibility again).

Except the library also is portable ;-)

Else you end up with:
Program runs only on systems with libraries X, Y, Z installed.

>> (G)FORTH IMO is a special case because it's (also) a development system.
>> Building (bootstrapping) a new FORTH system written in FORTH is quite
>> complicated, in contrast to languages with stand alone tools like
>> compiler, linker etc.
>
> Not really. Most self-respecting languages have their compiler(s)
> implemented in the language itself, resulting in having to bootstrap.

The FORTH compiler also is part of the current monolithic framework.
Replacing a WORD has immediate impact on the just running compiler and
everything else. A bug can make the current system crash immediately,
without diagnostics. Else the current WORDs can not be replaced
immediately, only after a full compilation and only by code that depends
on neither the old nor the new framrwork.

> AFAIK the problem Gforth has with Windows is not the bootstrapping;
> packaging and installation are different than for Unix.

Isn't that the same problem with every language?

DoDi

Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
> Most self-respecting languages have their compiler(s)
> implemented in the language itself, resulting in having to bootstrap.

This is a bit complicated for GCC and LLVM.

For both, the middle end (and back end) is implemented in C++,
so a C++ interface at class level is required, and that is a
bit daunting.

Examples: Gnat (GCC's Ada front end) is written in Ada, and its
Modula-2 front end is written in Modula-2. On the other hand,
the Fortran front end is written in C++ (well, mostly C with
C++ features hidden behind macros).

The very first Fortran compiler, of course, was written in
assembler.
[It was, but Fortran H, the 1960s optimizing compiler for S/360 was
written in Fortran with a few data structure extensions. -John]