When we last heard from the merry band of Millers, they were looking for
substantial funding from a VC or similar. I suppose that if they had
gotten it, we would have heard, so I guess they haven't.

But I think there are things they could do to move forward even without
a large investment. For example, they could develop an FPGA based
system, even if it required multiple FPGAs on a custom circuit board for
not huge amounts of money. Whether this is worthwhile, I cannot say.

Anyway, has all development stopped? Or is their "sweat equity" model
still going on?

Inquiring minds want to know.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
>When we last heard from the merry band of Millers, they were looking for
>substantial funding from a VC or similar. I suppose that if they had
>gotten it, we would have heard, so I guess they haven't.
>
>But I think there are things they could do to move forward even without
>a large investment. For example, they could develop an FPGA based
>system, even if it required multiple FPGAs on a custom circuit board for
>not huge amounts of money. Whether this is worthwhile, I cannot say.
>

There might even be some way of renting time on a real
emulator from cadence (Palladium) or synopsys (Zebu).

Although in my experience those who have them use them
24x7.

On 12/13/2023 10:25 AM, Stephen Fuld wrote:
> When we last heard from the merry band of Millers, they were looking for
> substantial funding from a VC or similar.  I suppose that if they had
> gotten it, we would have heard, so I guess they haven't.
>
> But I think there are things they could do to move forward even without
> a large investment.  For example, they could develop an FPGA based
> system, even if it required multiple FPGAs on a custom circuit board for
> not huge amounts of money.  Whether this is worthwhile, I cannot say.
>
> Anyway, has all development stopped?  Or is their "sweat equity" model
> still going on?
>
> Inquiring minds want to know.
>

Yeah, don't think I have seen anything from Ivan on here in a while...

In my case, I was doing everything on Spartan-7 and Artix-7 boards, and
had OK results (within the limits of what is possible on an FPGA).

Kinda wish it could be faster, but alas.

Sadly, anything much bigger (or faster) than the XC7A200T actually
requires paying money for the non-free version of Vivado...

Ironically, this makes the XC7A200T more valuable in a way than the
XC7K325T, as while technically smaller and weaker, it is basically the
biggest FPGA one can get before needing to hand over absurd amounts of
money to AMD/Xilinx.

Well, there is also the XC7K70T, which is technically faster, but has a
lot less LUTs.

And the XC7K160T, which is faster and only slightly smaller, but
significantly more expensive.

But, if one wants an FPGA they could "afford to put in stuff and
potentially have someone be willing to buy it", would need to aim a
little lower here. Can't put all that fancy of a soft-processor in an
XC7S50 or XC7A35T, but could be more reasonable to put into
consumer-electronics devices.

Though, sadly, a soft-processor can't really match something like an ARM
chip in terms of performance per dollar. Though, would be nice if
*someone* could dethrone ARM in terms of perf/$ (RISC-V holds promise,
but only really if someone releases a chip that is both cheap and fast).

And, custom ASIC's are only really an option if one has a huge amount of
money up-front.

Though printable electronics with semi-conductive ink seems promising,
but even here, the ink is stupid expensive and one would still need to
build a special-purpose printer to be able to make use of it (and not
particularly high-density; so probably would be physically much larger
and slower than a design running on an FPGA).

Though, not really sure what the densities or clock speeds of printed
electronics are like.

....

Though, at least in my case, it is all mostly a hobby project.
Unless "someone with a lot of money" thinks it is cool.

....

Though, one possible feature in my case of my project being FPGA based,
is that theoretically I could get one of those gameboy-like FPGA-based
emulator things and port my stuff to this, though most of these devices
seem to be based around the Cyclone V for whatever reason, ...

On Wed, 13 Dec 2023 08:25:39 -0800, Stephen Fuld
<sfuld@alumni.cmu.edu.invalid> wrote:

>When we last heard from the merry band of Millers, they were looking for
>substantial funding from a VC or similar. I suppose that if they had
>gotten it, we would have heard, so I guess they haven't.
>
>But I think there are things they could do to move forward even without
>a large investment. For example, they could develop an FPGA based
>system, even if it required multiple FPGAs on a custom circuit board for
>not huge amounts of money. Whether this is worthwhile, I cannot say.
>
>Anyway, has all development stopped? Or is their "sweat equity" model
>still going on?
>
>Inquiring minds want to know.

There was a post, ostensibly from Ivan, in their web forum just a few
days ago. No news though - just an acknowledgement of another user's
post.

Last I heard, the next (current?) round of financing was - at least in
part - to be used for FPGA "proof of concept" implementations.

Problem is the Mill really is a SoC, and (to me at least) the design
appears to be so complex that it would require a large, top-of-line
(read "expensive") FPGA to fit all the functionality.

Then there is their idea that everything - from VHDL to software build
toolchain to system software - be automatically generated from a
simple functional specification. Getting THAT right is likely proving
far more difficult than simply implementing a fixed design in an FPGA.

YMMV,
George

On 12/15/2023 11:48 AM, George Neuner wrote:
> On Wed, 13 Dec 2023 08:25:39 -0800, Stephen Fuld
> <sfuld@alumni.cmu.edu.invalid> wrote:
>
>> When we last heard from the merry band of Millers, they were looking for
>> substantial funding from a VC or similar. I suppose that if they had
>> gotten it, we would have heard, so I guess they haven't.
>>
>> But I think there are things they could do to move forward even without
>> a large investment. For example, they could develop an FPGA based
>> system, even if it required multiple FPGAs on a custom circuit board for
>> not huge amounts of money. Whether this is worthwhile, I cannot say.
>>
>> Anyway, has all development stopped? Or is their "sweat equity" model
>> still going on?
>>
>> Inquiring minds want to know.
>
> There was a post, ostensibly from Ivan, in their web forum just a few
> days ago. No news though - just an acknowledgement of another user's
> post.
>
>
> Last I heard, the next (current?) round of financing was - at least in
> part - to be used for FPGA "proof of concept" implementations.
>
> Problem is the Mill really is a SoC, and (to me at least) the design
> appears to be so complex that it would require a large, top-of-line
> (read "expensive") FPGA to fit all the functionality.
>

Yeah. the lower end isn't cheap, the upper end is absurd...

For FPGA's over $1k, almost makes more sense to ignore that they exist
(also this appears to be around the cutoff point for the free version of
Vivado as well; but one would have thought Xilinx would have already
gotten their money by someone having bought the FPGA?...).

> Then there is their idea that everything - from VHDL to software build
> toolchain to system software - be automatically generated from a
> simple functional specification. Getting THAT right is likely proving
> far more difficult than simply implementing a fixed design in an FPGA.
>

Yeah.

Long ago, I watched another project (FoNC, led by Alan Kay) that was
also trying to go this route. I think the idea was that they wanted to
try to find a way to describe the entire software stack (from OS to
applications) in under 20k lines.

Practically, it seemed to mostly end up going nowhere best I can tell, a
lot of "design", nothing that someone could actually use.

Though, if one sets the limits a little higher, there is a lot one can do:
One can at least, surely, make a usable compiler tool chain in under 1
million lines of code (at present, BGBCC weighs in at around 250 kLOC,
could be smaller; but, fitting a "basically functional" C compiler into
30k lines, or around the size of the Doom engine, seems a little harder).

Though, an intermediate option, would be trying to pull off a "semi
decent" compiler in under 100K lines.

If the compiler is kept smaller, it is faster to recompile from source.

Also, it would be nice to have a basically usable OS and core software
stack in under 1M lines.

Say, by not trying to be everything to everyone, and limiting how much
is allowed in the core OS (or is allowed within the build process for
the core OS).

Though, within moderate limits, 1M lines would basically be enough to fit:
A basic kernel;
(this excludes the Linux kernel, which is well over the size limit).
A (moderate sized) C compiler;
(but not GCC, which is also well over this size limit).
A shell+utils comparable to BusyBox;
Various core OS libraries and similar, etc.

For this, will assume an at least nominally POSIX like environment.

Programs that run on the OS would not be counted in the line-count budget.

How to deal with multi-platform portability would be more of an open
question, as this sort of thing tends to be a big source of code
expansion (or, for an OS kernel, the matter of hardware drivers, ...).

But, as can be noted, pretty much any project that gains mainstream
popularity seems to spiral out of control regarding code-size.

> YMMV,
> George

BGB wrote:

> On 12/15/2023 11:48 AM, George Neuner wrote:
>> On Wed, 13 Dec 2023 08:25:39 -0800, Stephen Fuld
>> <sfuld@alumni.cmu.edu.invalid> wrote:
>>
>>> When we last heard from the merry band of Millers, they were looking for
>>> substantial funding from a VC or similar. I suppose that if they had
>>> gotten it, we would have heard, so I guess they haven't.
>>>
>>> But I think there are things they could do to move forward even without
>>> a large investment. For example, they could develop an FPGA based
>>> system, even if it required multiple FPGAs on a custom circuit board for
>>> not huge amounts of money. Whether this is worthwhile, I cannot say.
>>>
>>> Anyway, has all development stopped? Or is their "sweat equity" model
>>> still going on?
>>>
>>> Inquiring minds want to know.
>>
>> There was a post, ostensibly from Ivan, in their web forum just a few
>> days ago. No news though - just an acknowledgement of another user's
>> post.
>>
>>
>> Last I heard, the next (current?) round of financing was - at least in
>> part - to be used for FPGA "proof of concept" implementations.
>>
>> Problem is the Mill really is a SoC, and (to me at least) the design
>> appears to be so complex that it would require a large, top-of-line
>> (read "expensive") FPGA to fit all the functionality.
>>

> Yeah. the lower end isn't cheap, the upper end is absurd...

Look into the cost of making a mask-set at 7nm or at 3nm. Then we can
have a discussion on how high the number has to be to rate absurd.

> For FPGA's over $1k, almost makes more sense to ignore that they exist
> (also this appears to be around the cutoff point for the free version of
> Vivado as well; but one would have thought Xilinx would have already
> gotten their money by someone having bought the FPGA?...).

>> Then there is their idea that everything - from VHDL to software build
>> toolchain to system software - be automatically generated from a
>> simple functional specification. Getting THAT right is likely proving
>> far more difficult than simply implementing a fixed design in an FPGA.
>>

> Yeah.

> Long ago, I watched another project (FoNC, led by Alan Kay) that was
> also trying to go this route. I think the idea was that they wanted to
> try to find a way to describe the entire software stack (from OS to
> applications) in under 20k lines.

Was the language of choice APL-like ??

> Practically, it seemed to mostly end up going nowhere best I can tell, a
> lot of "design", nothing that someone could actually use.

> Though, if one sets the limits a little higher, there is a lot one can do:
> One can at least, surely, make a usable compiler tool chain in under 1
> million lines of code (at present, BGBCC weighs in at around 250 kLOC,
> could be smaller; but, fitting a "basically functional" C compiler into
> 30k lines, or around the size of the Doom engine, seems a little harder).

> Though, an intermediate option, would be trying to pull off a "semi
> decent" compiler in under 100K lines.

> If the compiler is kept smaller, it is faster to recompile from source.

In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
10,000 lines of code per second for an IBM-like minicomputer (less decimal
and string) and did a pretty good job of spitting out high performance
code; on a machine with a 150ns cycle time.

We now have compilers struggling to achieve 10,000 lines per second per CPU
with machines of 0.2ns cycle time -- 75× faster {times the number of CPUs
thrown at the problem.}

> Also, it would be nice to have a basically usable OS and core software
> stack in under 1M lines.

There is no salable market for an OS that sheds featured for compactness.

> Say, by not trying to be everything to everyone, and limiting how much
> is allowed in the core OS (or is allowed within the build process for
> the core OS).

> Though, within moderate limits, 1M lines would basically be enough to fit:
> A basic kernel;
> (this excludes the Linux kernel, which is well over the size limit).

If there were an efficient way to run the device driver sack in user-mode
without privilege and only the MMI/O pages this driver can touch mapped
into his VAS. Poof none of the driver stack is in the kernel. --IF--

> A (moderate sized) C compiler;
> (but not GCC, which is also well over this size limit).

In 1990 C was a small language, In 2023 that statement is no longer true.
In 1990 the C compiler had 2 or 3 passes, in 2023 the LLVM compile has
<what> 35 passes (some of them duplicates as one pass converts into some-
thing a future pass will convert into something some other pass can
optimize.)
In 1990 your C compiler ran natively on your machine.
In 2023 your LLVM compiler compiles 6+ front end languages and compiles
to 20+ target ISAs and has to produce good code on all of them.

> A shell+utils comparable to BusyBox;

Until someone prevents someone else from writing new shells, filters,
and utilities, there is no way to moderate the growth in Shell+utils.

> Various core OS libraries and similar, etc.

> For this, will assume an at least nominally POSIX like environment.

> Programs that run on the OS would not be counted in the line-count budget.

> How to deal with multi-platform portability would be more of an open
> question, as this sort of thing tends to be a big source of code
> expansion (or, for an OS kernel, the matter of hardware drivers, ...).

> But, as can be noted, pretty much any project that gains mainstream
> popularity seems to spiral out of control regarding code-size.

With 20TB disk drives, 32 GB main memory sizes, Fiber internet;
what is the reason for worrying about something you can do almost
nothing about.

>> YMMV,

Indeed.

>> George

BGB wrote:
> On 12/15/2023 11:48 AM, George Neuner wrote:
>> On Wed, 13 Dec 2023 08:25:39 -0800, Stephen Fuld
>> <sfuld@alumni.cmu.edu.invalid> wrote:
>>
>>> When we last heard from the merry band of Millers, they were looking for
>>> substantial funding from a VC or similar. I suppose that if they had
>>> gotten it, we would have heard, so I guess they haven't.
>>>
>>> But I think there are things they could do to move forward even without
>>> a large investment. For example, they could develop an FPGA based
>>> system, even if it required multiple FPGAs on a custom circuit board for
>>> not huge amounts of money. Whether this is worthwhile, I cannot say.
>>>
>>> Anyway, has all development stopped? Or is their "sweat equity" model
>>> still going on?
>>>
>>> Inquiring minds want to know.
>>
>> There was a post, ostensibly from Ivan, in their web forum just a few
>> days ago. No news though - just an acknowledgement of another user's
>> post.
>>
>>
>> Last I heard, the next (current?) round of financing was - at least in
>> part - to be used for FPGA "proof of concept" implementations.
>>
>> Problem is the Mill really is a SoC, and (to me at least) the design
>> appears to be so complex that it would require a large, top-of-line
>> (read "expensive") FPGA to fit all the functionality.
>>
>
> Yeah. the lower end isn't cheap, the upper end is absurd...
>
> For FPGA's over $1k, almost makes more sense to ignore that they exist
> (also this appears to be around the cutoff point for the free version of
> Vivado as well; but one would have thought Xilinx would have already
> gotten their money by someone having bought the FPGA?...).

Found a recent article that says Xilinx prices run from 8$ to $100,
low end Intel fpga's start at $3, but the high end Stratix models
go from $10,000 to $100,000.

mitchalsup@aol.com (MitchAlsup) writes:
>BGB wrote:

>> For FPGA's over $1k, almost makes more sense to ignore that they exist
>> (also this appears to be around the cutoff point for the free version of
>> Vivado as well; but one would have thought Xilinx would have already
>> gotten their money by someone having bought the FPGA?...).

For anyone serious, an verif engineer can cost $500-1000/day. The FPGA
cost is in the noise.

For a hobby? Well...

>> If the compiler is kept smaller, it is faster to recompile from source.
>
>In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>10,000 lines of code per second for an IBM-like minicomputer (less decimal
>and string) and did a pretty good job of spitting out high performance
>code; on a machine with a 150ns cycle time.

As did our COBOL compiler (which ran in 50KB). But in both cases,
the languages were far simpler and much easier to generate efficient
code than languages like Modula, Pascal, C, et alia.

>> Though, within moderate limits, 1M lines would basically be enough to fit:
>> A basic kernel;
>> (this excludes the Linux kernel, which is well over the size limit).
>
>If there were an efficient way to run the device driver sack in user-mode
>without privilege and only the MMI/O pages this driver can touch mapped
>into his VAS. Poof none of the driver stack is in the kernel. --IF--

That's actually quite common and one of the raison d'etre of the
PCI Express SR-IOV feature. When you can present a virtual
function to the user directly (mapping the MMIO region into
the user mode virtual address space) the app had direct access
to the hardware. Interrupts are the only tricky part, and
the kernel virtio subsystem, which interfaces with the user
application via shared memory provides interrupt handling
to the application.

An I/OMMU provides memory protection for DMA operations initiated
by the virtual function ensuring it only accesses the application
virtual address space.

Scott Lurndal wrote:

> mitchalsup@aol.com (MitchAlsup) writes:
>>BGB wrote:

>>> For FPGA's over $1k, almost makes more sense to ignore that they exist
>>> (also this appears to be around the cutoff point for the free version of
>>> Vivado as well; but one would have thought Xilinx would have already
>>> gotten their money by someone having bought the FPGA?...).

> For anyone serious, an verif engineer can cost $500-1000/day. The FPGA
> cost is in the noise.

> For a hobby? Well...

>>> If the compiler is kept smaller, it is faster to recompile from source.
>>
>>In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>10,000 lines of code per second for an IBM-like minicomputer (less decimal
>>and string) and did a pretty good job of spitting out high performance
>>code; on a machine with a 150ns cycle time.

> As did our COBOL compiler (which ran in 50KB). But in both cases,
> the languages were far simpler and much easier to generate efficient
> code than languages like Modula, Pascal, C, et alia.

>>> Though, within moderate limits, 1M lines would basically be enough to fit:
>>> A basic kernel;
>>> (this excludes the Linux kernel, which is well over the size limit).
>>
>>If there were an efficient way to run the device driver sack in user-mode
>>without privilege and only the MMI/O pages this driver can touch mapped
>>into his VAS. Poof none of the driver stack is in the kernel. --IF--

> That's actually quite common and one of the raison d'etre of the
> PCI Express SR-IOV feature. When you can present a virtual
> function to the user directly (mapping the MMIO region into
> the user mode virtual address space) the app had direct access
> to the hardware. Interrupts are the only tricky part, and
> the kernel virtio subsystem, which interfaces with the user
> application via shared memory provides interrupt handling
> to the application.

> An I/OMMU provides memory protection for DMA operations initiated
> by the virtual function ensuring it only accesses the application
> virtual address space.

Why should device be able to access user VaS outside of the buffer the
user provided, OH so long ago ??

On 12/15/2023 2:59 PM, MitchAlsup wrote:
> BGB wrote:
>
>> On 12/15/2023 11:48 AM, George Neuner wrote:
>>> On Wed, 13 Dec 2023 08:25:39 -0800, Stephen Fuld
>>> <sfuld@alumni.cmu.edu.invalid> wrote:
>>>
>>>> When we last heard from the merry band of Millers, they were looking
>>>> for
>>>> substantial funding from a VC or similar.  I suppose that if they had
>>>> gotten it, we would have heard, so I guess they haven't.
>>>>
>>>> But I think there are things they could do to move forward even without
>>>> a large investment.  For example, they could develop an FPGA based
>>>> system, even if it required multiple FPGAs on a custom circuit board
>>>> for
>>>> not huge amounts of money.  Whether this is worthwhile, I cannot say.
>>>>
>>>> Anyway, has all development stopped?  Or is their "sweat equity" model
>>>> still going on?
>>>>
>>>> Inquiring minds want to know.
>>>
>>> There was a post, ostensibly from Ivan, in their web forum just a few
>>> days ago. No news though - just an acknowledgement of another user's
>>> post.
>>>
>>>
>>> Last I heard, the next (current?) round of financing was - at least in
>>> part - to be used for FPGA "proof of concept" implementations.
>>>
>>> Problem is the Mill really is a SoC, and (to me at least) the design
>>> appears to be so complex that it would require a large, top-of-line
>>> (read "expensive") FPGA to fit all the functionality.
>>>
>
>> Yeah. the lower end isn't cheap, the upper end is absurd...
>
> Look into the cost of making a mask-set at 7nm or at 3nm. Then we can
> have a discussion on how high the number has to be to rate absurd.
>

This sort of thing is only really within reach of big companies...

The Spartan and Artix boards are within reach of hobbyists.
Kintex is, sorta, if a person has a lot of money to burn on it.

>> For FPGA's over $1k, almost makes more sense to ignore that they exist
>> (also this appears to be around the cutoff point for the free version
>> of Vivado as well; but one would have thought Xilinx would have
>> already gotten their money by someone having bought the FPGA?...).
>
>
>>> Then there is their idea that everything - from VHDL to software build
>>> toolchain to system software - be automatically generated from a
>>> simple functional specification.  Getting THAT right is likely proving
>>> far more difficult than simply implementing a fixed design in an FPGA.
>>>
>
>> Yeah.
>
>> Long ago, I watched another project (FoNC, led by Alan Kay) that was
>> also trying to go this route. I think the idea was that they wanted to
>> try to find a way to describe the entire software stack (from OS to
>> applications) in under 20k lines.
>
> Was the language of choice APL-like ??
>

Alan Kay was known for Smalltalk, and the languages they were using were
using a Smalltalk like syntax IIRC.

I never really got much into Smalltalk though as it tended to be
difficult to make sense of.

But, I guess, they didn't achieve the goals of either keeping it under
the size limit, or of making something usable.

>> Practically, it seemed to mostly end up going nowhere best I can tell,
>> a lot of "design", nothing that someone could actually use.
>
>
>
>> Though, if one sets the limits a little higher, there is a lot one can
>> do:
>> One can at least, surely, make a usable compiler tool chain in under 1
>> million lines of code (at present, BGBCC weighs in at around 250 kLOC,
>> could be smaller; but, fitting a "basically functional" C compiler
>> into 30k lines, or around the size of the Doom engine, seems a little
>> harder).
>
>> Though, an intermediate option, would be trying to pull off a "semi
>> decent" compiler in under 100K lines.
>
>
>
>> If the compiler is kept smaller, it is faster to recompile from source.
>
> In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
> 10,000 lines of code per second for an IBM-like minicomputer (less
> decimal and string) and did a pretty good job of spitting out high
> performance
> code; on a machine with a 150ns cycle time.
>
> We now have compilers struggling to achieve 10,000 lines per second per CPU
> with machines of 0.2ns cycle time -- 75× faster {times the number of CPUs
> thrown at the problem.}
>

If the compiler is 250k lines of C, it can still compile in a few
seconds on a modern PC.

If if is several million lines with a bunch of C++ thrown in (or
entirely in C++), then it takes a bit longer.

Recompiling LLVM and Clang is a bit much even with a fairly beefy PC.

>> Also, it would be nice to have a basically usable OS and core software
>> stack in under 1M lines.
>
> There is no salable market for an OS that sheds featured for compactness.
>

Could be easier to port to new targets, less RAM and space needed.
If the footprint is small enough to fit on a moderately cheap SPI Flash,
one can use a moderately cheap SPI Flash.

Though, for end-user use, one is probably going to need things like a
web-browser and similar, and "small but actually useful" web browser
probably isn't going to happen (IOW: people aren't going to use
something that can't do much more than a basic subset of static HTML).

>> Say, by not trying to be everything to everyone, and limiting how much
>> is allowed in the core OS (or is allowed within the build process for
>> the core OS).
>
>> Though, within moderate limits, 1M lines would basically be enough to
>> fit:
>>    A basic kernel;
>>      (this excludes the Linux kernel, which is well over the size limit).
>
> If there were an efficient way to run the device driver sack in user-mode
> without privilege and only the MMI/O pages this driver can touch mapped
> into his VAS. Poof none of the driver stack is in the kernel.  --IF--
>

Yeah, or "superusermode" drivers (in my scheme).

Where the drivers aren't technically in the kernel, but still have
access to hardware MMIO and similar.

Though, absent some design changes, superusermode can easily bypass my
existing memory protection scheme if it so chooses. Would need to come
up with a way to allow actual usermode tasks to be able to have
selective access to MMIO to be able to have any hope of protecting the
OS from malicious drivers.

Though, if it needs to run on x86 or ARM, this is more of a problem, and
there is likely little practical alternative either than:
Run drivers in bare kernel space;
Run drivers in logical processes with a bunch of extra overhead.

>>    A (moderate sized) C compiler;
>>      (but not GCC, which is also well over this size limit).
>
> In 1990 C was a small language, In 2023 that statement is no longer true.
> In 1990 the C compiler had 2 or 3 passes, in 2023 the LLVM compile has
> <what> 35 passes (some of them duplicates as one pass converts into some-
> thing a future pass will convert into something some other pass can
> optimize.)
> In 1990 your C compiler ran natively on your machine.
> In 2023 your LLVM compiler compiles 6+ front end languages and compiles
> to 20+ target ISAs and has to produce good code on all of them.
>

C proper hasn't changed *that* much.
C++ kinda wrecks this.

>>    A shell+utils comparable to BusyBox;
>
> Until someone prevents someone else from writing new shells, filters,
> and utilities, there is no way to moderate the growth in Shell+utils.
>

Yeah...

If you want something like Bash + GNU CoreUtils, it is going to be a lot
bigger than something along the lines of Ash + BusyBox.

I was considering possibly reworking how shell works in my case;
currently the shell is in the kernel (though now splits off into
separate tasks for each shell instance), but a design more akin to
BusyBox could make more sense.

But, not entirely a fan of GPL (which BusyBox uses), and while ToyBox
has a better license, I am admittedly less of a fan of the main author
(in past interactions he had acted like a condescending jerk, this isn't
really a win for me even if the design and license seems good in other
areas).

>>    Various core OS libraries and similar, etc.
>
>> For this, will assume an at least nominally POSIX like environment.
>
>
>> Programs that run on the OS would not be counted in the line-count
>> budget.
>
>> How to deal with multi-platform portability would be more of an open
>> question, as this sort of thing tends to be a big source of code
>> expansion (or, for an OS kernel, the matter of hardware drivers, ...).
>
>> But, as can be noted, pretty much any project that gains mainstream
>> popularity seems to spiral out of control regarding code-size.
>
> With 20TB disk drives, 32 GB main memory sizes, Fiber internet;
> what is the reason for worrying about something you can do almost
> nothing about.
>

mitchalsup@aol.com (MitchAlsup) writes:
>Scott Lurndal wrote:
>
>> mitchalsup@aol.com (MitchAlsup) writes:
>>>BGB wrote:
>
>>>> For FPGA's over $1k, almost makes more sense to ignore that they exist
>>>> (also this appears to be around the cutoff point for the free version of
>>>> Vivado as well; but one would have thought Xilinx would have already
>>>> gotten their money by someone having bought the FPGA?...).
>
>> For anyone serious, an verif engineer can cost $500-1000/day. The FPGA
>> cost is in the noise.
>
>> For a hobby? Well...
>
>
>>>> If the compiler is kept smaller, it is faster to recompile from source.
>>>
>>>In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>>10,000 lines of code per second for an IBM-like minicomputer (less decimal
>>>and string) and did a pretty good job of spitting out high performance
>>>code; on a machine with a 150ns cycle time.
>
>> As did our COBOL compiler (which ran in 50KB). But in both cases,
>> the languages were far simpler and much easier to generate efficient
>> code than languages like Modula, Pascal, C, et alia.
>
>>>> Though, within moderate limits, 1M lines would basically be enough to fit:
>>>> A basic kernel;
>>>> (this excludes the Linux kernel, which is well over the size limit).
>>>
>>>If there were an efficient way to run the device driver sack in user-mode
>>>without privilege and only the MMI/O pages this driver can touch mapped
>>>into his VAS. Poof none of the driver stack is in the kernel. --IF--
>
>> That's actually quite common and one of the raison d'etre of the
>> PCI Express SR-IOV feature. When you can present a virtual
>> function to the user directly (mapping the MMIO region into
>> the user mode virtual address space) the app had direct access
>> to the hardware. Interrupts are the only tricky part, and
>> the kernel virtio subsystem, which interfaces with the user
>> application via shared memory provides interrupt handling
>> to the application.
>
>> An I/OMMU provides memory protection for DMA operations initiated
>> by the virtual function ensuring it only accesses the application
>> virtual address space.
>
>Why should device be able to access user VaS outside of the buffer the
>user provided, OH so long ago ??

Because the device wants to do DMA directly into or from the users
virtual address space. Bulk transfer, not MMIO accesses.

Think network controller fetching packets from userspace.

On 2023-12-16 0:39, Scott Lurndal wrote:
> mitchalsup@aol.com (MitchAlsup) writes:

[snip]

>> In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>> 10,000 lines of code per second for an IBM-like minicomputer (less decimal
>> and string) and did a pretty good job of spitting out high performance
>> code; on a machine with a 150ns cycle time.
>
> As did our COBOL compiler (which ran in 50KB).

Are you both sure that those numbers are really lines per *second*? They
seem improbably high, and compilation speeds in those years used to be
stated in lines per *minute*.

Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>On 2023-12-16 0:39, Scott Lurndal wrote:
>> mitchalsup@aol.com (MitchAlsup) writes:
>
> [snip]
>
>>> In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>> 10,000 lines of code per second for an IBM-like minicomputer (less decimal
>>> and string) and did a pretty good job of spitting out high performance
>>> code; on a machine with a 150ns cycle time.
>>
>> As did our COBOL compiler (which ran in 50KB).
>
>
>Are you both sure that those numbers are really lines per *second*? They
>seem improbably high, and compilation speeds in those years used to be
>stated in lines per *minute*.

Especially given that 10Klines/s is probably around 500KB/s which has
to be read from disk and probably a similar amount that has to be
written to disk. What were the I/O throughputs available at the time?

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
> Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>>On 2023-12-16 0:39, Scott Lurndal wrote:
>>> mitchalsup@aol.com (MitchAlsup) writes:
>>
>> [snip]
>>
>>>> In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>>> 10,000 lines of code per second for an IBM-like minicomputer (less decimal
>>>> and string) and did a pretty good job of spitting out high performance
>>>> code; on a machine with a 150ns cycle time.
>>>
>>> As did our COBOL compiler (which ran in 50KB).
>>
>>
>>Are you both sure that those numbers are really lines per *second*? They
>>seem improbably high, and compilation speeds in those years used to be
>>stated in lines per *minute*.
>
> Especially given that 10Klines/s is probably around 500KB/s which has
> to be read from disk and probably a similar amount that has to be
> written to disk. What were the I/O throughputs available at the time?

It depends a bit how the Fortran and Cobol statements were stored.
If they were stored in punched card format, 80 characters per line,
then it would be 800000 characters per second read. Object code,
probably much less, but the total could still come to around
1 MB/s.

The IBM 3350 (introduced in 1975) is probably fairly representative
of the high end of that era, it had a data transfer speed of 1198
kB/second, and a seek time of 25 milliseconds.

So, 10000 lines/s would almost definitely have been I/O bound at the
time.

Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>On 2023-12-16 0:39, Scott Lurndal wrote:
>> mitchalsup@aol.com (MitchAlsup) writes:
>
> [snip]
>
>>> In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>> 10,000 lines of code per second for an IBM-like minicomputer (less decimal
>>> and string) and did a pretty good job of spitting out high performance
>>> code; on a machine with a 150ns cycle time.
>>
>> As did our COBOL compiler (which ran in 50KB).
>
>
>Are you both sure that those numbers are really lines per *second*? They
>seem improbably high, and compilation speeds in those years used to be
>stated in lines per *minute*.

Yes, lines per minute is the proper metric. Note that for many
years, the compilation rate was bounded by the speed of the card
reader (300 to 600 cards per minute).

On 16/12/2023 07:22, Niklas Holsti wrote:
> On 2023-12-16 0:39, Scott Lurndal wrote:
>> mitchalsup@aol.com (MitchAlsup) writes:
>
>    [snip]
>
>>> In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>> 10,000 lines of code per second for an IBM-like minicomputer (less
>>> decimal
>>> and string) and did a pretty good job of spitting out high performance
>>> code; on a machine with a 150ns cycle time.
>>
>> As did our COBOL compiler (which ran in 50KB).
>
>
> Are you both sure that those numbers are really lines per *second*? They
> seem improbably high, and compilation speeds in those years used to be
> stated in lines per *minute*.
>

Almost certainly per minute.
I worked on a compiler in 1975 that ran on the most powerful ICL 1900.
It achieved 20K cards per minute and was considered to be very fast.

--
Bill F.

On 12/16/2023 12:04 PM, moi wrote:
> On 16/12/2023 07:22, Niklas Holsti wrote:
>> On 2023-12-16 0:39, Scott Lurndal wrote:
>>> mitchalsup@aol.com (MitchAlsup) writes:
>>
>>     [snip]
>>
>>>> In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>>> 10,000 lines of code per second for an IBM-like minicomputer (less
>>>> decimal
>>>> and string) and did a pretty good job of spitting out high performance
>>>> code; on a machine with a 150ns cycle time.
>>>
>>> As did our COBOL compiler (which ran in 50KB).
>>
>>
>> Are you both sure that those numbers are really lines per *second*?
>> They seem improbably high, and compilation speeds in those years used
>> to be stated in lines per *minute*.
>>
>
> Almost certainly per minute.
> I worked on a compiler in 1975 that ran on the most powerful ICL 1900.
> It achieved 20K cards per minute and was considered to be very fast.
>

Lines per minute seems to make sense.

Modern PC's are orders of magnitude faster, but still don't have
"instant" compile times by any means.

Could be faster though, but would likely need languages other than C or
(especially) C++.

For both languages, one has the overheads of needing to read in a whole
lot of header code (often expanding out to 100s of kB or sometimes a few
MB) often for only 5-20kB of actual source code.

C++ then ruins compiler speed with things like templates.

Though, final code generation often does take some extra time.

For example, in BGBCC, a lot of time tends to be spent in the
"WEXifier", which mostly tries to shuffle instructions around and bundle
them in parallel (a lot of this being in terms of the code for figuring
out whether instructions can swap places, be run in parallel, and for
comparing relative costs).

....

Scott Lurndal wrote:

> mitchalsup@aol.com (MitchAlsup) writes:
>>Scott Lurndal wrote:
>>
>>> mitchalsup@aol.com (MitchAlsup) writes:
>>>>BGB wrote:
>>
>>>>> For FPGA's over $1k, almost makes more sense to ignore that they exist
>>>>> (also this appears to be around the cutoff point for the free version of
>>>>> Vivado as well; but one would have thought Xilinx would have already
>>>>> gotten their money by someone having bought the FPGA?...).
>>
>>> For anyone serious, an verif engineer can cost $500-1000/day. The FPGA
>>> cost is in the noise.
>>
>>> For a hobby? Well...
>>
>>
>>>>> If the compiler is kept smaller, it is faster to recompile from source.
>>>>
>>>>In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>>>10,000 lines of code per second for an IBM-like minicomputer (less decimal
>>>>and string) and did a pretty good job of spitting out high performance
>>>>code; on a machine with a 150ns cycle time.
>>
>>> As did our COBOL compiler (which ran in 50KB). But in both cases,
>>> the languages were far simpler and much easier to generate efficient
>>> code than languages like Modula, Pascal, C, et alia.
>>
>>>>> Though, within moderate limits, 1M lines would basically be enough to fit:
>>>>> A basic kernel;
>>>>> (this excludes the Linux kernel, which is well over the size limit).
>>>>
>>>>If there were an efficient way to run the device driver sack in user-mode
>>>>without privilege and only the MMI/O pages this driver can touch mapped
>>>>into his VAS. Poof none of the driver stack is in the kernel. --IF--
>>
>>> That's actually quite common and one of the raison d'etre of the
>>> PCI Express SR-IOV feature. When you can present a virtual
>>> function to the user directly (mapping the MMIO region into
>>> the user mode virtual address space) the app had direct access
>>> to the hardware. Interrupts are the only tricky part, and
>>> the kernel virtio subsystem, which interfaces with the user
>>> application via shared memory provides interrupt handling
>>> to the application.
>>
>>> An I/OMMU provides memory protection for DMA operations initiated
>>> by the virtual function ensuring it only accesses the application
>>> virtual address space.
>>
>>Why should device be able to access user VaS outside of the buffer the
>>user provided, OH so long ago ??

> Because the device wants to do DMA directly into or from the users
> virtual address space. Bulk transfer, not MMIO accesses.

OK, I will ask the question in the contrapositive way::
If the user ask device to read into a buffer, why does the device get
to see everything of the user's space along with that buffer ?

The way you write you are assuming the device can write into the
user's code space when he ask for a read from one of his buffers !?!

You _could_ give device translations to anything and everything
in user space, but this seems excessive when the user only wants
the device to read/write small area inside his VaS.

OS code already has to manipulate PTE entries or MMU tables so
the device can write read-only and execute-only pages along with
removing write-permission on a page with data inbound from a device.

> Think network controller fetching packets from userspace.

MitchAlsup wrote:
> Scott Lurndal wrote:
>
>> mitchalsup@aol.com (MitchAlsup) writes:
>>> Scott Lurndal wrote:
>>>
>>>> mitchalsup@aol.com (MitchAlsup) writes:
>>>>> BGB wrote:
>>>
>>>>>> For FPGA's over $1k, almost makes more sense to ignore that they
>>>>>> exist (also this appears to be around the cutoff point for the
>>>>>> free version of Vivado as well; but one would have thought Xilinx
>>>>>> would have already gotten their money by someone having bought the
>>>>>> FPGA?...).
>>>
>>>> For anyone serious, an verif engineer can cost $500-1000/day. The
>>>> FPGA
>>>> cost is in the noise.
>>>
>>>> For a hobby? Well...
>>>
>>>
>>>>>> If the compiler is kept smaller, it is faster to recompile from
>>>>>> source.
>>>>>
>>>>> In 1979 I joined a company with a FORTRAN mostly-77- that compiled
>>>>> at 10,000 lines of code per second for an IBM-like minicomputer
>>>>> (less decimal and string) and did a pretty good job of spitting out
>>>>> high performance
>>>>> code; on a machine with a 150ns cycle time.
>>>
>>>> As did our COBOL compiler (which ran in 50KB). But in both cases,
>>>> the languages were far simpler and much easier to generate efficient
>>>> code than languages like Modula, Pascal, C, et alia.
>>>
>>>>>> Though, within moderate limits, 1M lines would basically be enough
>>>>>> to fit:
>>>>>> A basic kernel;
>>>>>> (this excludes the Linux kernel, which is well over the size
>>>>>> limit).
>>>>>
>>>>> If there were an efficient way to run the device driver sack in
>>>>> user-mode
>>>>> without privilege and only the MMI/O pages this driver can touch
>>>>> mapped
>>>>> into his VAS. Poof none of the driver stack is in the kernel. --IF--
>>>
>>>> That's actually quite common and one of the raison d'etre of the
>>>> PCI Express SR-IOV feature. When you can present a virtual
>>>> function to the user directly (mapping the MMIO region into
>>>> the user mode virtual address space) the app had direct access
>>>> to the hardware. Interrupts are the only tricky part, and
>>>> the kernel virtio subsystem, which interfaces with the user
>>>> application via shared memory provides interrupt handling
>>>> to the application.
>>>
>>>> An I/OMMU provides memory protection for DMA operations initiated
>>>> by the virtual function ensuring it only accesses the application
>>>> virtual address space.
>>>
>>> Why should device be able to access user VaS outside of the buffer
>>> the user provided, OH so long ago ??
>
>
>> Because the device wants to do DMA directly into or from the users
>> virtual address space. Bulk transfer, not MMIO accesses.
>
> OK, I will ask the question in the contrapositive way::
> If the user ask device to read into a buffer, why does the device get
> to see everything of the user's space along with that buffer ?
>
> The way you write you are assuming the device can write into the
> user's code space when he ask for a read from one of his buffers !?!
>
> You _could_ give device translations to anything and everything
> in user space, but this seems excessive when the user only wants
> the device to read/write small area inside his VaS.
>
> OS code already has to manipulate PTE entries or MMU tables so
> the device can write read-only and execute-only pages along with
> removing write-permission on a page with data inbound from a device.

The OS can't remove the page RW access for a user mode page while an
IO device is DMA writing the page, if that's what you meant,
as the DMA-in may be writing to a smaller buffer within a larger page.
It is perfectly normal for a thread to continue to work in buffer
bytes adjacent to the one currently involved in an async IO.

mitchalsup@aol.com (MitchAlsup) writes:
>Scott Lurndal wrote:
>
>> mitchalsup@aol.com (MitchAlsup) writes:
>>>Scott Lurndal wrote:
>>>
>>>> mitchalsup@aol.com (MitchAlsup) writes:
>>>>>BGB wrote:
>>>
>>>>>> For FPGA's over $1k, almost makes more sense to ignore that they exist
>>>>>> (also this appears to be around the cutoff point for the free version of
>>>>>> Vivado as well; but one would have thought Xilinx would have already
>>>>>> gotten their money by someone having bought the FPGA?...).
>>>
>>>> For anyone serious, an verif engineer can cost $500-1000/day. The FPGA
>>>> cost is in the noise.
>>>
>>>> For a hobby? Well...
>>>
>>>
>>>>>> If the compiler is kept smaller, it is faster to recompile from source.
>>>>>
>>>>>In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>>>>10,000 lines of code per second for an IBM-like minicomputer (less decimal
>>>>>and string) and did a pretty good job of spitting out high performance
>>>>>code; on a machine with a 150ns cycle time.
>>>
>>>> As did our COBOL compiler (which ran in 50KB). But in both cases,
>>>> the languages were far simpler and much easier to generate efficient
>>>> code than languages like Modula, Pascal, C, et alia.
>>>
>>>>>> Though, within moderate limits, 1M lines would basically be enough to fit:
>>>>>> A basic kernel;
>>>>>> (this excludes the Linux kernel, which is well over the size limit).
>>>>>
>>>>>If there were an efficient way to run the device driver sack in user-mode
>>>>>without privilege and only the MMI/O pages this driver can touch mapped
>>>>>into his VAS. Poof none of the driver stack is in the kernel. --IF--
>>>
>>>> That's actually quite common and one of the raison d'etre of the
>>>> PCI Express SR-IOV feature. When you can present a virtual
>>>> function to the user directly (mapping the MMIO region into
>>>> the user mode virtual address space) the app had direct access
>>>> to the hardware. Interrupts are the only tricky part, and
>>>> the kernel virtio subsystem, which interfaces with the user
>>>> application via shared memory provides interrupt handling
>>>> to the application.
>>>
>>>> An I/OMMU provides memory protection for DMA operations initiated
>>>> by the virtual function ensuring it only accesses the application
>>>> virtual address space.
>>>
>>>Why should device be able to access user VaS outside of the buffer the
>>>user provided, OH so long ago ??
>
>
>> Because the device wants to do DMA directly into or from the users
>> virtual address space. Bulk transfer, not MMIO accesses.
>
>OK, I will ask the question in the contrapositive way::
>If the user ask device to read into a buffer, why does the device get
>to see everything of the user's space along with that buffer ?

It doesn't, necessarily. The IOMMU translation table is a
proper subset of the user's virtual address space. The
application tells the kernel which portions of the address
space are valid DMA regions for the device to access.

On 12/16/2023 1:25 PM, EricP wrote:
> MitchAlsup wrote:
>> Scott Lurndal wrote:
>>
>>> mitchalsup@aol.com (MitchAlsup) writes:
>>>> Scott Lurndal wrote:
>>>>
>>>>> mitchalsup@aol.com (MitchAlsup) writes:
>>>>>> BGB wrote:
>>>>
>>>>>>> For FPGA's over $1k, almost makes more sense to ignore that they
>>>>>>> exist (also this appears to be around the cutoff point for the
>>>>>>> free version of Vivado as well; but one would have thought Xilinx
>>>>>>> would have already gotten their money by someone having bought
>>>>>>> the FPGA?...).
>>>>
>>>>> For anyone serious, an verif engineer can cost $500-1000/day.   The
>>>>> FPGA
>>>>> cost is in the noise.
>>>>
>>>>> For a hobby?  Well...
>>>>
>>>>
>>>>>>> If the compiler is kept smaller, it is faster to recompile from
>>>>>>> source.
>>>>>>
>>>>>> In 1979 I joined a company with a FORTRAN mostly-77- that compiled
>>>>>> at 10,000 lines of code per second for an IBM-like minicomputer
>>>>>> (less decimal and string) and did a pretty good job of spitting
>>>>>> out high performance
>>>>>> code; on a machine with a 150ns cycle time.
>>>>
>>>>> As did our COBOL compiler (which ran in 50KB).  But in both cases,
>>>>> the languages were far simpler and much easier to generate efficient
>>>>> code than languages like Modula, Pascal, C, et alia.
>>>>
>>>>>>> Though, within moderate limits, 1M lines would basically be
>>>>>>> enough to fit:
>>>>>>>    A basic kernel;
>>>>>>>      (this excludes the Linux kernel, which is well over the size
>>>>>>> limit).
>>>>>>
>>>>>> If there were an efficient way to run the device driver sack in
>>>>>> user-mode
>>>>>> without privilege and only the MMI/O pages this driver can touch
>>>>>> mapped
>>>>>> into his VAS. Poof none of the driver stack is in the kernel.  --IF--
>>>>
>>>>> That's actually quite common and one of the raison d'etre of the
>>>>> PCI Express SR-IOV feature.    When you can present a virtual
>>>>> function to the user directly (mapping the MMIO region into
>>>>> the user mode virtual address space) the app had direct access
>>>>> to the hardware.    Interrupts are the only tricky part, and
>>>>> the kernel virtio subsystem, which interfaces with the user
>>>>> application via shared memory provides interrupt handling
>>>>> to the application.
>>>>
>>>>> An I/OMMU provides memory protection for DMA operations initiated
>>>>> by the virtual function ensuring it only accesses the application
>>>>> virtual address space.
>>>>
>>>> Why should device be able to access user VaS outside of the buffer
>>>> the user provided, OH so long ago ??
>>
>>
>>> Because the device wants to do DMA directly into or from the users
>>> virtual address space.   Bulk transfer, not MMIO accesses.
>>
>> OK, I will ask the question in the contrapositive way::
>> If the user ask device to read into a buffer, why does the device get
>> to see everything of the user's space along with that buffer ?
>>
>> The way you write you are assuming the device can write into the
>> user's code space when he ask for a read from one of his buffers !?!
>>
>> You _could_ give device translations to anything and everything
>> in user space, but this seems excessive when the user only wants
>> the device to read/write small area inside his VaS.
>>
>> OS code already has to manipulate PTE entries or MMU tables so
>> the device can write read-only and execute-only pages along with
>> removing write-permission on a page with data inbound from a device.
>
> The OS can't remove the page RW access for a user mode page while an
> IO device is DMA writing the page, if that's what you meant,
> as the DMA-in may be writing to a smaller buffer within a larger page.
> It is perfectly normal for a thread to continue to work in buffer
> bytes adjacent to the one currently involved in an async IO.
>

One thing I don't get here is why there would be direct DMA between
userland and the device (at least for filesystem and similar).

Like, say, for a filesystem, it is presumably:
read syscall from user to OS;
route this to the corresponding VFS driver;
Requests spanning multiple blocks being broken up into parts;
VFS driver checks the block-cache / buffer-cache;
If found, copy from cache into user-space;
If not found, send request to the underlying block device;
Wait for response (and/or reschedule task for later);
Copy result back into userland.

Though, it may make sense that if a request isn't available immediately,
and there is some sort of DMA mechanism, the OS could block the task and
then resume it once the data becomes available. For polling IO, doesn't
likely make much difference as the CPU is basically stuck in a busy loop
either way until the IO finishes.

Though, could make sense for hardware accelerating pixel-copying
operations for a GUI.

For GUI, there would be multiple stages of copying, say:
Copying from user buffer to window buffer;
Copying from window buffer to screen buffer;
Copying from screen buffer to VRAM.

For video playback or GL, there may be an additional stage of copying
from GL's buffer to a user's buffer, then from the user's buffer to the
window buffer. Though, considering possibly adding a shortcut path where
GL and video codecs copy more directly into the window buffer (bypassing
needing to pass the frame data through the userland program).

Could be also possible maybe to have GL render directly into the window
buffer, which could be possible if they have the same format/resolution,
and the window buffer is physically mapped (say, for my current hardware
rasterizer module).

If running a program full-screen, it is possible to copy more directly
from the user buffer into VRAM, saving some time here.

Some time could be saved here if one had hardware support for these
sorts of "copy pixel buffers around and convert between formats" tasks,
but to be useful, this would need to be able to work with virtual
memory, which adds some complexity (would either need to be CPU-like
and/or have a page-walker; neither is particularly cheap).

Could maybe offload the task to the rasterizer module, but would need to
add a page-walker to the rasterizer... Though, trying to deal with some
scenarios (such as the final conversion/copy to VRAM) would add a lot of
extra complexity. For now, its framebuffer/zbuffer/textures need to be
in physically-mapped addresses (also with a 128-bit buffer alignment).

Though, cheaper could be to make use of the second CPU core, but then
schedule things like pixel copy operations to it (maybe also things like
vertex transform and similar for OpenGL). Currently, if enabled, the
second core hasn't seen a lot of use thus far in my case.

....

BGB <cr88192@gmail.com> schrieb:
> On 12/16/2023 12:04 PM, moi wrote:
>> On 16/12/2023 07:22, Niklas Holsti wrote:
>>> On 2023-12-16 0:39, Scott Lurndal wrote:
>>>> mitchalsup@aol.com (MitchAlsup) writes:
>>>
>>>     [snip]
>>>
>>>>> In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>>>> 10,000 lines of code per second for an IBM-like minicomputer (less
>>>>> decimal
>>>>> and string) and did a pretty good job of spitting out high performance
>>>>> code; on a machine with a 150ns cycle time.
>>>>
>>>> As did our COBOL compiler (which ran in 50KB).
>>>
>>>
>>> Are you both sure that those numbers are really lines per *second*?
>>> They seem improbably high, and compilation speeds in those years used
>>> to be stated in lines per *minute*.
>>>
>>
>> Almost certainly per minute.
>> I worked on a compiler in 1975 that ran on the most powerful ICL 1900.
>> It achieved 20K cards per minute and was considered to be very fast.
>>
>
> Lines per minute seems to make sense.
>
>
> Modern PC's are orders of magnitude faster, but still don't have
> "instant" compile times by any means.
>
> Could be faster though, but would likely need languages other than C or
> (especially) C++.

I assume you never worked with Turbo Pascal.

That was amazing. It compiled code so fast that it was never a
bother, to wait for it, even on a 8088 IBM PC running at 4.7 MHz.
The first version I ever used, 3.0 (?) compiled from memory to
memory, so even slow I/O (to floppy disc, at the time) was not
an issue.

This was made possible by using a streamlined one-pass compiler. It
didn't do much optimization, but when the alternative was BASIC, the
generated code was still extremely fast by comparision.

There were a few drawbacks. The biggest one was that programming errors
tended to freeze the machine. Another (not so important) was that,
if you were one of the lucky people to have an 80x87 coprocessor, the
generated code did not check for overflow of the coprocessor stack.

Scott Lurndal wrote:

> mitchalsup@aol.com (MitchAlsup) writes:
>>Scott Lurndal wrote:
>>
>>> mitchalsup@aol.com (MitchAlsup) writes:
>>>>Scott Lurndal wrote:
>>>>
>>>>> mitchalsup@aol.com (MitchAlsup) writes:
>>>>>>BGB wrote:
>>>>
>>>>>>> For FPGA's over $1k, almost makes more sense to ignore that they exist
>>>>>>> (also this appears to be around the cutoff point for the free version of
>>>>>>> Vivado as well; but one would have thought Xilinx would have already
>>>>>>> gotten their money by someone having bought the FPGA?...).
>>>>
>>>>> For anyone serious, an verif engineer can cost $500-1000/day. The FPGA
>>>>> cost is in the noise.
>>>>
>>>>> For a hobby? Well...
>>>>
>>>>
>>>>>>> If the compiler is kept smaller, it is faster to recompile from source.
>>>>>>
>>>>>>In 1979 I joined a company with a FORTRAN mostly-77- that compiled at
>>>>>>10,000 lines of code per second for an IBM-like minicomputer (less decimal
>>>>>>and string) and did a pretty good job of spitting out high performance
>>>>>>code; on a machine with a 150ns cycle time.
>>>>
>>>>> As did our COBOL compiler (which ran in 50KB). But in both cases,
>>>>> the languages were far simpler and much easier to generate efficient
>>>>> code than languages like Modula, Pascal, C, et alia.
>>>>
>>>>>>> Though, within moderate limits, 1M lines would basically be enough to fit:
>>>>>>> A basic kernel;
>>>>>>> (this excludes the Linux kernel, which is well over the size limit).
>>>>>>
>>>>>>If there were an efficient way to run the device driver sack in user-mode
>>>>>>without privilege and only the MMI/O pages this driver can touch mapped
>>>>>>into his VAS. Poof none of the driver stack is in the kernel. --IF--
>>>>
>>>>> That's actually quite common and one of the raison d'etre of the
>>>>> PCI Express SR-IOV feature. When you can present a virtual
>>>>> function to the user directly (mapping the MMIO region into
>>>>> the user mode virtual address space) the app had direct access
>>>>> to the hardware. Interrupts are the only tricky part, and
>>>>> the kernel virtio subsystem, which interfaces with the user
>>>>> application via shared memory provides interrupt handling
>>>>> to the application.
>>>>
>>>>> An I/OMMU provides memory protection for DMA operations initiated
>>>>> by the virtual function ensuring it only accesses the application
>>>>> virtual address space.
>>>>
>>>>Why should device be able to access user VaS outside of the buffer the
>>>>user provided, OH so long ago ??
>>
>>
>>> Because the device wants to do DMA directly into or from the users
>>> virtual address space. Bulk transfer, not MMIO accesses.
>>
>>OK, I will ask the question in the contrapositive way::
>>If the user ask device to read into a buffer, why does the device get
>>to see everything of the user's space along with that buffer ?

> It doesn't, necessarily. The IOMMU translation table is a
> proper subset of the user's virtual address space. The
> application tells the kernel which portions of the address
> space are valid DMA regions for the device to access.

Which is my point !! you only want the device to see that <small> subset
of the requesting application--not the whole address space. Done right
the device can still use the application virtual address, but the device
is not allowed to access stuff not associated with the request at hand
right now.

For example, you are a large entity and and Chinese disk drives are way
less expensive than non-Chinese; so you buy some. Would you let those
disk drives access anything in some requestors address space--no, you
would only allow that device to access the user supplied buffer and
whatever page rounding up that transpires.

Principle of least Privilege works in the I/O space too.

BGB-Alt wrote:

Why did you acquire an alt ?? Ego perhaps ??

BGB-Alt wrote:

> On 12/16/2023 1:25 PM, EricP wrote:
>> MitchAlsup wrote:

> One thing I don't get here is why there would be direct DMA between
> userland and the device (at least for filesystem and similar).

> Like, say, for a filesystem, it is presumably:
> read syscall from user to OS;
> route this to the corresponding VFS driver;
> Requests spanning multiple blocks being broken up into parts;
> VFS driver checks the block-cache / buffer-cache;
> If found, copy from cache into user-space;
> If not found, send request to the underlying block device;
> Wait for response (and/or reschedule task for later);
> Copy result back into userland.

This is correct enough for a file system buffered by a disk cache.

Are ALL file systems buffered in a disk cache ??

> Though, it may make sense that if a request isn't available immediately,
> and there is some sort of DMA mechanism, the OS could block the task and
> then resume it once the data becomes available. For polling IO, doesn't
> likely make much difference as the CPU is basically stuck in a busy loop
> either way until the IO finishes.

> Though, could make sense for hardware accelerating pixel-copying
> operations for a GUI.

> For GUI, there would be multiple stages of copying, say:
> Copying from user buffer to window buffer;
> Copying from window buffer to screen buffer;
> Copying from screen buffer to VRAM.

> For video playback or GL, there may be an additional stage of copying
> from GL's buffer to a user's buffer, then from the user's buffer to the
> window buffer. Though, considering possibly adding a shortcut path where
> GL and video codecs copy more directly into the window buffer (bypassing
> needing to pass the frame data through the userland program).

> Could be also possible maybe to have GL render directly into the window
> buffer, which could be possible if they have the same format/resolution,
> and the window buffer is physically mapped (say, for my current hardware
> rasterizer module).

> If running a program full-screen, it is possible to copy more directly
> from the user buffer into VRAM, saving some time here.

> Some time could be saved here if one had hardware support for these
> sorts of "copy pixel buffers around and convert between formats" tasks,
> but to be useful, this would need to be able to work with virtual
> memory, which adds some complexity (would either need to be CPU-like
> and/or have a page-walker; neither is particularly cheap).

I have MM (memory to memory move:: memmove() if you will) that transmits
up to 1 page of data as if atomically (single "bus" transaction.)

> Could maybe offload the task to the rasterizer module, but would need to
> add a page-walker to the rasterizer... Though, trying to deal with some
> scenarios (such as the final conversion/copy to VRAM) would add a lot of
> extra complexity. For now, its framebuffer/zbuffer/textures need to be
> in physically-mapped addresses (also with a 128-bit buffer alignment).

> Though, cheaper could be to make use of the second CPU core, but then
> schedule things like pixel copy operations to it (maybe also things like
> vertex transform and similar for OpenGL). Currently, if enabled, the
> second core hasn't seen a lot of use thus far in my case.

> ....