In article <u55psg$23uqp$1@dont-email.me>,
BGB-Alt <bohannonindustriesllc@gmail.com> wrote:
>On 5/30/2023 2:47 PM, Dan Cross wrote:
>> In article <u55c48$226ac$1@dont-email.me>, BGB <cr88192@gmail.com> wrote:
>>> I also mentioned that for all this I was *not* talking about an ISA for
>>> servers or data-centers, or even PC class systems, but was instead
>>> mostly focusing on embedded systems.
>>
>> Yup. Which seemed very odd when you posted things like this:
>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/qhN0N1_nAQAJ
>> in which you said,
>>
>> |Software cost is usually considered "virtually free" in comparison to
>> |hardware. So what if the virtual memory subsystem needs a few more kB
>> |due to a more complex hardware interface?, ...
>> |
>> |Would care more about performance if my benchmarks showed it "actually
>> |mattered" in terms of macro-scale performance.
>>
>> If, as you acknowledge, you aren't familiar with the problem
>> space, why make such statements?
>
>I have implemented virtual memory subsystems, and virtual memory *does*
>matter to some extent in my use-cases.

Yes, but we were talking about server workloads. I'm fairly
certain we made that clear.

>There is also a difference between "claiming to be an expert" and "not
>knowing anything whatsoever".

I never said you don't know anything whatsoever. I said you
would do well not to assume that you know more than you do; this
advice is universal.

>Just not to the extent that one needs to care about how its performance
>scales to something like a server or data-center.
>
>Think something with say 128 or 256 MB of RAM.
>Not kB, and not GB.

When I started out, we could fit 30-40 interactive timesharing
users on a VAX 8550 with 32 MiB of RAM, and departmental file
servers had about 128MiB of RAM. We cared very much how
performance scaled on such systems.

>You may still have a few GB of virtual address space though (and a few
>GB of pagefile).

You keep talking about paging to secondary storage; you brought
that up in alt.os.development and are talking about it again.
It seems entirely irrelevant.

>There are plenty of embedded systems where one still has a need for
>virtual memory.
>
>But, also ones where it may make sense to directly map things to
>underlying pages (without going through the pagefile), and to run
>multiple programs in a shared address space, etc.

I don't know why you keep talking about "the pagefile". Do you
think that use of a "pagefile" is necessary for using paged
virtual memory and hardware page tables?

>> Or when you said,
>>
>> |Most would not likely consider needing to use a few additional lookup
>> |tables or similar to be a significant issue.
>>
>> here:
>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/D3kUmGxHBgAJ
>>
>> even though we'd been telling you that it _did_ matter?
>
>ASID remapping tables still don't necessarily imply server or
>data-center workloads.

Again, that was the context; specifically, we were talking about
hypervisors. This was explained to you. You disagreed. Near
as I can tell, you don't have much relevant experience or
theoretical background on which to base such disagreement.

>They also don't necessarily mean that I am overly concerned with shaving
>a few kB off the OS kernel, nor are things "totally wrecked" by how many
>clock-cycles are spent in the TLB Miss ISR (within reason).

If you want to build an ISA for embedded applications with the
set of properties that you have implemented, have at it. But
recognize what, exactly, it actually is.

- Dan C.

On Tuesday, May 30, 2023 at 5:10:39 PM UTC-5, Dan Cross wrote:
> In article <u55psg$23uqp$1...@dont-email.me>,
> BGB-Alt <bohannonin...@gmail.com> wrote:
> >On 5/30/2023 2:47 PM, Dan Cross wrote:
> >> In article <u55c48$226ac$1...@dont-email.me>, BGB <cr8...@gmail.com> wrote:
> >>> I also mentioned that for all this I was *not* talking about an ISA for
> >>> servers or data-centers, or even PC class systems, but was instead
> >>> mostly focusing on embedded systems.
> >>
> >> Yup. Which seemed very odd when you posted things like this:
> >> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/qhN0N1_nAQAJ
> >> in which you said,
> >>
> >> |Software cost is usually considered "virtually free" in comparison to
> >> |hardware. So what if the virtual memory subsystem needs a few more kB
> >> |due to a more complex hardware interface?, ...
> >> |
> >> |Would care more about performance if my benchmarks showed it "actually
> >> |mattered" in terms of macro-scale performance.
> >>
> >> If, as you acknowledge, you aren't familiar with the problem
> >> space, why make such statements?
> >
> >I have implemented virtual memory subsystems, and virtual memory *does*
> >matter to some extent in my use-cases.
<
> Yes, but we were talking about server workloads. I'm fairly
> certain we made that clear.
<
I an not so sure we are talking server workloads, but I am sure we are not
talking about "small memory few application" design points. I guess I
would use the term "General Purpose workloads".
<
> >There is also a difference between "claiming to be an expert" and "not
> >knowing anything whatsoever".
<
> I never said you don't know anything whatsoever. I said you
> would do well not to assume that you know more than you do; this
> advice is universal.
<
Universal and nearly universally ignored.......
<
> >Just not to the extent that one needs to care about how its performance
> >scales to something like a server or data-center.
> >
> >Think something with say 128 or 256 MB of RAM.
> >Not kB, and not GB.
<
> When I started out, we could fit 30-40 interactive timesharing
> users on a VAX 8550 with 32 MiB of RAM, and departmental file
> servers had about 128MiB of RAM. We cared very much how
> performance scaled on such systems.
<
We got 8 people on a PDP-8 !! 10 people on a PDP-10, I and 30 people
on a System 360/67 with 22 disk drives. Although if you wanted real
performance from the /67 you showed up at 3:00 AM......
<
> >You may still have a few GB of virtual address space though (and a few
> >GB of pagefile).
> You keep talking about paging to secondary storage; you brought
> that up in alt.os.development and are talking about it again.
> It seems entirely irrelevant.
> >There are plenty of embedded systems where one still has a need for
> >virtual memory.
> >
> >But, also ones where it may make sense to directly map things to
> >underlying pages (without going through the pagefile), and to run
> >multiple programs in a shared address space, etc.
> I don't know why you keep talking about "the pagefile". Do you
> think that use of a "pagefile" is necessary for using paged
> virtual memory and hardware page tables?
> >> Or when you said,
> >>
> >> |Most would not likely consider needing to use a few additional lookup
> >> |tables or similar to be a significant issue.
> >>
> >> here:
> >> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/D3kUmGxHBgAJ
> >>
> >> even though we'd been telling you that it _did_ matter?
> >
> >ASID remapping tables still don't necessarily imply server or
> >data-center workloads.
<
I brought u the G-bit and ASIDs in order to expose a poor* design choice
made oh so long ago. I brought it up to illustrate that what seemed to
be a good-thing way back then, has morphed into another attack vector
today (or another imposition of HyperVisor activities.}
<
ASIDs are used to make the TLB/MMU more efficient, and occasionally
the upper layers of the cache hierarchy.
<
(*) where it was not until the advent of Hypervisors that the choice
went from "a good addition" to "oh crap we did that to ourselves"
{moderated, of course, by EricP's comments.}
<
> Again, that was the context; specifically, we were talking about
> hypervisors. This was explained to you. You disagreed. Near
> as I can tell, you don't have much relevant experience or
> theoretical background on which to base such disagreement.
<
> >They also don't necessarily mean that I am overly concerned with shaving
> >a few kB off the OS kernel, nor are things "totally wrecked" by how many
> >clock-cycles are spent in the TLB Miss ISR (within reason).
<
> If you want to build an ISA for embedded applications with the
> set of properties that you have implemented, have at it. But
> recognize what, exactly, it actually is.
<
and continue to caveat it for what it is.
>
> - Dan C.

MitchAlsup <MitchAlsup@aol.com> writes:
>On Tuesday, May 30, 2023 at 5:10:39=E2=80=AFPM UTC-5, Dan Cross wrote:

>> When I started out, we could fit 30-40 interactive timesharing=20
>> users on a VAX 8550 with 32 MiB of RAM, and departmental file=20
>> servers had about 128MiB of RAM. We cared very much how=20
>> performance scaled on such systems.
><
>We got 8 people on a PDP-8 !! 10 people on a PDP-10, I and 30 people
>on a System 360/67 with 22 disk drives. Although if you wanted real
>performance from the /67 you showed up at 3:00 AM......

We also had 8 people on a PDP-8 (TSS/8.24). We had a farm of
four 11/780's (4MB RAM each, plus a shared 4MB MA-780 unit),
front-ended by a custom PDP-11/45 terminal concentrator. This
farm supported all University departments and pretty much put
the 370 PCM (NAS, Itel) systems out of business.

We stashed the pascal and fortran compilers read-only regions
in the MA-780 such that they were shared by all four 780's
when running compiles.

Our internal management software utilized shared mailboxes
in the MA-780 for inter-host communications, as well as
routing DECnet through the MA-780 directly.

On 5/30/2023 6:09 PM, MitchAlsup wrote:
> On Tuesday, May 30, 2023 at 5:10:39 PM UTC-5, Dan Cross wrote:
>> In article <u55psg$23uqp$1...@dont-email.me>,
>> BGB-Alt <bohannonin...@gmail.com> wrote:
>>> On 5/30/2023 2:47 PM, Dan Cross wrote:
>>>> In article <u55c48$226ac$1...@dont-email.me>, BGB <cr8...@gmail.com> wrote:
>>>>> I also mentioned that for all this I was *not* talking about an ISA for
>>>>> servers or data-centers, or even PC class systems, but was instead
>>>>> mostly focusing on embedded systems.
>>>>
>>>> Yup. Which seemed very odd when you posted things like this:
>>>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/qhN0N1_nAQAJ
>>>> in which you said,
>>>>
>>>> |Software cost is usually considered "virtually free" in comparison to
>>>> |hardware. So what if the virtual memory subsystem needs a few more kB
>>>> |due to a more complex hardware interface?, ...
>>>> |
>>>> |Would care more about performance if my benchmarks showed it "actually
>>>> |mattered" in terms of macro-scale performance.
>>>>
>>>> If, as you acknowledge, you aren't familiar with the problem
>>>> space, why make such statements?
>>>
>>> I have implemented virtual memory subsystems, and virtual memory *does*
>>> matter to some extent in my use-cases.
> <
>> Yes, but we were talking about server workloads. I'm fairly
>> certain we made that clear.
> <
> I an not so sure we are talking server workloads, but I am sure we are not
> talking about "small memory few application" design points. I guess I
> would use the term "General Purpose workloads".
> <

Yeah.

IIRC, I said before that I was not talking about server workloads (nor
am I personally all that interested in server workloads...).

Though, there is a fuzzy line in terms of "general purpose".

One could be like:
Well, it is general purpose in that if someone did an ASIC version,
maybe they could stick it in a cellphone or something?...

>>> There is also a difference between "claiming to be an expert" and "not
>>> knowing anything whatsoever".
> <
>> I never said you don't know anything whatsoever. I said you
>> would do well not to assume that you know more than you do; this
>> advice is universal.
> <
> Universal and nearly universally ignored.......
> <
>>> Just not to the extent that one needs to care about how its performance
>>> scales to something like a server or data-center.
>>>
>>> Think something with say 128 or 256 MB of RAM.
>>> Not kB, and not GB.
> <
>> When I started out, we could fit 30-40 interactive timesharing
>> users on a VAX 8550 with 32 MiB of RAM, and departmental file
>> servers had about 128MiB of RAM. We cared very much how
>> performance scaled on such systems.
> <
> We got 8 people on a PDP-8 !! 10 people on a PDP-10, I and 30 people
> on a System 360/67 with 22 disk drives. Although if you wanted real
> performance from the /67 you showed up at 3:00 AM......
> <

Yeah, performance is to some extent relative though.

Also, there is the whole "Amdahl's Law" thing, ...

>>> You may still have a few GB of virtual address space though (and a few
>>> GB of pagefile).
>> You keep talking about paging to secondary storage; you brought
>> that up in alt.os.development and are talking about it again.
>> It seems entirely irrelevant.
>>> There are plenty of embedded systems where one still has a need for
>>> virtual memory.
>>>
>>> But, also ones where it may make sense to directly map things to
>>> underlying pages (without going through the pagefile), and to run
>>> multiple programs in a shared address space, etc.
>> I don't know why you keep talking about "the pagefile". Do you
>> think that use of a "pagefile" is necessary for using paged
>> virtual memory and hardware page tables?

No.

But, it is the most common use-case.

>>>> Or when you said,
>>>>
>>>> |Most would not likely consider needing to use a few additional lookup
>>>> |tables or similar to be a significant issue.
>>>>
>>>> here:
>>>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/D3kUmGxHBgAJ
>>>>
>>>> even though we'd been telling you that it _did_ matter?
>>>
>>> ASID remapping tables still don't necessarily imply server or
>>> data-center workloads.
> <
> I brought u the G-bit and ASIDs in order to expose a poor* design choice
> made oh so long ago. I brought it up to illustrate that what seemed to
> be a good-thing way back then, has morphed into another attack vector
> today (or another imposition of HyperVisor activities.}
> <
> ASIDs are used to make the TLB/MMU more efficient, and occasionally
> the upper layers of the cache hierarchy.
> <

Yes.

In my case, setting the ASID to 0 effectively marks a page as global.
But, yeah, probably don't want global pages in such a context.
Maybe it is better to not have any concept of global pages?...

> (*) where it was not until the advent of Hypervisors that the choice
> went from "a good addition" to "oh crap we did that to ourselves"
> {moderated, of course, by EricP's comments.}
> <

I guess this is a relevant point.

It is looking like "exact match only" for ASIDs is probably the
preferable strategy.

>> Again, that was the context; specifically, we were talking about
>> hypervisors. This was explained to you. You disagreed. Near
>> as I can tell, you don't have much relevant experience or
>> theoretical background on which to base such disagreement.
> <
>>> They also don't necessarily mean that I am overly concerned with shaving
>>> a few kB off the OS kernel, nor are things "totally wrecked" by how many
>>> clock-cycles are spent in the TLB Miss ISR (within reason).
> <
>> If you want to build an ISA for embedded applications with the
>> set of properties that you have implemented, have at it. But
>> recognize what, exactly, it actually is.
> <
> and continue to caveat it for what it is.

I had thought I had usually kept it "obvious enough" what BJX2 is and is
not meant to be.

I am also pretty sure "Hey, man, stick an FPGA soft processor in a web
server" was not one of them...

Well, even if I flip-flop some between "embedded controller tasks", and
"running Doom and Quake and similar".

Even my focus on neural nets is more in relation to "process image data
coming in from a camera module" (say, for directing motor-control tasks
or similar).

It taking a certain amount of "computational go power" to process
real-time video data from a camera module, or audio data from a
microphone, ...

But, in some sense, the features one needs for real-time image
processing happen to overlap to some extent with what one needs for
OpenGL rasterization, ...

Sometimes I had thought, maybe emulating x86 and running MS-DOS era
software and/or Win3.11 on it could be cool. But, as noted, this has
mostly gotten caught up in inertia.

According to MitchAlsup <MitchAlsup@aol.com>:
>> When I started out, we could fit 30-40 interactive timesharing
>> users on a VAX 8550 with 32 MiB of RAM, and departmental file
>> servers had about 128MiB of RAM. We cared very much how
>> performance scaled on such systems.
><
>We got 8 people on a PDP-8 !! 10 people on a PDP-10, I and 30 people
>on a System 360/67 with 22 disk drives. Although if you wanted real
>performance from the /67 you showed up at 3:00 AM......

In the 1970s DTSS ran over 200 users on a GE 635, about the same
performance as a KA-10, with very good response time. It was a real
timesharing system where you could write programs in a bunch of
different languages, not just BASIC. Its internal design was more like
that of SABRE or CICS, with most people talking to the transaction
monitor which only started a process for you when you typed RUN.
--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

On Tuesday, May 30, 2023 at 8:41:19 PM UTC-5, John Levine wrote:
> According to MitchAlsup <Mitch...@aol.com>:
> >> When I started out, we could fit 30-40 interactive timesharing
> >> users on a VAX 8550 with 32 MiB of RAM, and departmental file
> >> servers had about 128MiB of RAM. We cared very much how
> >> performance scaled on such systems.
> ><
> >We got 8 people on a PDP-8 !! 10 people on a PDP-10, I and 30 people
> >on a System 360/67 with 22 disk drives. Although if you wanted real
> >performance from the /67 you showed up at 3:00 AM......
<
> In the 1970s DTSS ran over 200 users on a GE 635, about the same
> performance as a KA-10, with very good response time. It was a real
> timesharing system where you could write programs in a bunch of
> different languages, not just BASIC. Its internal design was more like
> that of SABRE or CICS, with most people talking to the transaction
> monitor which only started a process for you when you typed RUN.
<
Late spring 1978, I was working for NCR corp in Cambridge Ohio
logged onto a CDC 7600 in San Diego on a TI Silent700, typing
in 8085 assembly code. All of a sudden the TI room went silent,
10 seconds past, 20 seconds past, then all of a sudden, the TI
were active again, but we were no logged onto a CDC 7600 in
Chicago.
<
We can't even get functionality like today................
> --
> Regards,
> John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
> Please consider the environment before reading this e-mail. https://jl.ly

In article <b1890135-df8e-4fd9-9127-d2fed8c9a797n@googlegroups.com>,
MitchAlsup <MitchAlsup@aol.com> wrote:
>On Tuesday, May 30, 2023 at 5:10:39 PM UTC-5, Dan Cross wrote:
>> In article <u55psg$23uqp$1...@dont-email.me>,
>> BGB-Alt <bohannonin...@gmail.com> wrote:
>> >On 5/30/2023 2:47 PM, Dan Cross wrote:
>> >> In article <u55c48$226ac$1...@dont-email.me>, BGB <cr8...@gmail.com> wrote:
>> >>> I also mentioned that for all this I was *not* talking about an ISA for
>> >>> servers or data-centers, or even PC class systems, but was instead
>> >>> mostly focusing on embedded systems.
>> >>
>> >> Yup. Which seemed very odd when you posted things like this:
>> >> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/qhN0N1_nAQAJ
>> >> in which you said,
>> >>
>> >> |Software cost is usually considered "virtually free" in comparison to
>> >> |hardware. So what if the virtual memory subsystem needs a few more kB
>> >> |due to a more complex hardware interface?, ...
>> >> |
>> >> |Would care more about performance if my benchmarks showed it "actually
>> >> |mattered" in terms of macro-scale performance.
>> >>
>> >> If, as you acknowledge, you aren't familiar with the problem
>> >> space, why make such statements?
>> >
>> >I have implemented virtual memory subsystems, and virtual memory *does*
>> >matter to some extent in my use-cases.
><
>> Yes, but we were talking about server workloads. I'm fairly
>> certain we made that clear.
><
>I an not so sure we are talking server workloads, but I am sure we are not
>talking about "small memory few application" design points. I guess I
>would use the term "General Purpose workloads".

We were in alt.os.development. The entire thing evolved out of
a discussion about x86S. Indeed, in hist first post on the
matter, BGB suggested his BJX2 as a replacement for x86:
https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/gIeuGffeCAAJ

In which he says,

|Ironically, if one goes over to software managed TLB, then the whole
|"nested page table" thing can disappear into the noise (as it is all
|software).

Which was the genesis of this entire thread of discussion: We
were explicitly talking about server-class CPUs. If he wants to
backtrack now and talk about embedded processors, whatever, but
that's certainly not how the discussion started.

>> >There is also a difference between "claiming to be an expert" and "not
>> >knowing anything whatsoever".
><
>> I never said you don't know anything whatsoever. I said you
>> would do well not to assume that you know more than you do; this
>> advice is universal.
><
>Universal and nearly universally ignored.......

Indeed. I myself an guilty.

>> >Just not to the extent that one needs to care about how its performance
>> >scales to something like a server or data-center.
>> >
>> >Think something with say 128 or 256 MB of RAM.
>> >Not kB, and not GB.
><
>> When I started out, we could fit 30-40 interactive timesharing
>> users on a VAX 8550 with 32 MiB of RAM, and departmental file
>> servers had about 128MiB of RAM. We cared very much how
>> performance scaled on such systems.
><
>We got 8 people on a PDP-8 !! 10 people on a PDP-10, I and 30 people
>on a System 360/67 with 22 disk drives. Although if you wanted real
>performance from the /67 you showed up at 3:00 AM......
><
>> >You may still have a few GB of virtual address space though (and a few
>> >GB of pagefile).
>> You keep talking about paging to secondary storage; you brought
>> that up in alt.os.development and are talking about it again.
>> It seems entirely irrelevant.
>> >There are plenty of embedded systems where one still has a need for
>> >virtual memory.
>> >
>> >But, also ones where it may make sense to directly map things to
>> >underlying pages (without going through the pagefile), and to run
>> >multiple programs in a shared address space, etc.
>> I don't know why you keep talking about "the pagefile". Do you
>> think that use of a "pagefile" is necessary for using paged
>> virtual memory and hardware page tables?
>> >> Or when you said,
>> >>
>> >> |Most would not likely consider needing to use a few additional lookup
>> >> |tables or similar to be a significant issue.
>> >>
>> >> here:
>> >> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/D3kUmGxHBgAJ
>> >>
>> >> even though we'd been telling you that it _did_ matter?
>> >
>> >ASID remapping tables still don't necessarily imply server or
>> >data-center workloads.
><
>I brought u the G-bit and ASIDs in order to expose a poor* design choice
>made oh so long ago. I brought it up to illustrate that what seemed to
>be a good-thing way back then, has morphed into another attack vector
>today (or another imposition of HyperVisor activities.}
><
>ASIDs are used to make the TLB/MMU more efficient, and occasionally
>the upper layers of the cache hierarchy.
><
>(*) where it was not until the advent of Hypervisors that the choice
>went from "a good addition" to "oh crap we did that to ourselves"
>{moderated, of course, by EricP's comments.}
><
>> Again, that was the context; specifically, we were talking about
>> hypervisors. This was explained to you. You disagreed. Near
>> as I can tell, you don't have much relevant experience or
>> theoretical background on which to base such disagreement.
><
>> >They also don't necessarily mean that I am overly concerned with shaving
>> >a few kB off the OS kernel, nor are things "totally wrecked" by how many
>> >clock-cycles are spent in the TLB Miss ISR (within reason).
><
>> If you want to build an ISA for embedded applications with the
>> set of properties that you have implemented, have at it. But
>> recognize what, exactly, it actually is.
><
>and continue to caveat it for what it is.

Indeed. And not draw inferences from that to server-class
workloads.

- Dan C.

In article <u566jj$25dja$1@dont-email.me>, BGB <cr88192@gmail.com> wrote:
> On Tuesday, May 30, 2023 at 5:10:39 PM UTC-5, Dan Cross wrote:
>> I don't know why you keep talking about "the pagefile". Do you
>> think that use of a "pagefile" is necessary for using paged
>> virtual memory and hardware page tables?
>
>No.
>
>But, it is the most common use-case.

Not even close.

- Dan C.

On 5/31/2023 7:04 AM, Dan Cross wrote:
> In article <b1890135-df8e-4fd9-9127-d2fed8c9a797n@googlegroups.com>,
> MitchAlsup <MitchAlsup@aol.com> wrote:
>> On Tuesday, May 30, 2023 at 5:10:39 PM UTC-5, Dan Cross wrote:
>>> In article <u55psg$23uqp$1...@dont-email.me>,
>>> BGB-Alt <bohannonin...@gmail.com> wrote:
>>>> On 5/30/2023 2:47 PM, Dan Cross wrote:
>>>>> In article <u55c48$226ac$1...@dont-email.me>, BGB <cr8...@gmail.com> wrote:
>>>>>> I also mentioned that for all this I was *not* talking about an ISA for
>>>>>> servers or data-centers, or even PC class systems, but was instead
>>>>>> mostly focusing on embedded systems.
>>>>>
>>>>> Yup. Which seemed very odd when you posted things like this:
>>>>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/qhN0N1_nAQAJ
>>>>> in which you said,
>>>>>
>>>>> |Software cost is usually considered "virtually free" in comparison to
>>>>> |hardware. So what if the virtual memory subsystem needs a few more kB
>>>>> |due to a more complex hardware interface?, ...
>>>>> |
>>>>> |Would care more about performance if my benchmarks showed it "actually
>>>>> |mattered" in terms of macro-scale performance.
>>>>>
>>>>> If, as you acknowledge, you aren't familiar with the problem
>>>>> space, why make such statements?
>>>>
>>>> I have implemented virtual memory subsystems, and virtual memory *does*
>>>> matter to some extent in my use-cases.
>> <
>>> Yes, but we were talking about server workloads. I'm fairly
>>> certain we made that clear.
>> <
>> I an not so sure we are talking server workloads, but I am sure we are not
>> talking about "small memory few application" design points. I guess I
>> would use the term "General Purpose workloads".
>
> We were in alt.os.development. The entire thing evolved out of
> a discussion about x86S. Indeed, in hist first post on the
> matter, BGB suggested his BJX2 as a replacement for x86:
> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/gIeuGffeCAAJ
>
> In which he says,
>
> |Ironically, if one goes over to software managed TLB, then the whole
> |"nested page table" thing can disappear into the noise (as it is all
> |software).
>
> Which was the genesis of this entire thread of discussion: We
> were explicitly talking about server-class CPUs. If he wants to
> backtrack now and talk about embedded processors, whatever, but
> that's certainly not how the discussion started.
>

I missed that point, it seems...
And/or forgot, I am not sure which.

But, yeah, I guess maybe one can argue it doesn't make sense for server
class CPUs, but there are uses for emulation/virtualization in embedded
contexts as well (I had more thought of virtualization more like
"emulation, but with hardware accelerated memory access").

Say, sort of like what is used in something like the Dolphin emulator
(for Game Cube and Wii), which tries to give the emulator its own native
address space on the hosts CPU, rather than emulating all of the memory
accesses.

One could conceivably use it as well in something like a "retro console"
or similar.

One could likely emulate x86 to good effect in any case, like what
Transmeta did.

I don't think the point was that BJX2 would replace x86 directly, but
used more like an emulator, but (I don't remember anyways) thinking of
that having been in a server context (as opposed to, say, something more
like DOSBox or QEMU).

>>>> There is also a difference between "claiming to be an expert" and "not
>>>> knowing anything whatsoever".
>> <
>>> I never said you don't know anything whatsoever. I said you
>>> would do well not to assume that you know more than you do; this
>>> advice is universal.
>> <
>> Universal and nearly universally ignored.......
>
> Indeed. I myself an guilty.
>
>>>> Just not to the extent that one needs to care about how its performance
>>>> scales to something like a server or data-center.
>>>>
>>>> Think something with say 128 or 256 MB of RAM.
>>>> Not kB, and not GB.
>> <
>>> When I started out, we could fit 30-40 interactive timesharing
>>> users on a VAX 8550 with 32 MiB of RAM, and departmental file
>>> servers had about 128MiB of RAM. We cared very much how
>>> performance scaled on such systems.
>> <
>> We got 8 people on a PDP-8 !! 10 people on a PDP-10, I and 30 people
>> on a System 360/67 with 22 disk drives. Although if you wanted real
>> performance from the /67 you showed up at 3:00 AM......
>> <
>>>> You may still have a few GB of virtual address space though (and a few
>>>> GB of pagefile).
>>> You keep talking about paging to secondary storage; you brought
>>> that up in alt.os.development and are talking about it again.
>>> It seems entirely irrelevant.
>>>> There are plenty of embedded systems where one still has a need for
>>>> virtual memory.
>>>>
>>>> But, also ones where it may make sense to directly map things to
>>>> underlying pages (without going through the pagefile), and to run
>>>> multiple programs in a shared address space, etc.
>>> I don't know why you keep talking about "the pagefile". Do you
>>> think that use of a "pagefile" is necessary for using paged
>>> virtual memory and hardware page tables?
>>>>> Or when you said,
>>>>>
>>>>> |Most would not likely consider needing to use a few additional lookup
>>>>> |tables or similar to be a significant issue.
>>>>>
>>>>> here:
>>>>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/D3kUmGxHBgAJ
>>>>>
>>>>> even though we'd been telling you that it _did_ matter?
>>>>
>>>> ASID remapping tables still don't necessarily imply server or
>>>> data-center workloads.
>> <
>> I brought u the G-bit and ASIDs in order to expose a poor* design choice
>> made oh so long ago. I brought it up to illustrate that what seemed to
>> be a good-thing way back then, has morphed into another attack vector
>> today (or another imposition of HyperVisor activities.}
>> <
>> ASIDs are used to make the TLB/MMU more efficient, and occasionally
>> the upper layers of the cache hierarchy.
>> <
>> (*) where it was not until the advent of Hypervisors that the choice
>> went from "a good addition" to "oh crap we did that to ourselves"
>> {moderated, of course, by EricP's comments.}
>> <
>>> Again, that was the context; specifically, we were talking about
>>> hypervisors. This was explained to you. You disagreed. Near
>>> as I can tell, you don't have much relevant experience or
>>> theoretical background on which to base such disagreement.
>> <
>>>> They also don't necessarily mean that I am overly concerned with shaving
>>>> a few kB off the OS kernel, nor are things "totally wrecked" by how many
>>>> clock-cycles are spent in the TLB Miss ISR (within reason).
>> <
>>> If you want to build an ISA for embedded applications with the
>>> set of properties that you have implemented, have at it. But
>>> recognize what, exactly, it actually is.
>> <
>> and continue to caveat it for what it is.
>
> Indeed. And not draw inferences from that to server-class
> workloads.
>
> - Dan C.
>

MitchAlsup wrote:
> I recently added process priority to my ATOMIC feature and I am thinking
> about adding privilege level, too::
> <
> If a higher priority process accesses a cache line participating in an ATOMIC
> event, the lower priority process holding this line fails its ATOMIC event and
> the higher priority process gets to make forward progress.
> <
> If a lower priority process accesses a cache line participating in an ATOMIC
> event, the lower priority process fails its ATOMIC event and the higher
> priority process gets to continue to make forward progress.
> <
> If a process receives a NaK and was attempting an ATOMIC event, then
> the event fails (and you start over at the top), if it was not attempting
> an event, then the access is restarted. Only a process attempting an
> ATOMIC event that has already touched all of its cache lines can
> respond with a NaK.

Allowing priority to jump ahead in a service queue removes any
bounded delay guarantees for the lower priority and allows starvation.
I think this priority feature should be an option selectable at the
start of each transaction.

On Wednesday, May 31, 2023 at 12:57:52 PM UTC-5, EricP wrote:
> MitchAlsup wrote:
> > I recently added process priority to my ATOMIC feature and I am thinking
> > about adding privilege level, too::
> > <
> > If a higher priority process accesses a cache line participating in an ATOMIC
> > event, the lower priority process holding this line fails its ATOMIC event and
> > the higher priority process gets to make forward progress.
> > <
> > If a lower priority process accesses a cache line participating in an ATOMIC
> > event, the lower priority process fails its ATOMIC event and the higher
> > priority process gets to continue to make forward progress.
> > <
> > If a process receives a NaK and was attempting an ATOMIC event, then
> > the event fails (and you start over at the top), if it was not attempting
> > an event, then the access is restarted. Only a process attempting an
> > ATOMIC event that has already touched all of its cache lines can
> > respond with a NaK.
<
> Allowing priority to jump ahead in a service queue removes any
> bounded delay guarantees for the lower priority and allows starvation.
> I think this priority feature should be an option selectable at the
> start of each transaction.
<
What do you do when the lower priority thread does not want said
option, but the higher priority thread does ?
<
Also note: If you don't do something like this, you get more priority
inversion, where lower priority threads steel cycles from higher priority
threads.

On 5/31/2023 10:57 AM, EricP wrote:
> MitchAlsup wrote:
>> I recently added process priority to my ATOMIC feature and I am thinking
>> about adding privilege level, too::
>> <
>> If a higher priority process accesses a cache line participating in an
>> ATOMIC
>> event, the lower priority process holding this line fails its ATOMIC
>> event and the higher priority process gets to make forward progress.
>> <
>> If a lower priority process accesses a cache line participating in an
>> ATOMIC
>> event, the lower priority process fails its ATOMIC event and the higher
>> priority process gets to continue to make forward progress.
>> <
>> If a process receives a NaK and was attempting an ATOMIC event, then
>> the event fails (and you start over at the top), if it was not attempting
>> an event, then the access is restarted. Only a process attempting an
>> ATOMIC event that has already touched all of its cache lines can
>> respond with a NaK.
>
> Allowing priority to jump ahead in a service queue removes any
> bounded delay guarantees for the lower priority and allows starvation.

Yes. There are several partial solutions, and lots of variations, but I
don't think there is a "one size fits all" solution.

> I think this priority feature should be an option selectable at the
> start of each transaction.

So where does a request without priority indicator rank versus one with
the indicator? either way, you still have the possibility of
starvation. And you need some sort of enforced policy to say who is
allowed to use which option.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Wednesday, May 31, 2023 at 2:11:10 PM UTC-5, Stephen Fuld wrote:
> On 5/31/2023 10:57 AM, EricP wrote:
> > MitchAlsup wrote:
> >> I recently added process priority to my ATOMIC feature and I am thinking
> >> about adding privilege level, too::
> >> <
> >> If a higher priority process accesses a cache line participating in an
> >> ATOMIC
> >> event, the lower priority process holding this line fails its ATOMIC
> >> event and the higher priority process gets to make forward progress.
> >> <
> >> If a lower priority process accesses a cache line participating in an
> >> ATOMIC
> >> event, the lower priority process fails its ATOMIC event and the higher
> >> priority process gets to continue to make forward progress.
> >> <
> >> If a process receives a NaK and was attempting an ATOMIC event, then
> >> the event fails (and you start over at the top), if it was not attempting
> >> an event, then the access is restarted. Only a process attempting an
> >> ATOMIC event that has already touched all of its cache lines can
> >> respond with a NaK.
> >
> > Allowing priority to jump ahead in a service queue removes any
> > bounded delay guarantees for the lower priority and allows starvation.
> Yes. There are several partial solutions, and lots of variations, but I
> don't think there is a "one size fits all" solution.
> > I think this priority feature should be an option selectable at the
> > start of each transaction.
> So where does a request without priority indicator rank versus one with
> the indicator? either way, you still have the possibility of
> starvation. And you need some sort of enforced policy to say who is
> allowed to use which option.
<
Also note: A higher priority ATOMIC event only interferes with a lower
priority event when the same data is touched. So, if the HPE and the LPE
use different locks, there is no interference.
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

MitchAlsup <MitchAlsup@aol.com> writes:
>On Wednesday, May 31, 2023 at 2:11:10=E2=80=AFPM UTC-5, Stephen Fuld wrote:
>> On 5/31/2023 10:57 AM, EricP wrote:=20
>> > MitchAlsup wrote:=20
>> >> I recently added process priority to my ATOMIC feature and I am thinki=
>ng=20
>> >> about adding privilege level, too::=20
>> >> <=20
>> >> If a higher priority process accesses a cache line participating in an=
>=20
>> >> ATOMIC=20
>> >> event, the lower priority process holding this line fails its ATOMIC=
>=20
>> >> event and the higher priority process gets to make forward progress.=
>=20
>> >> <=20
>> >> If a lower priority process accesses a cache line participating in an=
>=20
>> >> ATOMIC=20
>> >> event, the lower priority process fails its ATOMIC event and the highe=
>r=20
>> >> priority process gets to continue to make forward progress.=20
>> >> <=20
>> >> If a process receives a NaK and was attempting an ATOMIC event, then=
>=20
>> >> the event fails (and you start over at the top), if it was not attempt=
>ing=20
>> >> an event, then the access is restarted. Only a process attempting an=
>=20
>> >> ATOMIC event that has already touched all of its cache lines can=20
>> >> respond with a NaK.=20
>> >=20
>> > Allowing priority to jump ahead in a service queue removes any=20
>> > bounded delay guarantees for the lower priority and allows starvation.
>> Yes. There are several partial solutions, and lots of variations, but I=
>=20
>> don't think there is a "one size fits all" solution.
>> > I think this priority feature should be an option selectable at the=20
>> > start of each transaction.
>> So where does a request without priority indicator rank versus one with=
>=20
>> the indicator? either way, you still have the possibility of=20
>> starvation. And you need some sort of enforced policy to say who is=20
>> allowed to use which option.
><
>Also note: A higher priority ATOMIC event only interferes with a lower
>priority event when the same data is touched. So, if the HPE and the LPE
>use different locks, there is no interference.

One of the acid tests at a national lab that will go unnamed is for
all cores to pound on a single cache line (using a spin lock). Starvation
was a 'fail'.

AMD (and I presume Intel) have a feature in the L2 that will, if it
cannot get ownership of the cache line in a certain amount of time,
assert the system-wide bus lock (which I would hope is granted round
robin, not based on any notion of priority) in order to make forward
progress.

On Wednesday, May 31, 2023 at 2:57:48 PM UTC-5, Scott Lurndal wrote:
> MitchAlsup <Mitch...@aol.com> writes:
> >On Wednesday, May 31, 2023 at 2:11:10=E2=80=AFPM UTC-5, Stephen Fuld wrote:
> >> On 5/31/2023 10:57 AM, EricP wrote:=20
> >> > MitchAlsup wrote:=20
> >> >> I recently added process priority to my ATOMIC feature and I am thinki=
> >ng=20
> >> >> about adding privilege level, too::=20
> >> >> <=20
> >> >> If a higher priority process accesses a cache line participating in an=
> >=20
> >> >> ATOMIC=20
> >> >> event, the lower priority process holding this line fails its ATOMIC=
> >=20
> >> >> event and the higher priority process gets to make forward progress..=
> >=20
> >> >> <=20
> >> >> If a lower priority process accesses a cache line participating in an=
> >=20
> >> >> ATOMIC=20
> >> >> event, the lower priority process fails its ATOMIC event and the highe=
> >r=20
> >> >> priority process gets to continue to make forward progress.=20
> >> >> <=20
> >> >> If a process receives a NaK and was attempting an ATOMIC event, then=
> >=20
> >> >> the event fails (and you start over at the top), if it was not attempt=
> >ing=20
> >> >> an event, then the access is restarted. Only a process attempting an=
> >=20
> >> >> ATOMIC event that has already touched all of its cache lines can=20
> >> >> respond with a NaK.=20
> >> >=20
> >> > Allowing priority to jump ahead in a service queue removes any=20
> >> > bounded delay guarantees for the lower priority and allows starvation.
> >> Yes. There are several partial solutions, and lots of variations, but I=
> >=20
> >> don't think there is a "one size fits all" solution.
> >> > I think this priority feature should be an option selectable at the=20
> >> > start of each transaction.
> >> So where does a request without priority indicator rank versus one with=
> >=20
> >> the indicator? either way, you still have the possibility of=20
> >> starvation. And you need some sort of enforced policy to say who is=20
> >> allowed to use which option.
> ><
> >Also note: A higher priority ATOMIC event only interferes with a lower
> >priority event when the same data is touched. So, if the HPE and the LPE
> >use different locks, there is no interference.
<
> One of the acid tests at a national lab that will go unnamed is for
> all cores to pound on a single cache line (using a spin lock). Starvation
> was a 'fail'.
<
Are the contenders of the same priority ?? Are they of the same Privilege ??
>
> AMD (and I presume Intel) have a feature in the L2 that will, if it
> cannot get ownership of the cache line in a certain amount of time,
> assert the system-wide bus lock (which I would hope is granted round
> robin, not based on any notion of priority) in order to make forward
> progress.

On Wednesday, May 31, 2023 at 2:11:10 PM UTC-5, Stephen Fuld wrote:
> On 5/31/2023 10:57 AM, EricP wrote:
> > MitchAlsup wrote:
> >> I recently added process priority to my ATOMIC feature and I am thinking
> >> about adding privilege level, too::
> >> <
> >> If a higher priority process accesses a cache line participating in an
> >> ATOMIC
> >> event, the lower priority process holding this line fails its ATOMIC
> >> event and the higher priority process gets to make forward progress.
> >> <
> >> If a lower priority process accesses a cache line participating in an
> >> ATOMIC
> >> event, the lower priority process fails its ATOMIC event and the higher
> >> priority process gets to continue to make forward progress.
> >> <
> >> If a process receives a NaK and was attempting an ATOMIC event, then
> >> the event fails (and you start over at the top), if it was not attempting
> >> an event, then the access is restarted. Only a process attempting an
> >> ATOMIC event that has already touched all of its cache lines can
> >> respond with a NaK.
> >
> > Allowing priority to jump ahead in a service queue removes any
> > bounded delay guarantees for the lower priority and allows starvation.
> Yes. There are several partial solutions, and lots of variations, but I
> don't think there is a "one size fits all" solution.
> > I think this priority feature should be an option selectable at the
> > start of each transaction.
<
> So where does a request without priority indicator rank versus one with
> the indicator? either way, you still have the possibility of
> starvation. And you need some sort of enforced policy to say who is
> allowed to use which option.
<
Ok, I see the problem.
<
The ATOMIC primitives in My 66000 are not "like" those in other architectures.
Like LL-SC there is a duration over which the event is taking place, and unlike
LL-SC up to 8 cache lines can participate. It is only this duration that is subject
to priority.
<
But the way SW uses these features is that some thread grabs some lock
and the grabbing of that lock is the beginning of the critical section--and
the grabbing of that lock is what is ATOMIC.
<
When the primitives in My 66000 are used to "grab a lock" the duration of
the grabbing is 1-or-2 instructions, whereas the critical section that follows
is of unbounded length. It is the grabbing of the lock that is subject to priority,
and once the lock has been grabbed, the critical section can proceed apace.
<
My 66000 ATOMIC events can be significantly more powerful--such as moving
an element from one place to another is a single event--which requires 5 cache
lines to participate. My 66000 LL-SC equivalents can implement LL-SC, TS, TTS,
CAS, double compare, single swap, double compare double swap, double compare
triple swap, .....
<
But if you use these the way SW typically uses the atomic instructions on other
architectures, the duration of when priority is considered is only the grabbing of
the lock an dis not associated with the critical section that successful grabbling
of the lock enabled.
<
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On Sunday, May 28, 2023 at 4:02:07 PM UTC+3, Thomas Koenig wrote:
> Scott Lurndal <sc...@slp53.sl.home> schrieb:
>
> >> Power and PowerPC
>
> [...]
>
> > Obsolete CPUs, all designed three decades ago.
>
> Power10 became available in 2021, and Power11 (presumably) is on the
> way; patches are already appearing in gcc.

And, of course, both of them have HW-managed TLBs.

POWER10 hardware page walkers has not one but *two* modes of operation.
Hashed Page Table (HPT) mode is here in order to support old versions of AIX.
Radix mode is very similar to x86-64/ARM64 and used by Linux.
I don't know which of them used by newest AIX.
Also I don't know if POWER11 still supports HPT.

On POWER10, when a TLB miss occurs, up to four page-walk requests can be
processed concurrently, which, I think, at least matches capabilities of big
x86-64 and ARM64 cores and may be even exceeds them.

On 5/31/2023 12:26 PM, BGB wrote:
> On 5/31/2023 7:04 AM, Dan Cross wrote:
>> In article <b1890135-df8e-4fd9-9127-d2fed8c9a797n@googlegroups.com>,
>> MitchAlsup  <MitchAlsup@aol.com> wrote:
>>> On Tuesday, May 30, 2023 at 5:10:39 PM UTC-5, Dan Cross wrote:
>>>> In article <u55psg$23uqp$1...@dont-email.me>,
>>>> BGB-Alt <bohannonin...@gmail.com> wrote:
>>>>> On 5/30/2023 2:47 PM, Dan Cross wrote:
>>>>>> In article <u55c48$226ac$1...@dont-email.me>, BGB
>>>>>> <cr8...@gmail.com> wrote:
>>>>>>> I also mentioned that for all this I was *not* talking about an
>>>>>>> ISA for
>>>>>>> servers or data-centers, or even PC class systems, but was instead
>>>>>>> mostly focusing on embedded systems.
>>>>>>
>>>>>> Yup. Which seemed very odd when you posted things like this:
>>>>>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/qhN0N1_nAQAJ
>>>>>> in which you said,
>>>>>>
>>>>>> |Software cost is usually considered "virtually free" in
>>>>>> comparison to
>>>>>> |hardware. So what if the virtual memory subsystem needs a few
>>>>>> more kB
>>>>>> |due to a more complex hardware interface?, ...
>>>>>> |
>>>>>> |Would care more about performance if my benchmarks showed it
>>>>>> "actually
>>>>>> |mattered" in terms of macro-scale performance.
>>>>>>
>>>>>> If, as you acknowledge, you aren't familiar with the problem
>>>>>> space, why make such statements?
>>>>>
>>>>> I have implemented virtual memory subsystems, and virtual memory
>>>>> *does*
>>>>> matter to some extent in my use-cases.
>>> <
>>>> Yes, but we were talking about server workloads. I'm fairly
>>>> certain we made that clear.
>>> <
>>> I an not so sure we are talking server workloads, but I am sure we
>>> are not
>>> talking about "small memory few application" design points. I guess I
>>> would use the term "General Purpose workloads".
>>
>> We were in alt.os.development.  The entire thing evolved out of
>> a discussion about x86S.  Indeed, in hist first post on the
>> matter, BGB suggested his BJX2 as a replacement for x86:
>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/gIeuGffeCAAJ
>>
>> In which he says,
>>
>> |Ironically, if one goes over to software managed TLB, then the whole
>> |"nested page table" thing can disappear into the noise (as it is all
>> |software).
>>
>> Which was the genesis of this entire thread of discussion: We
>> were explicitly talking about server-class CPUs.  If he wants to
>> backtrack now and talk about embedded processors, whatever, but
>> that's certainly not how the discussion started.
>>
>
> I missed that point, it seems...
> And/or forgot, I am not sure which.
>
>
> But, yeah, I guess maybe one can argue it doesn't make sense for server
> class CPUs, but there are uses for emulation/virtualization in embedded
> contexts as well (I had more thought of virtualization more like
> "emulation, but with hardware accelerated memory access").
>

So, not so much "whatever it is exactly they are using virtualization
for in servers", and more like "multi-way virtual console for running
retro games" (with "virtualization" being more understood as using
hardware-accelerated address translation, rather than software emulating
every memory access).

Say, if you could emulate MS-DOS and era-appropriate games, or maybe
emulate a PlayStation or Dreamcast, etc...

Where, say, if sensibly designed, the underlying hardware shouldn't
hinder emulating the various hardware behavior and addressing schemes
used by the various types of game consoles and other things.

Not all of which use page tables, nor even necessarily particularly
linear addressing schemes. But, do generally use schemes defined for the
original hardware in question.

It doesn't seem desirable to directly emulate nearly every one of the
possible addressing schemes directly in hardware. But, say, one wants
hardware that can mimic the various addressing schemes that may be in use.

Or, basically, sort of like an N-way multi-architecture virtual machine...

At the moment, this is still all a bit hypothetical though.

>
> Say, sort of like what is used in something like the Dolphin emulator
> (for Game Cube and Wii), which tries to give the emulator its own native
> address space on the hosts CPU, rather than emulating all of the memory
> accesses.
>

IIRC, it was something like (on capable hardware) they create a
dedicated virtual address space which can mimic the native address space
used by the game consoles.

When this sort of thing is done, apart from inefficiencies in the JIT
translation or emulating features which don't map from one ISA to
another, it being possible to get "near native" performance.

> One could conceivably use it as well in something like a "retro console"
> or similar.
>
>
> One could likely emulate x86 to good effect in any case, like what
> Transmeta did.
>
> I don't think the point was that BJX2 would replace x86 directly, but
> used more like an emulator, but (I don't remember anyways) thinking of
> that having been in a server context (as opposed to, say, something more
> like DOSBox or QEMU).
>

But, yeah, IIRC, part of my motivation for the original line of thinking
is that, once x86-64 drops the 'x86' part, it has effectively lost a big
part of why a person might want to use x86 in the first place (versus,
say, ARM or RISC-V).

Namely: Backwards compatibility with older 32-bit era software, and/or
the ability to set up "retro boxes" running MS-DOS and similar.

Though, did see a video with a guy basically going through a lot of
hassle trying to get MS-DOS running effectively on a Core 2 (and the
issue that most more-modern audio chipsets lack backwards compatibility
with Sound-Blaster and Adlib). And, trying to hack around the Core 2
being too fast and breaking the delay-loops used in a lot of older games
(setting the CPU clock speed as slow as possible, and the memory latency
as high as possible), ...

But, apparently, trying to run MS-DOS "natively" on anything much newer
being an exercise in futility.

AFAIK, ARM could (in theory) do a pretty good emulation layer for x86
(glacial DOSBox performance notwithstanding), but is not an open ISA.

RISC-V is an open ISA, but I am skeptical about how high performance an
emulator one could implement on it (short of significant ISA extensions).

What all does this leave as alternatives?...

Granted, most of the faster targets aren't going to work on current
FPGAs, so this part is hypothetical in that one would likely need an
ASIC implementation of the processors.

At present, an ASIC implementation of BJX2 probably isn't going to
happen though.

And, say, there are limits as to what kinds of hardware one can emulate
directly in an FPGA (say, in something like the MiSTer project), due
mostly to limitations in FPGA size and the maximum clock-speeds one can
support.

....

>
>
>>>>> There is also a difference between "claiming to be an expert" and "not
>>>>> knowing anything whatsoever".
>>> <
>>>> I never said you don't know anything whatsoever. I said you
>>>> would do well not to assume that you know more than you do; this
>>>> advice is universal.
>>> <
>>> Universal and nearly universally ignored.......
>>
>> Indeed.  I myself an guilty.
>>
>>>>> Just not to the extent that one needs to care about how its
>>>>> performance
>>>>> scales to something like a server or data-center.
>>>>>
>>>>> Think something with say 128 or 256 MB of RAM.
>>>>> Not kB, and not GB.
>>> <
>>>> When I started out, we could fit 30-40 interactive timesharing
>>>> users on a VAX 8550 with 32 MiB of RAM, and departmental file
>>>> servers had about 128MiB of RAM. We cared very much how
>>>> performance scaled on such systems.
>>> <
>>> We got 8 people on a PDP-8 !! 10 people on a PDP-10, I and 30 people
>>> on a System 360/67 with 22 disk drives. Although if you wanted real
>>> performance from the /67 you showed up at 3:00 AM......
>>> <
>>>>> You may still have a few GB of virtual address space though (and a few
>>>>> GB of pagefile).
>>>> You keep talking about paging to secondary storage; you brought
>>>> that up in alt.os.development and are talking about it again.
>>>> It seems entirely irrelevant.
>>>>> There are plenty of embedded systems where one still has a need for
>>>>> virtual memory.
>>>>>
>>>>> But, also ones where it may make sense to directly map things to
>>>>> underlying pages (without going through the pagefile), and to run
>>>>> multiple programs in a shared address space, etc.
>>>> I don't know why you keep talking about "the pagefile". Do you
>>>> think that use of a "pagefile" is necessary for using paged
>>>> virtual memory and hardware page tables?
>>>>>> Or when you said,
>>>>>>
>>>>>> |Most would not likely consider needing to use a few additional
>>>>>> lookup
>>>>>> |tables or similar to be a significant issue.
>>>>>>
>>>>>> here:
>>>>>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/D3kUmGxHBgAJ
>>>>>>
>>>>>> even though we'd been telling you that it _did_ matter?
>>>>>
>>>>> ASID remapping tables still don't necessarily imply server or
>>>>> data-center workloads.
>>> <
>>> I brought u the G-bit and ASIDs in order to expose a poor* design choice
>>> made oh so long ago. I brought it up to illustrate that what seemed to
>>> be a good-thing way back then, has morphed into another attack vector
>>> today (or another imposition of HyperVisor activities.}
>>> <
>>> ASIDs are used to make the TLB/MMU more efficient, and occasionally
>>> the upper layers of the cache hierarchy.
>>> <
>>> (*) where it was not until the advent of Hypervisors that the choice
>>> went from "a good addition" to "oh crap we did that to ourselves"
>>> {moderated, of course, by EricP's comments.}
>>> <
>>>> Again, that was the context; specifically, we were talking about
>>>> hypervisors. This was explained to you. You disagreed. Near
>>>> as I can tell, you don't have much relevant experience or
>>>> theoretical background on which to base such disagreement.
>>> <
>>>>> They also don't necessarily mean that I am overly concerned with
>>>>> shaving
>>>>> a few kB off the OS kernel, nor are things "totally wrecked" by how
>>>>> many
>>>>> clock-cycles are spent in the TLB Miss ISR (within reason).
>>> <
>>>> If you want to build an ISA for embedded applications with the
>>>> set of properties that you have implemented, have at it. But
>>>> recognize what, exactly, it actually is.
>>> <
>>> and continue to caveat it for what it is.
>>
>> Indeed.  And not draw inferences from that to server-class
>> workloads.
>>
>>     - Dan C.
>>
>

On 5/31/2023 4:09 PM, Michael S wrote:
> On Sunday, May 28, 2023 at 4:02:07 PM UTC+3, Thomas Koenig wrote:
>> Scott Lurndal <sc...@slp53.sl.home> schrieb:
>>
>>>> Power and PowerPC
>>
>> [...]
>>
>>> Obsolete CPUs, all designed three decades ago.
>>
>> Power10 became available in 2021, and Power11 (presumably) is on the
>> way; patches are already appearing in gcc.
>
> And, of course, both of them have HW-managed TLBs.
>
> POWER10 hardware page walkers has not one but *two* modes of operation.
> Hashed Page Table (HPT) mode is here in order to support old versions of AIX.
> Radix mode is very similar to x86-64/ARM64 and used by Linux.
> I don't know which of them used by newest AIX.
> Also I don't know if POWER11 still supports HPT.
>
> On POWER10, when a TLB miss occurs, up to four page-walk requests can be
> processed concurrently, which, I think, at least matches capabilities of big
> x86-64 and ARM64 cores and may be even exceeds them.

Yeah, it soon became apparent (from more looking) that my initial survey
and classification was flawed (it was based mostly on quick-and-dirty
web-searches, as my first-hand experience isn't quite that extensive).

Power and PowerPC apparently using different strategies based on the
model (rather than one strategy across the whole processor line).

IA-64 using Inverted Page Tables, rather the SW TLB.

....

I had started looking at possibly adding "Inverted Page Table" support
to BJX2, as it most closely matches the existing architecture and
implementation.

There are traditional page-tables as well, but at present it seems
better "at the moment" to leave these up to software.

Had specified a "Hash+N" table strategy (1), which will try to at least
be viable for a direct hardware page-walker implementation (while
avoiding the ASLR + Addr96 = BAD issue).

But, harder is coming up with a good hashing scheme that remains
consistent across page-size and table depth (and would work well for
both a HW and SW implementation).

Current variants are mostly along the lines of "mask the high-address to
exclude the parts covered by the page table; and XOR the bits together",
but needs to be able to produce a 8, 10, or 12 bit hash (depending on
page size).

Though, one option being to XOR things down to 16-bits, and then use a
final XOR based on the page size.

1: Top-level as a hash table, with lower levels as conventional page table.

MitchAlsup wrote:
> On Wednesday, May 31, 2023 at 12:57:52 PM UTC-5, EricP wrote:
>> MitchAlsup wrote:
>>> I recently added process priority to my ATOMIC feature and I am thinking
>>> about adding privilege level, too::
>>> <
>>> If a higher priority process accesses a cache line participating in an ATOMIC
>>> event, the lower priority process holding this line fails its ATOMIC event and
>>> the higher priority process gets to make forward progress.
>>> <
>>> If a lower priority process accesses a cache line participating in an ATOMIC
>>> event, the lower priority process fails its ATOMIC event and the higher
>>> priority process gets to continue to make forward progress.
>>> <
>>> If a process receives a NaK and was attempting an ATOMIC event, then
>>> the event fails (and you start over at the top), if it was not attempting
>>> an event, then the access is restarted. Only a process attempting an
>>> ATOMIC event that has already touched all of its cache lines can
>>> respond with a NaK.
> <
>> Allowing priority to jump ahead in a service queue removes any
>> bounded delay guarantees for the lower priority and allows starvation.
>> I think this priority feature should be an option selectable at the
>> start of each transaction.
> <
> What do you do when the lower priority thread does not want said
> option, but the higher priority thread does ?
> <
> Also note: If you don't do something like this, you get more priority
> inversion, where lower priority threads steel cycles from higher priority
> threads.

I think it is a good idea - yes, it is the difference between being
priority inversion oblivious (PIO) and priority inversion aware (PIA)
and I think both are valid.

My knee jerk concern is PIA could make ESM less usable in real time contexts,
either by making transactions have unbounded delay or by forcing users to
raise their priority to a common level before and lower it after to
ensure predictable behavior and negate much of ESM's advantage.

And I don't think the scenario you describe is likely -
that threads would share some virtual space, have common data structures,
coordinate their locking and updates in common access code,
but then disagree as to whether it should use PIO or PIA.

Also it is easily resolvable. This only matters on collisions:
- If both are oblivious use fifo.
- If both are aware choose higher priority, if equal priority use fifo.
- if aware and oblivious, pick the one that cares and choose aware.

MitchAlsup <MitchAlsup@aol.com> writes:
>On Wednesday, May 31, 2023 at 2:57:48=E2=80=AFPM UTC-5, Scott Lurndal wrote=
>:

>> One of the acid tests at a national lab that will go unnamed is for=20
>> all cores to pound on a single cache line (using a spin lock). Starvation=
>=20
>> was a 'fail'.=20
><
>Are the contenders of the same priority ?? Are they of the same Privilege ?=

Yes. Although I'm not aware of either priority or privilege
being factored into Intel or AMD's coherency protocols.

On Wednesday, May 31, 2023 at 5:32:41 PM UTC-5, EricP wrote:
> MitchAlsup wrote:
> > On Wednesday, May 31, 2023 at 12:57:52 PM UTC-5, EricP wrote:
> >> MitchAlsup wrote:
> >>> I recently added process priority to my ATOMIC feature and I am thinking
> >>> about adding privilege level, too::
> >>> <
> >>> If a higher priority process accesses a cache line participating in an ATOMIC
> >>> event, the lower priority process holding this line fails its ATOMIC event and
> >>> the higher priority process gets to make forward progress.
> >>> <
> >>> If a lower priority process accesses a cache line participating in an ATOMIC
> >>> event, the lower priority process fails its ATOMIC event and the higher
> >>> priority process gets to continue to make forward progress.
> >>> <
> >>> If a process receives a NaK and was attempting an ATOMIC event, then
> >>> the event fails (and you start over at the top), if it was not attempting
> >>> an event, then the access is restarted. Only a process attempting an
> >>> ATOMIC event that has already touched all of its cache lines can
> >>> respond with a NaK.
> > <
> >> Allowing priority to jump ahead in a service queue removes any
> >> bounded delay guarantees for the lower priority and allows starvation.
> >> I think this priority feature should be an option selectable at the
> >> start of each transaction.
> > <
> > What do you do when the lower priority thread does not want said
> > option, but the higher priority thread does ?
> > <
> > Also note: If you don't do something like this, you get more priority
> > inversion, where lower priority threads steel cycles from higher priority
> > threads.
<
> I think it is a good idea - yes, it is the difference between being
> priority inversion oblivious (PIO) and priority inversion aware (PIA)
> and I think both are valid.
>
> My knee jerk concern is PIA could make ESM less usable in real time contexts,
> either by making transactions have unbounded delay or by forcing users to
> raise their priority to a common level before and lower it after to
> ensure predictable behavior and negate much of ESM's advantage.
<
The alternative was to loan the priority of the process with late interference
and higher priority to the process with the lock so the one with the lock make
progress faster than his nominal priority. {The problem is when the extra
priority is not needed and one has a PIO thread, how do you get it back ??}
>
> And I don't think the scenario you describe is likely -
> that threads would share some virtual space, have common data structures,
> coordinate their locking and updates in common access code,
> but then disagree as to whether it should use PIO or PIA.
<
Hey, I'm a HW guy just trying to figure out what functionality is needed to
allow SW sort it all out.
>
> Also it is easily resolvable. This only matters on collisions:
> - If both are oblivious use fifo.
> - If both are aware choose higher priority, if equal priority use fifo.
> - if aware and oblivious, pick the one that cares and choose aware.

On Wednesday, May 31, 2023 at 6:02:24 PM UTC-5, Scott Lurndal wrote:
> MitchAlsup <Mitch...@aol.com> writes:
> >On Wednesday, May 31, 2023 at 2:57:48=E2=80=AFPM UTC-5, Scott Lurndal wrote=
> >:
>
> >> One of the acid tests at a national lab that will go unnamed is for=20
> >> all cores to pound on a single cache line (using a spin lock). Starvation=
> >=20
> >> was a 'fail'.=20
> ><
> >Are the contenders of the same priority ?? Are they of the same Privilege ?=
>
> Yes. Although I'm not aware of either priority or privilege
> being factored into Intel or AMD's coherency protocols.
<
Yes, priority and privilege are not considered in AMD or Intel (and I suspect
ARM). However, I was looking at a "bus interconnect" that is cache-line wide
(1-beat is 512-bits) and thinking that a 8-bit command, 64-bit address and
a handful of other bits, there are a lot of bits left over, so it would create
no real disturbance on the "bus" is priority (6-bits) and/or privilege (2-bits)
tag long with the request.
<
And as I explained above, there is an ATOMIC event, and SW today mainly
uses the event to guard a critical section. It is only when you get into
CAS, DCAS,... where you are trying to perform "some" of the critical
section in the ATOMIC event (or not having a critical section at all).

In article <u58001$2f7p9$1@dont-email.me>, BGB <cr88192@gmail.com> wrote:
>On 5/31/2023 7:04 AM, Dan Cross wrote:
>> We were in alt.os.development. The entire thing evolved out of
>> a discussion about x86S. Indeed, in hist first post on the
>> matter, BGB suggested his BJX2 as a replacement for x86:
>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/gIeuGffeCAAJ
>>
>> In which he says,
>>
>> |Ironically, if one goes over to software managed TLB, then the whole
>> |"nested page table" thing can disappear into the noise (as it is all
>> |software).
>>
>> Which was the genesis of this entire thread of discussion: We
>> were explicitly talking about server-class CPUs. If he wants to
>> backtrack now and talk about embedded processors, whatever, but
>> that's certainly not how the discussion started.
>>
>
>I missed that point, it seems...
>And/or forgot, I am not sure which.

The subject of the thread was "x86-S".

>But, yeah, I guess maybe one can argue it doesn't make sense for server
>class CPUs, but there are uses for emulation/virtualization in embedded
>contexts as well (I had more thought of virtualization more like
>"emulation, but with hardware accelerated memory access").

Emulation != Virtualization in this context
https://en.wikipedia.org/wiki/Popek_and_Goldberg_virtualization_requirements

- Dan C.

On 5/31/2023 7:16 PM, Dan Cross wrote:
> In article <u58001$2f7p9$1@dont-email.me>, BGB <cr88192@gmail.com> wrote:
>> On 5/31/2023 7:04 AM, Dan Cross wrote:
>>> We were in alt.os.development. The entire thing evolved out of
>>> a discussion about x86S. Indeed, in hist first post on the
>>> matter, BGB suggested his BJX2 as a replacement for x86:
>>> https://groups.google.com/g/alt.os.development/c/doFB22kNPJE/m/gIeuGffeCAAJ
>>>
>>> In which he says,
>>>
>>> |Ironically, if one goes over to software managed TLB, then the whole
>>> |"nested page table" thing can disappear into the noise (as it is all
>>> |software).
>>>
>>> Which was the genesis of this entire thread of discussion: We
>>> were explicitly talking about server-class CPUs. If he wants to
>>> backtrack now and talk about embedded processors, whatever, but
>>> that's certainly not how the discussion started.
>>>
>>
>> I missed that point, it seems...
>> And/or forgot, I am not sure which.
>
> The subject of the thread was "x86-S".
>

Where did either the linked article or thread prior to this point say it
was an exclusively server feature?...

It was talking about simplifying x86 to be x86-64 only, but nothing
really about servers or data-centers...

As I saw it, more like, x86-S kinda eliminates some of the main "selling
points" of "classic x86".

The only time I remember any of this "server stuff" came up (much later
in the thread), I said that it was outside the target domain of BJX2.

It would have been pointless to have been "covertly" talking about
servers only to be like, "yeah, no, that is not what I am doing here".

But, then again, it is not exactly like modern PCs are really all that
usable for retro-computing in the first place (with most of the PC
retro-computing community mostly trading in 20+ year old parts).

At this point, there isn't really much reason to keep using x86 apart
from inertia. You could have pretty much everything jump over to ARM or
similar and emulation, and if it were pulled off well, hardly anyone
would notice...

It has been a sinking ship for a while, like the need to resort to
emulators to run DOS or Win16 software on modern Windows (because MS
didn't add emulation for these as a built-in feature in the move to
64-bits).

Nevermind if the retro-computing guys mostly dislike emulation, and
instead want everything running on "original hardware" (well, except
if/when modern PCs become retro). (Say, "Start hoarding now, this stuff
will be gold in 25 or 30 years...", particularly the random/obscure "junk").

OTOH, just saw a video recently where someone was talking about their
difficulty getting ahold of an obscure variant of Game Boy knock-off
that was sold mostly in Brazil (AKA: "Cougar Boy"). (Kind of a flop when
it was new, rare and expensive now).

>> But, yeah, I guess maybe one can argue it doesn't make sense for server
>> class CPUs, but there are uses for emulation/virtualization in embedded
>> contexts as well (I had more thought of virtualization more like
>> "emulation, but with hardware accelerated memory access").
>
> Emulation != Virtualization in this context
> https://en.wikipedia.org/wiki/Popek_and_Goldberg_virtualization_requirements
>

Possibly, but this is not the only way I have seen the term used...

I was using it in a more narrow sense of "does not need to emulate
memory access and address translation entirely in software".

But, not necessarily in the sense of "does not require a JIT cache or
translation stages" (nor in the sense that the hardware interfaces
provided in the VM match those of the native machine).

Well, at least short of having some way to have a fully programmable
(software-defined) instruction decoder.

....