One annoyance with my project is that I can't run a core that can run
exactly the same code along the range of FPGA's I want to use:
XC7S25: Can only run a simplistic RISC-like core (with no FPU or MMU)
XC7S50: Currently running a 2-wide configuration with 32 GPRs
Reduced features, eg: no 128-bit ALU ops in this case, ...
XC7A100T: Can run 3-wide with 64 GPRs, no issue.
Can also run 128-bit ALU ops, ...

Generally, one needs to rebuild the code for each configuration.
Code built for small configurations will perform worse on bigger
configurations. Code built for wide configurations will not run on small
configurations.

There is a lot of "minor stuff" that may break when going between 32 and
64 GPRs.
Running programs built for 64 GPRs on a kernel built in 32 GPR mode will
tend to explode, ...

I also don't necessarily want to have separate builds of of all the
static libraries for each configuration, ...

Though, this is partly inescapable as I have not come up with ideal ways
to gloss over "sizeof(void *)" and "intptr" and some other issues (well,
though there is an "__intptr" type whose main property in this case is
to be the same width as "__sizeof(void *)" and similar).

This part is in an "almost works" category, but code may act buggy or
break in weird ways if linked against a library compiled for a different
width (mostly because type-consistency in a lot of the existing codebase
is a bit hit or miss, and the "gold standard" of using #ifdef for
everything doesn't entirely work in this case).

Though, my wonky compiler architecture has allowed a partial workaround
for a limited set of cases (BGBCC specific):
.ifarch feature_expr //ASM
__declspec(ifarch(feature_expr)) //C declarations
__ifarch(feature_expr) { ... } //C code

Where, conventional #ifdef only works in the preprocessor stage, which
can't deal with variability in the final configuration without a full
recompile.

For example, noting as my existing task scheduler code would explode
with a 32 GPRs or without the XMOV (96-bit VA's) extension, I ended up
needing to add some hackery:
...
taskern=(TKPE_TaskInfoKern *)task->krnlptr;
memcpy(
taskern->ctx_regsave,
__arch_isrsave,
__ARCH_SIZEOF_REGSAVE__);
__ifarch(!has_xgpr) //deal with 32 GPRs
{
taskern->ctx_regsave[TKPE_REGSAVE_GBR] =
taskern->ctx_regsave[TKPE_REGSAVE_GBR_LO];
taskern->ctx_regsave[TKPE_REGSAVE_LR] =
taskern->ctx_regsave[TKPE_REGSAVE_LR_LO];
taskern->ctx_regsave[TKPE_REGSAVE_SPC] =
taskern->ctx_regsave[TKPE_REGSAVE_SPC_LO];
taskern->ctx_regsave[TKPE_REGSAVE_EXSR] =
taskern->ctx_regsave[TKPE_REGSAVE_EXSR_LO];
}
...
__ifarch(has_xmov) //only do this if we have XMOV
{
taskern->ctx_regsave[TKPE_REGSAVE_PCH]=__arch_pch;
taskern->ctx_regsave[TKPE_REGSAVE_GBH]=__arch_gbh;
}
...

Where, it mostly deals with the 32 GPR config by copying some of the
registers around so that they are in the correct places for the other
code (with most of the kernel assuming 64 GPRs, but needing to wrangle
it around when saving and restoring task contexts).

While __ARCH_SIZEOF_REGSAVE__ looks like a normal preprocessor constant,
it doesn't actually expand to a constant, but instead an "architecture
defined variable" (which turns into a constant later on).

Similarly, the "__arch_reg" variables allow exposing the underlying CPU
control registers as-if they were C variables (though, the exact
mechanism differs slightly in this case).

Where, as noted, BGBCC doesn't handle static libraries by compiling
directly to machine code, but instead first compiles to a stack-oriented
bytecode format (RIL, along vaguely similar lines to JVM or .NET
bytecode), which is then translated to 3-address-code and then machine
code while compiling the final binary.

So, in this case, if the "ifarch" remains flexible in the RIL code, then
it can resolved once the RIL is translated to 3AC for the final
compilation (this is typically also when things like "struct layout" and
"sizeof(whatever)" is fully sorted out).

RIL is kinda messy in terms of its design, and the whole image basically
needs to be decoded in a linear fashion (it is not a conventional
structured format, rather the entire image, including all the metadata,
is expressed as a long linear stream of bytecode ops), but "basically
works"... Most efforts to work on replacements have tended to stall though.

For global declarations, it may cause the "global object" to be
suppressed, where the compiler pretends as-if it did not exist.

For code-blocks, it is more mundane:
It is compiled mostly like a normal "if" block, except that the "if()"
expression is resolved to a constant based on architectural feature
flags and (may) allow the resulting basic-block to be either included or
omitted from the binary (though, may not be sufficient to avoid the
compiler from complaining about missing dependencies if the variables or
functions used only exist along certain branches of the "ifarch" tree).

These are mostly using normal C expression syntax, with some
restrictions (one can't access normal runtime variables or call
functions or similar; only use things that will be able to evaluate to a
constant at compile time).

But, the final binaries still end up being specific to a particular
configuration, which is still kind of annoying.

And, getting everything in place so that I can use an Arty S7-50 to
drive a small robot around is being a lot more effort than ideal (mostly
because this has led to a *lot* of these types of issues, as a lot of my
code had implicitly drifted to assumptions of a "bigger" target, and I
need to scale things back some and get things working again).

Well, and in addition still need to write the robot control code and
other stuff, still with some uncertainty about the whole "camera module"
thing (maybe I should just get some IR or ultrasonic distance sensors?...).

Granted, "easy path" in this case would be "just use a RasPi", but alas.
My recent tinkering with neural nets was also partly related to this
sub-case, as I had intended to try to use NNs for processing camera
data, but this still seems like it is "pushing it".

....

Any thoughts?...

On Monday, July 3, 2023 at 12:14:37 PM UTC-5, BGB wrote:
> One annoyance with my project is that I can't run a core that can run
> exactly the same code along the range of FPGA's I want to use:
> XC7S25: Can only run a simplistic RISC-like core (with no FPU or MMU)
> XC7S50: Currently running a 2-wide configuration with 32 GPRs
> Reduced features, eg: no 128-bit ALU ops in this case, ...
> XC7A100T: Can run 3-wide with 64 GPRs, no issue.
> Can also run 128-bit ALU ops, ...
>
>
>
> Generally, one needs to rebuild the code for each configuration.
> Code built for small configurations will perform worse on bigger
> configurations. Code built for wide configurations will not run on small
> configurations.
<
This is exactly why my architecture allows for the HW to perform the narrow
to wide transformations. There is 1 ISA model for everything from 1-wide
in-order to 8-wide GBOoO. Code that runs optimally on the 1-wide machine
is within spitting distance of running optimally on the GBOoO.
<
Then the compiler can be targeted at just the ISA and be model ignorant.
>
> There is a lot of "minor stuff" that may break when going between 32 and
> 64 GPRs.
<
So, don't do that......choose one or the other
<
> Running programs built for 64 GPRs on a kernel built in 32 GPR mode will
> tend to explode, ...
<
So, don't do that......choose one or the other
>
> I also don't necessarily want to have separate builds of of all the
> static libraries for each configuration, ...
>
<
So, don't do that......choose one
>
> Though, this is partly inescapable as I have not come up with ideal ways
> to gloss over "sizeof(void *)" and "intptr" and some other issues (well,
> though there is an "__intptr" type whose main property in this case is
> to be the same width as "__sizeof(void *)" and similar).
>
> This part is in an "almost works" category, but code may act buggy or
> break in weird ways if linked against a library compiled for a different
> width (mostly because type-consistency in a lot of the existing codebase
> is a bit hit or miss, and the "gold standard" of using #ifdef for
> everything doesn't entirely work in this case).
>
>
> Though, my wonky compiler architecture has allowed a partial workaround
> for a limited set of cases (BGBCC specific):
> .ifarch feature_expr //ASM
> __declspec(ifarch(feature_expr)) //C declarations
> __ifarch(feature_expr) { ... } //C code
>
> Where, conventional #ifdef only works in the preprocessor stage, which
> can't deal with variability in the final configuration without a full
> recompile.
>
>
> For example, noting as my existing task scheduler code would explode
> with a 32 GPRs or without the XMOV (96-bit VA's) extension, I ended up
> needing to add some hackery:
> ...
> taskern=(TKPE_TaskInfoKern *)task->krnlptr;
> memcpy(
> taskern->ctx_regsave,
> __arch_isrsave,
> __ARCH_SIZEOF_REGSAVE__);
> __ifarch(!has_xgpr) //deal with 32 GPRs
> {
> taskern->ctx_regsave[TKPE_REGSAVE_GBR] =
> taskern->ctx_regsave[TKPE_REGSAVE_GBR_LO];
> taskern->ctx_regsave[TKPE_REGSAVE_LR] =
> taskern->ctx_regsave[TKPE_REGSAVE_LR_LO];
> taskern->ctx_regsave[TKPE_REGSAVE_SPC] =
> taskern->ctx_regsave[TKPE_REGSAVE_SPC_LO];
> taskern->ctx_regsave[TKPE_REGSAVE_EXSR] =
> taskern->ctx_regsave[TKPE_REGSAVE_EXSR_LO];
> }
> ...
> __ifarch(has_xmov) //only do this if we have XMOV
> {
> taskern->ctx_regsave[TKPE_REGSAVE_PCH]=__arch_pch;
> taskern->ctx_regsave[TKPE_REGSAVE_GBH]=__arch_gbh;
> }
> ...
<
It is stuff like this that caused my architecture to take a detour.
<
I don't have instructions save/restore registers or details
of the state of the thread or core. These pieces of information
are associated with a memory address, and when it comes time
to save them, HW sequencer knows where they go and puts them
there, at the same time HW sequencer knows where the next set
are coming from and can be loading the new set while pushing
out the old set.
<
When control arrives after a "switch" the code is already using
the register file of the new context--including its stack pointer.
Thus the code does not have to do anything other than the job
that needed done based on the reason of the switch to this new
set of state.
<
The register file itself operates like a write-back cache of 4-lines.
The Core state operates like a write-back cache of 1-cache line.
HW sequencing is smart enough to begin the fetch of core state
and register file BEFORE the switch is initiated, thus the core
continues processing the old state until the new state is ready to
take over.
<
All of the state inside a core can be accessed via memory mapped
control registers--including the registers. So, if a core stops talking
to the rest of the system, one can go in and look at what the core
was doing and the instant of where it stopped--remotely.
<
All other control registers {L2 cache, L3 cache, PCIe hostBridge,
devices, timers,...} are all memory mapped.
<
There is no CPUID-like instruction, all the information of each
core is accessible via configuration space on the capabilities
list of what smells like a PCIe header.
>
> Where, it mostly deals with the 32 GPR config by copying some of the
> registers around so that they are in the correct places for the other
> code (with most of the kernel assuming 64 GPRs, but needing to wrangle
> it around when saving and restoring task contexts).
>
> While __ARCH_SIZEOF_REGSAVE__ looks like a normal preprocessor constant,
> it doesn't actually expand to a constant, but instead an "architecture
> defined variable" (which turns into a constant later on).
>
> Similarly, the "__arch_reg" variables allow exposing the underlying CPU
> control registers as-if they were C variables (though, the exact
> mechanism differs slightly in this case).
>
>
>
> Where, as noted, BGBCC doesn't handle static libraries by compiling
> directly to machine code, but instead first compiles to a stack-oriented
> bytecode format (RIL, along vaguely similar lines to JVM or .NET
> bytecode), which is then translated to 3-address-code and then machine
> code while compiling the final binary.
>
> So, in this case, if the "ifarch" remains flexible in the RIL code, then
> it can resolved once the RIL is translated to 3AC for the final
> compilation (this is typically also when things like "struct layout" and
> "sizeof(whatever)" is fully sorted out).
>
>
> RIL is kinda messy in terms of its design, and the whole image basically
> needs to be decoded in a linear fashion (it is not a conventional
> structured format, rather the entire image, including all the metadata,
> is expressed as a long linear stream of bytecode ops), but "basically
> works"... Most efforts to work on replacements have tended to stall though.
>
>
> For global declarations, it may cause the "global object" to be
> suppressed, where the compiler pretends as-if it did not exist.
>
> For code-blocks, it is more mundane:
> It is compiled mostly like a normal "if" block, except that the "if()"
> expression is resolved to a constant based on architectural feature
> flags and (may) allow the resulting basic-block to be either included or
> omitted from the binary (though, may not be sufficient to avoid the
> compiler from complaining about missing dependencies if the variables or
> functions used only exist along certain branches of the "ifarch" tree).
>
> These are mostly using normal C expression syntax, with some
> restrictions (one can't access normal runtime variables or call
> functions or similar; only use things that will be able to evaluate to a
> constant at compile time).
>
>
>
> But, the final binaries still end up being specific to a particular
> configuration, which is still kind of annoying.
<
So, fix it. Nobody is standing over you preventing you from fixing it.
>
> And, getting everything in place so that I can use an Arty S7-50 to
> drive a small robot around is being a lot more effort than ideal (mostly
> because this has led to a *lot* of these types of issues, as a lot of my
> code had implicitly drifted to assumptions of a "bigger" target, and I
> need to scale things back some and get things working again).
>
> Well, and in addition still need to write the robot control code and
> other stuff, still with some uncertainty about the whole "camera module"
> thing (maybe I should just get some IR or ultrasonic distance sensors?...).
>
> Granted, "easy path" in this case would be "just use a RasPi", but alas.
> My recent tinkering with neural nets was also partly related to this
> sub-case, as I had intended to try to use NNs for processing camera
> data, but this still seems like it is "pushing it".
>
> ...
>
>
> Any thoughts?...
<
Get an FPGA that has 10M gates and 500 useable pins.
But this has cost drawbacks.........

On Monday, July 3, 2023 at 5:55:31 PM UTC-4, MitchAlsup wrote:
> On Monday, July 3, 2023 at 12:14:37 PM UTC-5, BGB wrote:
> > One annoyance with my project is that I can't run a core that can run
> > exactly the same code along the range of FPGA's I want to use:
> > XC7S25: Can only run a simplistic RISC-like core (with no FPU or MMU)
> > XC7S50: Currently running a 2-wide configuration with 32 GPRs
> > Reduced features, eg: no 128-bit ALU ops in this case, ...
> > XC7A100T: Can run 3-wide with 64 GPRs, no issue.
> > Can also run 128-bit ALU ops, ...
> >
> >
> >
> > Generally, one needs to rebuild the code for each configuration.
> > Code built for small configurations will perform worse on bigger
> > configurations. Code built for wide configurations will not run on small
> > configurations.
> <
> This is exactly why my architecture allows for the HW to perform the narrow
> to wide transformations. There is 1 ISA model for everything from 1-wide
> in-order to 8-wide GBOoO. Code that runs optimally on the 1-wide machine
> is within spitting distance of running optimally on the GBOoO.
> <
> Then the compiler can be targeted at just the ISA and be model ignorant.
> >
> > There is a lot of "minor stuff" that may break when going between 32 and
> > 64 GPRs.
> <
> So, don't do that......choose one or the other
> <
> > Running programs built for 64 GPRs on a kernel built in 32 GPR mode will
> > tend to explode, ...
> <
> So, don't do that......choose one or the other
> >
> > I also don't necessarily want to have separate builds of of all the
> > static libraries for each configuration, ...
> >
> <
> So, don't do that......choose one
> >
> > Though, this is partly inescapable as I have not come up with ideal ways
> > to gloss over "sizeof(void *)" and "intptr" and some other issues (well,
> > though there is an "__intptr" type whose main property in this case is
> > to be the same width as "__sizeof(void *)" and similar).
> >
> > This part is in an "almost works" category, but code may act buggy or
> > break in weird ways if linked against a library compiled for a different
> > width (mostly because type-consistency in a lot of the existing codebase
> > is a bit hit or miss, and the "gold standard" of using #ifdef for
> > everything doesn't entirely work in this case).
> >
> >
> > Though, my wonky compiler architecture has allowed a partial workaround
> > for a limited set of cases (BGBCC specific):
> > .ifarch feature_expr //ASM
> > __declspec(ifarch(feature_expr)) //C declarations
> > __ifarch(feature_expr) { ... } //C code
> >
> > Where, conventional #ifdef only works in the preprocessor stage, which
> > can't deal with variability in the final configuration without a full
> > recompile.
> >
> >
> > For example, noting as my existing task scheduler code would explode
> > with a 32 GPRs or without the XMOV (96-bit VA's) extension, I ended up
> > needing to add some hackery:
> > ...
> > taskern=(TKPE_TaskInfoKern *)task->krnlptr;
> > memcpy(
> > taskern->ctx_regsave,
> > __arch_isrsave,
> > __ARCH_SIZEOF_REGSAVE__);
> > __ifarch(!has_xgpr) //deal with 32 GPRs
> > {
> > taskern->ctx_regsave[TKPE_REGSAVE_GBR] =
> > taskern->ctx_regsave[TKPE_REGSAVE_GBR_LO];
> > taskern->ctx_regsave[TKPE_REGSAVE_LR] =
> > taskern->ctx_regsave[TKPE_REGSAVE_LR_LO];
> > taskern->ctx_regsave[TKPE_REGSAVE_SPC] =
> > taskern->ctx_regsave[TKPE_REGSAVE_SPC_LO];
> > taskern->ctx_regsave[TKPE_REGSAVE_EXSR] =
> > taskern->ctx_regsave[TKPE_REGSAVE_EXSR_LO];
> > }
> > ...
> > __ifarch(has_xmov) //only do this if we have XMOV
> > {
> > taskern->ctx_regsave[TKPE_REGSAVE_PCH]=__arch_pch;
> > taskern->ctx_regsave[TKPE_REGSAVE_GBH]=__arch_gbh;
> > }
> > ...
> <
> It is stuff like this that caused my architecture to take a detour.
> <
> I don't have instructions save/restore registers or details
> of the state of the thread or core. These pieces of information
> are associated with a memory address, and when it comes time
> to save them, HW sequencer knows where they go and puts them
> there, at the same time HW sequencer knows where the next set
> are coming from and can be loading the new set while pushing
> out the old set.
> <
> When control arrives after a "switch" the code is already using
> the register file of the new context--including its stack pointer.
> Thus the code does not have to do anything other than the job
> that needed done based on the reason of the switch to this new
> set of state.
> <
> The register file itself operates like a write-back cache of 4-lines.
> The Core state operates like a write-back cache of 1-cache line.
> HW sequencing is smart enough to begin the fetch of core state
> and register file BEFORE the switch is initiated, thus the core
> continues processing the old state until the new state is ready to
> take over.
> <
> All of the state inside a core can be accessed via memory mapped
> control registers--including the registers. So, if a core stops talking
> to the rest of the system, one can go in and look at what the core
> was doing and the instant of where it stopped--remotely.
> <
> All other control registers {L2 cache, L3 cache, PCIe hostBridge,
> devices, timers,...} are all memory mapped.
> <
> There is no CPUID-like instruction, all the information of each
> core is accessible via configuration space on the capabilities
> list of what smells like a PCIe header.
> >
> > Where, it mostly deals with the 32 GPR config by copying some of the
> > registers around so that they are in the correct places for the other
> > code (with most of the kernel assuming 64 GPRs, but needing to wrangle
> > it around when saving and restoring task contexts).
> >
> > While __ARCH_SIZEOF_REGSAVE__ looks like a normal preprocessor constant,
> > it doesn't actually expand to a constant, but instead an "architecture
> > defined variable" (which turns into a constant later on).
> >
> > Similarly, the "__arch_reg" variables allow exposing the underlying CPU
> > control registers as-if they were C variables (though, the exact
> > mechanism differs slightly in this case).
> >
> >
> >
> > Where, as noted, BGBCC doesn't handle static libraries by compiling
> > directly to machine code, but instead first compiles to a stack-oriented
> > bytecode format (RIL, along vaguely similar lines to JVM or .NET
> > bytecode), which is then translated to 3-address-code and then machine
> > code while compiling the final binary.
> >
> > So, in this case, if the "ifarch" remains flexible in the RIL code, then
> > it can resolved once the RIL is translated to 3AC for the final
> > compilation (this is typically also when things like "struct layout" and
> > "sizeof(whatever)" is fully sorted out).
> >
> >
> > RIL is kinda messy in terms of its design, and the whole image basically
> > needs to be decoded in a linear fashion (it is not a conventional
> > structured format, rather the entire image, including all the metadata,
> > is expressed as a long linear stream of bytecode ops), but "basically
> > works"... Most efforts to work on replacements have tended to stall though.
> >
> >
> > For global declarations, it may cause the "global object" to be
> > suppressed, where the compiler pretends as-if it did not exist.
> >
> > For code-blocks, it is more mundane:
> > It is compiled mostly like a normal "if" block, except that the "if()"
> > expression is resolved to a constant based on architectural feature
> > flags and (may) allow the resulting basic-block to be either included or
> > omitted from the binary (though, may not be sufficient to avoid the
> > compiler from complaining about missing dependencies if the variables or
> > functions used only exist along certain branches of the "ifarch" tree).
> >
> > These are mostly using normal C expression syntax, with some
> > restrictions (one can't access normal runtime variables or call
> > functions or similar; only use things that will be able to evaluate to a
> > constant at compile time).
> >
> >
> >
> > But, the final binaries still end up being specific to a particular
> > configuration, which is still kind of annoying.
> <
> So, fix it. Nobody is standing over you preventing you from fixing it.
> >
> > And, getting everything in place so that I can use an Arty S7-50 to
> > drive a small robot around is being a lot more effort than ideal (mostly
> > because this has led to a *lot* of these types of issues, as a lot of my
> > code had implicitly drifted to assumptions of a "bigger" target, and I
> > need to scale things back some and get things working again).
> >
> > Well, and in addition still need to write the robot control code and
> > other stuff, still with some uncertainty about the whole "camera module"
> > thing (maybe I should just get some IR or ultrasonic distance sensors?....).
> >
> > Granted, "easy path" in this case would be "just use a RasPi", but alas..
> > My recent tinkering with neural nets was also partly related to this
> > sub-case, as I had intended to try to use NNs for processing camera
> > data, but this still seems like it is "pushing it".
> >
> > ...
> >
> >
> > Any thoughts?...
> <
> Get an FPGA that has 10M gates and 500 useable pins.
> But this has cost drawbacks.........

On 7/3/2023 4:55 PM, MitchAlsup wrote:
> On Monday, July 3, 2023 at 12:14:37 PM UTC-5, BGB wrote:
>> One annoyance with my project is that I can't run a core that can run
>> exactly the same code along the range of FPGA's I want to use:
>> XC7S25: Can only run a simplistic RISC-like core (with no FPU or MMU)
>> XC7S50: Currently running a 2-wide configuration with 32 GPRs
>> Reduced features, eg: no 128-bit ALU ops in this case, ...
>> XC7A100T: Can run 3-wide with 64 GPRs, no issue.
>> Can also run 128-bit ALU ops, ...
>>
>>
>>
>> Generally, one needs to rebuild the code for each configuration.
>> Code built for small configurations will perform worse on bigger
>> configurations. Code built for wide configurations will not run on small
>> configurations.
> <
> This is exactly why my architecture allows for the HW to perform the narrow
> to wide transformations. There is 1 ISA model for everything from 1-wide
> in-order to 8-wide GBOoO. Code that runs optimally on the 1-wide machine
> is within spitting distance of running optimally on the GBOoO.
> <
> Then the compiler can be targeted at just the ISA and be model ignorant.

The issue is doing this effectively without causing LUT cost or timing
issues.

I have a possible way to execute 1-wide code as a 2-way superscalar, but
it would only really be practical on the larger profiles.

Basic option being to use a pattern matching table to flag ops as "valid
prefix" and "valid suffix", and whenever a match is found (and there are
no register conflicts) the pair executes as a 2-wide bundle.

However, this would still leave an incentive for "jumbo prefixes", which
are also more of an issue on 1-wide cores, since bundling can't fully
"fix" the issue of having a constant-load as a series of
serially-dependent instructions.

There is a way to make jumbo prefixes work:
Jumbo prefixes operate on an internal state variable in the decoder;
They are executed as multiple sequential instructions;
Exception handling may not be directed to the middle of a multi-part
instruction.

But, then they still have a higher LUT cost.

Though, in terms of "one ISA that does everything" (without breaking
binary compatibility between the profiles), a more minimalist RISC-like
ISA might almost make more sense.

One could almost argue for RISC-V here, but I still have some of my own
annoyances with RISC-V, and I suspect it may still not be "optimal" in
terms of cost-balancing all of its features (or its use of encoding space).

But, as-is, I have ended up defining my currently used "mainline"
configuration as an "H profile", and the profile I am currently using on
the XC7S50 as an "I profile", and they are not binary compatible with
each other, which is kinda "rather annoying"...

So, say:
H profile: 64 GPRs, WEX-3W
48-bit address space
Pass 16 arguments in registers, ...
I profile: 32 GPRs, WEX-2W
48-bit address space
Pass 8 arguments in registers.

For a while, I had presumed that WEX-2W was dead so I could ignore it,
but sort of ended up needing to revive it because 3W eats too many LUTs,
and the 1-wide profiles suck too much (they lack MOV.X or SIMD ops;
which were important for the use case).

In theory, it could be possible to compose bundles which work on WEX-2W
but break on an implementation running WEX-3W; however my emulator's
lint tool will not allow these (and my compiler will not emit them), so
at lease 2W code will run on a 3W core (but not the reverse).

But, like, presence or absence of SIMD would also break binary
compatibility, and I don't know of any "cheap" way to get SIMD like
performance gains while also minimizing LUT cost.

Though, the main "expensive" feature here being the 4-wide low-precision
SIMD unit (~ 1.5k or 2k LUT). There is a "slightly cheaper" option of
implementing SIMD by internal pipelining within the main FPU, but this
requires a 10-cycle latency for each SIMD op (vs 3L/1T with the a
dedicated low-precision unit). Similarly, 10-cycle SIMD ops are slow
enough to preclude running neural nets on this.

But, like, I am not going to buy an Arty A7-100T to use on the robot,
this is like a $280 board...

Vs, say, $150 for the Arty S7-50 (well, and I already have one of these
boards, so wouldn't need to buy it).

The CMod-A7 and CMod-S7 are insufficient for what I want to do with it.

>>
>> There is a lot of "minor stuff" that may break when going between 32 and
>> 64 GPRs.
> <
> So, don't do that......choose one or the other
> <

It is a cost tradeoff...

32 is cheaper.

64: More expensive but faster.

On the XC7S25 and XC7S50, the number of registers does seem to effect
the resource cost.

I am not sure how much is actually due to the register arrays:
LUT5 can do 3 bits per LUT
LUT6 only does 2 bits per LUT
Or to the size of the register field and associated logic in the
instruction decoder.

I sort of suspect the latter, as using the larger (7-bit) ID space for
32 GPRs pays nearly the full cost of having 64 GPRs (vs using 6-bit IDs
for 32 GPRs).

Say:
32 GPRs uses a 6-bit register ID
00..1F: GPR Array
20..2F: SPRs
30..3F: CRs
64 GPRs uses a 7-bit register ID
00..3F: GPR Array
40..5F: SPRs
60..7F: CRs

Only C0..C15 exists in the 32 GPR config, whereas the 64 GPR case also
has C16..C31. The C32..C63 space is technically aliased to the SPR space.

The SPR space has a few registers, but is mostly filled up with
internal-use "pseudo registers".

As-is, technically the 64 GPR configuration is required to enable some
ISA features (for example, XMOV couldn't be supported without XGPR
because there is no ID space for XMOV's registers).

Granted, the cost difference isn't that large either way (mostly just an
issue when resource cost is cramped).

>> Running programs built for 64 GPRs on a kernel built in 32 GPR mode will
>> tend to explode, ...
> <
> So, don't do that......choose one or the other
>>
>> I also don't necessarily want to have separate builds of of all the
>> static libraries for each configuration, ...
>>
> <
> So, don't do that......choose one

I have sort of avoided this issue via "__ifarch" and similar...
But, within standard C, this would have been unavoidable.

Still does mean I need to do multiple builds of the final binaries.

Well, and also separate builds of the Boot ROM, as the ROM is not itself
entirely immune to the inter-profile binary compatibility issues
(though, more primarily related to features being enabled or disabled
for the "sanity tests", rather than the Boot ROM actually needing all
that many optional features).

You can't sanity test features that are absent.
Not sanity testing something isn't ideal either.

Could check feature flags in CPUID to enable/disable various sanity
tests, but this adds its own complexities (and then there is a non-zero
overlap between which features the ASM code will sanity test, and which
ones my C compiler will attempt to use if they are flagged as enabled).

Would otherwise need some feature to allow ASM code to temporarily
enable ISA features (for the assembler) as relevant and then have them
revert once the ASM code is done with them.

>>
>> Though, this is partly inescapable as I have not come up with ideal ways
>> to gloss over "sizeof(void *)" and "intptr" and some other issues (well,
>> though there is an "__intptr" type whose main property in this case is
>> to be the same width as "__sizeof(void *)" and similar).
>>
>> This part is in an "almost works" category, but code may act buggy or
>> break in weird ways if linked against a library compiled for a different
>> width (mostly because type-consistency in a lot of the existing codebase
>> is a bit hit or miss, and the "gold standard" of using #ifdef for
>> everything doesn't entirely work in this case).
>>
>>
>> Though, my wonky compiler architecture has allowed a partial workaround
>> for a limited set of cases (BGBCC specific):
>> .ifarch feature_expr //ASM
>> __declspec(ifarch(feature_expr)) //C declarations
>> __ifarch(feature_expr) { ... } //C code
>>
>> Where, conventional #ifdef only works in the preprocessor stage, which
>> can't deal with variability in the final configuration without a full
>> recompile.
>>
>>
>> For example, noting as my existing task scheduler code would explode
>> with a 32 GPRs or without the XMOV (96-bit VA's) extension, I ended up
>> needing to add some hackery:
>> ...
>> taskern=(TKPE_TaskInfoKern *)task->krnlptr;
>> memcpy(
>> taskern->ctx_regsave,
>> __arch_isrsave,
>> __ARCH_SIZEOF_REGSAVE__);
>> __ifarch(!has_xgpr) //deal with 32 GPRs
>> {
>> taskern->ctx_regsave[TKPE_REGSAVE_GBR] =
>> taskern->ctx_regsave[TKPE_REGSAVE_GBR_LO];
>> taskern->ctx_regsave[TKPE_REGSAVE_LR] =
>> taskern->ctx_regsave[TKPE_REGSAVE_LR_LO];
>> taskern->ctx_regsave[TKPE_REGSAVE_SPC] =
>> taskern->ctx_regsave[TKPE_REGSAVE_SPC_LO];
>> taskern->ctx_regsave[TKPE_REGSAVE_EXSR] =
>> taskern->ctx_regsave[TKPE_REGSAVE_EXSR_LO];
>> }
>> ...
>> __ifarch(has_xmov) //only do this if we have XMOV
>> {
>> taskern->ctx_regsave[TKPE_REGSAVE_PCH]=__arch_pch;
>> taskern->ctx_regsave[TKPE_REGSAVE_GBH]=__arch_gbh;
>> }
>> ...
> <
> It is stuff like this that caused my architecture to take a detour.
> <
> I don't have instructions save/restore registers or details
> of the state of the thread or core. These pieces of information
> are associated with a memory address, and when it comes time
> to save them, HW sequencer knows where they go and puts them
> there, at the same time HW sequencer knows where the next set
> are coming from and can be loading the new set while pushing
> out the old set.
> <
> When control arrives after a "switch" the code is already using
> the register file of the new context--including its stack pointer.
> Thus the code does not have to do anything other than the job
> that needed done based on the reason of the switch to this new
> set of state.
> <
> The register file itself operates like a write-back cache of 4-lines.
> The Core state operates like a write-back cache of 1-cache line.
> HW sequencing is smart enough to begin the fetch of core state
> and register file BEFORE the switch is initiated, thus the core
> continues processing the old state until the new state is ready to
> take over.
> <
> All of the state inside a core can be accessed via memory mapped
> control registers--including the registers. So, if a core stops talking
> to the rest of the system, one can go in and look at what the core
> was doing and the instant of where it stopped--remotely.
> <

On 7/3/2023 6:30 PM, robf...@gmail.com wrote:
> On Monday, July 3, 2023 at 5:55:31 PM UTC-4, MitchAlsup wrote:
>> On Monday, July 3, 2023 at 12:14:37 PM UTC-5, BGB wrote:
>>> One annoyance with my project is that I can't run a core that can run
>>> exactly the same code along the range of FPGA's I want to use:
>>> XC7S25: Can only run a simplistic RISC-like core (with no FPU or MMU)
>>> XC7S50: Currently running a 2-wide configuration with 32 GPRs
>>> Reduced features, eg: no 128-bit ALU ops in this case, ...
>>> XC7A100T: Can run 3-wide with 64 GPRs, no issue.
>>> Can also run 128-bit ALU ops, ...
>>>
>>>
>>>
>>> Generally, one needs to rebuild the code for each configuration.
>>> Code built for small configurations will perform worse on bigger
>>> configurations. Code built for wide configurations will not run on small
>>> configurations.
>> <
>> This is exactly why my architecture allows for the HW to perform the narrow
>> to wide transformations. There is 1 ISA model for everything from 1-wide
>> in-order to 8-wide GBOoO. Code that runs optimally on the 1-wide machine
>> is within spitting distance of running optimally on the GBOoO.
>> <
>> Then the compiler can be targeted at just the ISA and be model ignorant.
>>>
>>> There is a lot of "minor stuff" that may break when going between 32 and
>>> 64 GPRs.
>> <
>> So, don't do that......choose one or the other
>> <
>>> Running programs built for 64 GPRs on a kernel built in 32 GPR mode will
>>> tend to explode, ...
>> <
>> So, don't do that......choose one or the other
>>>
>>> I also don't necessarily want to have separate builds of of all the
>>> static libraries for each configuration, ...
>>>
>> <
>> So, don't do that......choose one
>>>
>>> Though, this is partly inescapable as I have not come up with ideal ways
>>> to gloss over "sizeof(void *)" and "intptr" and some other issues (well,
>>> though there is an "__intptr" type whose main property in this case is
>>> to be the same width as "__sizeof(void *)" and similar).
>>>
>>> This part is in an "almost works" category, but code may act buggy or
>>> break in weird ways if linked against a library compiled for a different
>>> width (mostly because type-consistency in a lot of the existing codebase
>>> is a bit hit or miss, and the "gold standard" of using #ifdef for
>>> everything doesn't entirely work in this case).
>>>
>>>
>>> Though, my wonky compiler architecture has allowed a partial workaround
>>> for a limited set of cases (BGBCC specific):
>>> .ifarch feature_expr //ASM
>>> __declspec(ifarch(feature_expr)) //C declarations
>>> __ifarch(feature_expr) { ... } //C code
>>>
>>> Where, conventional #ifdef only works in the preprocessor stage, which
>>> can't deal with variability in the final configuration without a full
>>> recompile.
>>>
>>>
>>> For example, noting as my existing task scheduler code would explode
>>> with a 32 GPRs or without the XMOV (96-bit VA's) extension, I ended up
>>> needing to add some hackery:
>>> ...
>>> taskern=(TKPE_TaskInfoKern *)task->krnlptr;
>>> memcpy(
>>> taskern->ctx_regsave,
>>> __arch_isrsave,
>>> __ARCH_SIZEOF_REGSAVE__);
>>> __ifarch(!has_xgpr) //deal with 32 GPRs
>>> {
>>> taskern->ctx_regsave[TKPE_REGSAVE_GBR] =
>>> taskern->ctx_regsave[TKPE_REGSAVE_GBR_LO];
>>> taskern->ctx_regsave[TKPE_REGSAVE_LR] =
>>> taskern->ctx_regsave[TKPE_REGSAVE_LR_LO];
>>> taskern->ctx_regsave[TKPE_REGSAVE_SPC] =
>>> taskern->ctx_regsave[TKPE_REGSAVE_SPC_LO];
>>> taskern->ctx_regsave[TKPE_REGSAVE_EXSR] =
>>> taskern->ctx_regsave[TKPE_REGSAVE_EXSR_LO];
>>> }
>>> ...
>>> __ifarch(has_xmov) //only do this if we have XMOV
>>> {
>>> taskern->ctx_regsave[TKPE_REGSAVE_PCH]=__arch_pch;
>>> taskern->ctx_regsave[TKPE_REGSAVE_GBH]=__arch_gbh;
>>> }
>>> ...
>> <
>> It is stuff like this that caused my architecture to take a detour.
>> <
>> I don't have instructions save/restore registers or details
>> of the state of the thread or core. These pieces of information
>> are associated with a memory address, and when it comes time
>> to save them, HW sequencer knows where they go and puts them
>> there, at the same time HW sequencer knows where the next set
>> are coming from and can be loading the new set while pushing
>> out the old set.
>> <
>> When control arrives after a "switch" the code is already using
>> the register file of the new context--including its stack pointer.
>> Thus the code does not have to do anything other than the job
>> that needed done based on the reason of the switch to this new
>> set of state.
>> <
>> The register file itself operates like a write-back cache of 4-lines.
>> The Core state operates like a write-back cache of 1-cache line.
>> HW sequencing is smart enough to begin the fetch of core state
>> and register file BEFORE the switch is initiated, thus the core
>> continues processing the old state until the new state is ready to
>> take over.
>> <
>> All of the state inside a core can be accessed via memory mapped
>> control registers--including the registers. So, if a core stops talking
>> to the rest of the system, one can go in and look at what the core
>> was doing and the instant of where it stopped--remotely.
>> <
>> All other control registers {L2 cache, L3 cache, PCIe hostBridge,
>> devices, timers,...} are all memory mapped.
>> <
>> There is no CPUID-like instruction, all the information of each
>> core is accessible via configuration space on the capabilities
>> list of what smells like a PCIe header.
>>>
>>> Where, it mostly deals with the 32 GPR config by copying some of the
>>> registers around so that they are in the correct places for the other
>>> code (with most of the kernel assuming 64 GPRs, but needing to wrangle
>>> it around when saving and restoring task contexts).
>>>
>>> While __ARCH_SIZEOF_REGSAVE__ looks like a normal preprocessor constant,
>>> it doesn't actually expand to a constant, but instead an "architecture
>>> defined variable" (which turns into a constant later on).
>>>
>>> Similarly, the "__arch_reg" variables allow exposing the underlying CPU
>>> control registers as-if they were C variables (though, the exact
>>> mechanism differs slightly in this case).
>>>
>>>
>>>
>>> Where, as noted, BGBCC doesn't handle static libraries by compiling
>>> directly to machine code, but instead first compiles to a stack-oriented
>>> bytecode format (RIL, along vaguely similar lines to JVM or .NET
>>> bytecode), which is then translated to 3-address-code and then machine
>>> code while compiling the final binary.
>>>
>>> So, in this case, if the "ifarch" remains flexible in the RIL code, then
>>> it can resolved once the RIL is translated to 3AC for the final
>>> compilation (this is typically also when things like "struct layout" and
>>> "sizeof(whatever)" is fully sorted out).
>>>
>>>
>>> RIL is kinda messy in terms of its design, and the whole image basically
>>> needs to be decoded in a linear fashion (it is not a conventional
>>> structured format, rather the entire image, including all the metadata,
>>> is expressed as a long linear stream of bytecode ops), but "basically
>>> works"... Most efforts to work on replacements have tended to stall though.
>>>
>>>
>>> For global declarations, it may cause the "global object" to be
>>> suppressed, where the compiler pretends as-if it did not exist.
>>>
>>> For code-blocks, it is more mundane:
>>> It is compiled mostly like a normal "if" block, except that the "if()"
>>> expression is resolved to a constant based on architectural feature
>>> flags and (may) allow the resulting basic-block to be either included or
>>> omitted from the binary (though, may not be sufficient to avoid the
>>> compiler from complaining about missing dependencies if the variables or
>>> functions used only exist along certain branches of the "ifarch" tree).
>>>
>>> These are mostly using normal C expression syntax, with some
>>> restrictions (one can't access normal runtime variables or call
>>> functions or similar; only use things that will be able to evaluate to a
>>> constant at compile time).
>>>
>>>
>>>
>>> But, the final binaries still end up being specific to a particular
>>> configuration, which is still kind of annoying.
>> <
>> So, fix it. Nobody is standing over you preventing you from fixing it.
>>>
>>> And, getting everything in place so that I can use an Arty S7-50 to
>>> drive a small robot around is being a lot more effort than ideal (mostly
>>> because this has led to a *lot* of these types of issues, as a lot of my
>>> code had implicitly drifted to assumptions of a "bigger" target, and I
>>> need to scale things back some and get things working again).
>>>
>>> Well, and in addition still need to write the robot control code and
>>> other stuff, still with some uncertainty about the whole "camera module"
>>> thing (maybe I should just get some IR or ultrasonic distance sensors?...).
>>>
>>> Granted, "easy path" in this case would be "just use a RasPi", but alas.
>>> My recent tinkering with neural nets was also partly related to this
>>> sub-case, as I had intended to try to use NNs for processing camera
>>> data, but this still seems like it is "pushing it".
>>>
>>> ...
>>>
>>>
>>> Any thoughts?...
>> <
>> Get an FPGA that has 10M gates and 500 useable pins.
>> But this has cost drawbacks.........
>
> This topic may be evidence that the ‘one size fits all’ idea is very challenging
> to achieve.
>

On 7/3/2023 10:53 PM, BGB wrote:

....

>
> Yeah.
>
>
> If I could have a "simple and consistent" solution across the range of
> FPGAs I am trying to use, that would be good.
>
> One could argue for, say, using a simpler RISC-like ISA, and then using
> superscalar or OoO on the bigger cores.
>
> But, the threshold where superscalar or OoO looks like an attractive
> option seems to be higher than what I am dealing with here...
>
>
>
> Well, and then for example, the SweRV core has an interesting design,
> and is an "actual superscalar", but (despite being a 32-bit core) its
> inability to run much faster than around 25 MHz or so is concerning...
>
> Also kinda eats the FPGA...
>
>
> MicroBlaze can run at higher clock-speeds, but AFAIK it is a
> single-issue RISC (can give decent performance, but also tends to run
> the FPGA pretty hot).
>
> So, MicroBlaze is, it looks like:
>   32x 32-bit GPRs
>   Single Precision FPU (optional)
>     DAZ/FTZ semantics.
>   Software managed TLB (Optional)
>   1 wide
>   Supports both fixed displacement and register-indexed addressing.
>   Most immediate ops have 16-bit immediate values.
>     But, also only 16-bit branch displacements.
>   Seems to use "Compare-with-zero and branch" semantics.
>
>
> Contrast, BJX2 Core is:
>   32x|64x 64-bit GPRs
>   Double Precision FPU (most configs)
>     Also DAZ/FTZ
>   Software managed TLB (most configs)
>   1/2/3 wide (3-wide in bigger configs)
>
>
> But, the issue being that code built to assume a feature is enabled will
> not run on a core which lacks that feature, but the more feature-rich
> cores will not fit on smaller FPGAs.
>
> Code built for the more limited cores will not run as efficiently on
> bigger cores.
>
> Like, in the minimal configurations, software-emulated floating point is
> used, but if one is calling runtime functions to perform floating point
> math operations, this is a lot slower than using the floating point
> instructions directly, ... (but, if you want to trap and emulate the
> instructions on the smaller cores; this is even slower...).
>
>
>
> For related reasons, probably no one would likely consider building a
> full-scale computer running MicroBlaze or similar.
>
>
> Though, ironically, in some ways MicroBlaze would likely still be less
> limited than RV32I... Well, except that nearly every function call in
> MicroBlaze is going to need to compose the branch displacement as a
> multi-op sequence or similar because, reasons (you really need like 20
> bits or so for this...).
>

Hmm, looks more, and it seems MicroBlaze has a functional equivalent of
a Jumbo prefix in the form of the IMM instruction.

It also appears they are using a similar exception dispatch mechanism
(branch to a offset relative to a vector register which then encodes a
branch to the ISR entry point).

The spec describes them as using a somewhat smaller TLB though:
64-entry, 1-way associative
Seemingly with an ITLB/DTLB split in the mix as well.

However, their memory access is aligned-only, which potentially lessens
the need for set-associativity in the TLB (with an I/D split and
aligned-only access, it is possible that a 1-way TLB is sufficient).

This is different from the 256x 4-way shared TLB used by BJX2.

So, say:
Direct Displacement Addressing:
BJX2 : 9-bit, scaled by element size
RISC-V : 12-bit unscaled
MicroBlaze: 16-bit unscaled
Aligned only
Indexed Addressing:
BJX2 : Scaled by element size
RISC-V : N/A
MicroBlaze: Unscaled
Integer with Immediate:
BJX2 : 9-bit (3RI), 16-bit (2RI)
RISC-V : 12-bit (3RI)
MicroBlaze: 16-bit (3RI)
Branch Displacement:
BJX2 : 20-bit (scale=2)
RISC-V : 20-bit
MicroBlaze: 16-bit (scale=1)
With Branch Delay Slots...
Conditional Branches:
BJX2 :
BT/BF, 20-bit disp;
Compare(Zero)+Branch, 8-bit (Optional)
RISC-V : Compare+Branch, 12-bit
MicroBlaze: CompareZero+Branch, 16-bit
Jumbo-like Prefix:
BJX2 : Yes, adds 24 bits
RISC-V : No
MicroBlaze: Yes, adds 16 bits.
FPU (Scalar):
BJX2 : GPRs, Binary64
RISC-V : FPRs, Binary32 and Binary64
MicroBlaze: GPRs, Binary32
Integer Divide:
BJX2 : Yes, MULQ Extension
RISC-V : Yes, M extension
MicroBlaze: Yes, Optional

Exclusive to BJX2 (for this set):
Predicated instructions (via SR.T flag)
Instruction Bundles
128-bit ALU ops (ALUX)
SIMD
...

Exclusive to RISC-V:
The use of dedicated floating-point registers.

Exclusive to MicroBlaze:
Aligned-only memory access;
Branch Delay Slots;
...

....

BGB <cr88192@gmail.com> schrieb:
> On 7/3/2023 4:55 PM, MitchAlsup wrote:
>> On Monday, July 3, 2023 at 12:14:37 PM UTC-5, BGB wrote:
>>> One annoyance with my project is that I can't run a core that can run
>>> exactly the same code along the range of FPGA's I want to use:
>>> XC7S25: Can only run a simplistic RISC-like core (with no FPU or MMU)
>>> XC7S50: Currently running a 2-wide configuration with 32 GPRs
>>> Reduced features, eg: no 128-bit ALU ops in this case, ...
>>> XC7A100T: Can run 3-wide with 64 GPRs, no issue.
>>> Can also run 128-bit ALU ops, ...
>>>
>>>
>>>
>>> Generally, one needs to rebuild the code for each configuration.
>>> Code built for small configurations will perform worse on bigger
>>> configurations. Code built for wide configurations will not run on small
>>> configurations.
>> <
>> This is exactly why my architecture allows for the HW to perform the narrow
>> to wide transformations. There is 1 ISA model for everything from 1-wide
>> in-order to 8-wide GBOoO. Code that runs optimally on the 1-wide machine
>> is within spitting distance of running optimally on the GBOoO.
>> <
>> Then the compiler can be targeted at just the ISA and be model ignorant.
>
> The issue is doing this effectively without causing LUT cost or timing
> issues.
>
>
> I have a possible way to execute 1-wide code as a 2-way superscalar, but
> it would only really be practical on the larger profiles.

If you want bundles, you could just execute the individual
instructions sequentially on your smallest core. Unless you throw
out the NOPs during decoding and give the CPU something else to
do, you would then suffer the performance penalty for the NOPs
you generate.

Or, if you have a bit to spare, you can put in a hint that an
instruction can be executed consecutively with the next (or the
preceding) one. This would save you the logic needed to analyze
register dependencies. Low-end implementations could just ignore
the bit, middle-end implementations would use it and high-end
implementations would not need it, wasting a bit.

Or, you could structure your ISA so that the dependency analysis
becomes simple - always put the destination register and source
register(s) in the same place, and have a opcode simple bit pattern
to show which instructions have which registers. Then, having
a two-wide in-order implementation would become simpler.

Think of what a Boolean function "Can the instruction sequence a,b
be executed consecutively" would look like. The simpler this is,
the better for your analysis and your LUT count.

A highly regular structure, like the RISC designs had, would certainly
make this easier.

On 7/4/2023 9:10 AM, Thomas Koenig wrote:
> BGB <cr88192@gmail.com> schrieb:
>> On 7/3/2023 4:55 PM, MitchAlsup wrote:
>>> On Monday, July 3, 2023 at 12:14:37 PM UTC-5, BGB wrote:
>>>> One annoyance with my project is that I can't run a core that can run
>>>> exactly the same code along the range of FPGA's I want to use:
>>>> XC7S25: Can only run a simplistic RISC-like core (with no FPU or MMU)
>>>> XC7S50: Currently running a 2-wide configuration with 32 GPRs
>>>> Reduced features, eg: no 128-bit ALU ops in this case, ...
>>>> XC7A100T: Can run 3-wide with 64 GPRs, no issue.
>>>> Can also run 128-bit ALU ops, ...
>>>>
>>>>
>>>>
>>>> Generally, one needs to rebuild the code for each configuration.
>>>> Code built for small configurations will perform worse on bigger
>>>> configurations. Code built for wide configurations will not run on small
>>>> configurations.
>>> <
>>> This is exactly why my architecture allows for the HW to perform the narrow
>>> to wide transformations. There is 1 ISA model for everything from 1-wide
>>> in-order to 8-wide GBOoO. Code that runs optimally on the 1-wide machine
>>> is within spitting distance of running optimally on the GBOoO.
>>> <
>>> Then the compiler can be targeted at just the ISA and be model ignorant.
>>
>> The issue is doing this effectively without causing LUT cost or timing
>> issues.
>>
>>
>> I have a possible way to execute 1-wide code as a 2-way superscalar, but
>> it would only really be practical on the larger profiles.
>
> If you want bundles, you could just execute the individual
> instructions sequentially on your smallest core. Unless you throw
> out the NOPs during decoding and give the CPU something else to
> do, you would then suffer the performance penalty for the NOPs
> you generate.
>

This part is not the main issue...

Also, luckily, my encoding scheme does not waste space with a bunch of
NOPs. Rather the bundles are a variable number of 32-bit instruction
words with a wide-execute flag.

The pipeline also has interlock handling, so there is no real need for
"timing NOPs" either.

A much bigger issue is, for example, that the 2-wide and 3-wide profiles
assume the existence of a MOV.X instruction (Load/Store 128-bit pair),
which does not exist on 1-wide as it requires more register ports than
are available.

Would otherwise need some mechanism to decompose it in the pipeline into
a pair of MOV.Q instructions.

Though, MOV.X isn't an issue for WEX-3W code on WEX-2W.

Rather, it is that code built for WEX-3W tends to assume other
extensions (such as MULQ or ALUX). So, stuff breaks as soon as it hits
an attempt to use an Integer Divide instruction or 128-bit ALU op or
similar.

Trying to running 3W on 2W will cause the core to fall back to scalar
operation, as 3W bundling rules aren't valid for 2W.

One could assume that the compiler not use these.

Jumbo operations were originally a problem for 1-wide, but I have ended
up using a hack apparently similar to what MicroBlaze uses, namely the
Jumbo-prefix itself behaves like a NOP as far as the EX stages are
concerned, but captures its state in a special internal register, which
then combines with the following instruction. One has to basically flag
the prefixes as "No, Interrupt can't land here.", similar to those
generated by interlock stalls, so it is a "special" type of NOP.

For the 2-wide, the Jumbo prefixes are decoded the same as for 3-wide.

Just as-is, the 3rd decoder is limited to F8 block instructions, eg:
FEab-cdef-FE01-2345-F811-6789 MOV 0xABCDEF0123456789, R17
So, it can handle 96-bit 2RI MOV and ADD and similar, but not much else.

Limiting the 3rd decoder to the F8 block serving to shave off some LUTs
in this case.

For the most part, Imm57s ALU ops or Disp57s Load/Store, while
theoretically possible to encode, are "not a thing" at present.

Mostly because the number of times these cases tend to come up (where an
Imm33s or Disp33s is insufficient) is "vanishingly small", so it ends up
debatable of whether they are worth the cost of having them enabled in
the decoder (with the closest thing to a "killer app" being to use them
as immediate values for SIMD instructions).

Also, currently Disp33s is the largest format supported natively by the
AGU (going any bigger requiring the use of ALU instructions).

> Or, if you have a bit to spare, you can put in a hint that an
> instruction can be executed consecutively with the next (or the
> preceding) one. This would save you the logic needed to analyze
> register dependencies. Low-end implementations could just ignore
> the bit, middle-end implementations would use it and high-end
> implementations would not need it, wasting a bit.
>

This is pretty much how the encoding scheme works already, and was
designed to operate.

Achieving this scalability in practice isn't as easy.

One would need to also ensure that all of the cores have the same basic
feature set in terms of scalar instructions as well, which is where the
main problem lies with the profiles.

> Or, you could structure your ISA so that the dependency analysis
> becomes simple - always put the destination register and source
> register(s) in the same place, and have a opcode simple bit pattern
> to show which instructions have which registers. Then, having
> a two-wide in-order implementation would become simpler.
>

Luckily, register fields don't move around that much.

Going superscalar or OoO should be very well possible with BJX2.

Just, the bigger question is if one can do a "good enough" superscalar
or OoO core on a Spartan or Artix FPGA to make it worthwhile.

The fastest (in terms of MHz) options thus far tend to be 1-wide RISC
cores (which one can run at ~ 100 MHz).

Or, I can run a VLIW core at 50 MHz, with enough "extra" to mostly
compensate for the lower clock-speed.

Also, at 50 and 100 MHz, performance is more limited by memory bandwidth
in most of my test cases than by clock-speed.

Also, despite the higher clock-speed, relative performance will tank if
one achieves the higher clock-speed by reducing the L1 cache size (so,
maintaining 16K or 32K of L1 cache seems to be "fairly important" for
performance).

An intermediate option is 75 MHz, but it is "very painful" trying to get
or keep the core passing timing at 75MHz (with both 3-wide VLIW and
"decent" L1 caches).

If one drops to 25 MHz, the loss of clock-speed ends up hurting more
than anything one could likely gain from doing so.

Say, for example, even with minimal memory and instruction latency, Doom
(and similar) at 25 MHz would still run slower than 50MHz, shifting from
being memory-bound to being limited by how quickly it can execute
instructions (and it would also rarely go much over around 12 fps or so).

One can also do a 16-bit core and run it at around 200 MHz, but... this
also isn't terribly useful...

> Think of what a Boolean function "Can the instruction sequence a,b
> be executed consecutively" would look like. The simpler this is,
> the better for your analysis and your LUT count.
>
> A highly regular structure, like the RISC designs had, would certainly
> make this easier.

Most of my instructions follow a few forms (for 32-bit ops):
FZnm-ZeoZ //3R
FZnm-ZeZZ //2R
FZnm-Zeii //3RI (Imm9)
FZnZ-Zeii //2RI (Imm10)
FZZZ-ZeoZ //1R ("OP Ro", but Ro understood as Rn)
FZZn-iiii //2RI (Imm16)
FZii-Ziii //IMM (BRA and BSR)

Where, Rn, Rm, Ro are the 3 registers; the 'e' field adding bit 4 for
each register: Znmo.

This leaves the Imm16 block as an outlier with a different register
encoding from all the other ops. But, there was no good way around this
short of dog-chewing the immediate field, and in this case I prioritized
having a contiguous immediate over a consistent register field.

Unlike RISC-V, I tried to minimize dog-chewing the immediate fields.

I also prioritized a layout where I could express encodings sensibly in
a hexadecimal-based notation (rather than binary), and where there was
sufficient detail in the notation to be able to unambiguously encode or
decode the instructions (without needing a big blob of text or diagrams
to try to express how a given instruction in question laid out its fields).

There are also 16-bit ops, mostly following a pattern:
ZZnm //2R
ZZni //1RI (Imm4)
ZZnZ //1R
ZZii //Imm8 / Disp8
Znii //1RI (Imm8) (MOV and ADD)
Ziii //Imm12 (MOV Imm12, R0)

The 16-bit ops being mostly limited to R0..R15 (some Load/Store ops also
have variants to encode R16..R31). There are no 3R encodings in 16-bit land.

On the first 16 bit word, the ranges are:
F0: Mostly 3R, 2R, and 1R ops.
F1: Load/Store Disp9 ops
F2: ALU Imm9 ops.
F3: Reserved / Implementation-Extension
F4..F7: Repeat F0..F3, flagged as bundled.
F8: 2RI Imm16 ops
F9: Reserved
FA: MOV Imm24u, R0
FB: MOV Imm24n, R0
FC/FD: Repeat F8/F9, bundled
FE: Jumbo Prefix (Immed Extension)
FF: Jumbo Prefix (Opcode Extension)

On 7/3/2023 2:55 PM, MitchAlsup wrote:

snip

> All of the state inside a core can be accessed via memory mapped
> control registers--including the registers. So, if a core stops talking
> to the rest of the system, one can go in and look at what the core
> was doing and the instant of where it stopped--remotely.
> <
> All other control registers {L2 cache, L3 cache, PCIe hostBridge,
> devices, timers,...} are all memory mapped.
> <
> There is no CPUID-like instruction, all the information of each
> core is accessible via configuration space on the capabilities
> list of what smells like a PCIe header.

How does a user program, which presumably has access to 64 bits of user
address space, and can't modify its page table entries, get access to
perform loads from the configuration space?

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 7/5/2023 10:32 AM, Stephen Fuld wrote:
> On 7/3/2023 2:55 PM, MitchAlsup wrote:
>
> snip
>
>> All of the state inside a core can be accessed via memory mapped
>> control registers--including the registers. So, if a core stops talking
>> to the rest of the system, one can go in and look at what the core
>> was doing and the instant of where it stopped--remotely.
>> <
>> All other control registers {L2 cache, L3 cache, PCIe hostBridge,
>> devices, timers,...} are all memory mapped.
>> <
>> There is no CPUID-like instruction, all the information of each
>> core is accessible via configuration space on the capabilities
>> list of what smells like a PCIe header.
>
> How does a user program, which presumably has access to 64 bits of user
> address space, and can't modify its page table entries, get access to
> perform loads from the configuration space?
>

Hmm... Yeah, that does seem like a mystery.

CPUID is simpler, and doesn't need to be all that fancy.

Making it essentially something like a special case MOV that fetches
magic constants and similar works OK.

Well, and a few more potentially controversial features, like a hardware
RNG and (possible) clock-cycle counter (though, one could argue that a
clock-cycle counter is a potential security risk).

There is also a microsecond clock, but I had put this in MMIO space.

But, using CPUID for this could potentially avoid an ~ 4k cycle overhead
from calling functions like "clock()" or "clock_gettime()" and similar
in userland (userland being unable to directly access the MMIO based
timers).

Also, timing via delay loops sucks, probably better to be able to
quickly fetch an accurate time and drive logic as appropriate.

Though, whether all this is "in the spirit" of a CPUID instruction is
debatable.

Otherwise, recently noted that my XC7A200T can survive me increasing the
size of the L2 cache to 1MB, effectively throwing nearly all the
Block-RAM in the FPGA at the L2 cache...

But, on the upside, it seems that with a 1MB L2 cache, Doom is seemingly
no longer memory-IO bound, and can (finally) give a consistent 20-34 fps...

(In the emulator model, with 1MB of L2, it is getting over a 90% L2 hit
rate).

Things like Heretic and Hexen also get a little faster.
Not nearly so much effect on Quake and similar though.

On Wednesday, July 5, 2023 at 10:32:41 AM UTC-5, Stephen Fuld wrote:
> On 7/3/2023 2:55 PM, MitchAlsup wrote:
>
> snip
> > All of the state inside a core can be accessed via memory mapped
> > control registers--including the registers. So, if a core stops talking
> > to the rest of the system, one can go in and look at what the core
> > was doing and the instant of where it stopped--remotely.
> > <
> > All other control registers {L2 cache, L3 cache, PCIe hostBridge,
> > devices, timers,...} are all memory mapped.
> > <
> > There is no CPUID-like instruction, all the information of each
> > core is accessible via configuration space on the capabilities
> > list of what smells like a PCIe header.
> How does a user program, which presumably has access to 64 bits of user
> address space, and can't modify its page table entries, get access to
> perform loads from the configuration space?
<
Why would a host not grant read access to the configuration stuff ??
>
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
>On 7/3/2023 2:55 PM, MitchAlsup wrote:
>
>snip
>
>> All of the state inside a core can be accessed via memory mapped
>> control registers--including the registers. So, if a core stops talking
>> to the rest of the system, one can go in and look at what the core
>> was doing and the instant of where it stopped--remotely.
>> <
>> All other control registers {L2 cache, L3 cache, PCIe hostBridge,
>> devices, timers,...} are all memory mapped.
>> <
>> There is no CPUID-like instruction, all the information of each
>> core is accessible via configuration space on the capabilities
>> list of what smells like a PCIe header.
>
>How does a user program, which presumably has access to 64 bits of user
>address space, and can't modify its page table entries, get access to
>perform loads from the configuration space?

User mode code generally doesn't require access to system control
registers. ARMv8 has privileged controls that can open certain
registers up to non-privileged accesses (e.g. the timer counter register).

The registers can be grouped and aligned on page boundaries which would
allow a kernel to grant access to certain functional subsets of the
privileged control registers to user mode code via a va to pa mapping.

On 7/5/2023 10:13 AM, MitchAlsup wrote:
> On Wednesday, July 5, 2023 at 10:32:41 AM UTC-5, Stephen Fuld wrote:
>> On 7/3/2023 2:55 PM, MitchAlsup wrote:
>>
>> snip
>>> All of the state inside a core can be accessed via memory mapped
>>> control registers--including the registers. So, if a core stops talking
>>> to the rest of the system, one can go in and look at what the core
>>> was doing and the instant of where it stopped--remotely.
>>> <
>>> All other control registers {L2 cache, L3 cache, PCIe hostBridge,
>>> devices, timers,...} are all memory mapped.
>>> <
>>> There is no CPUID-like instruction, all the information of each
>>> core is accessible via configuration space on the capabilities
>>> list of what smells like a PCIe header.
>> How does a user program, which presumably has access to 64 bits of user
>> address space, and can't modify its page table entries, get access to
>> perform loads from the configuration space?
> <
> Why would a host not grant read access to the configuration stuff ??

I presume it would, but if a particular address maps to a page in the
configuration space, then the user loses access to the same addressed
page in the user space. I don't think losing access to a few pages is a
big deal, but it does mean the user doesn't really have a full 64 bit
user address space. Again, not a big deal, but it should be documented.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 7/5/2023 10:57 AM, Scott Lurndal wrote:
> Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
>> On 7/3/2023 2:55 PM, MitchAlsup wrote:
>>
>> snip
>>
>>> All of the state inside a core can be accessed via memory mapped
>>> control registers--including the registers. So, if a core stops talking
>>> to the rest of the system, one can go in and look at what the core
>>> was doing and the instant of where it stopped--remotely.
>>> <
>>> All other control registers {L2 cache, L3 cache, PCIe hostBridge,
>>> devices, timers,...} are all memory mapped.
>>> <
>>> There is no CPUID-like instruction, all the information of each
>>> core is accessible via configuration space on the capabilities
>>> list of what smells like a PCIe header.
>>
>> How does a user program, which presumably has access to 64 bits of user
>> address space, and can't modify its page table entries, get access to
>> perform loads from the configuration space?
>
> User mode code generally doesn't require access to system control
> registers. ARMv8 has privileged controls that can open certain
> registers up to non-privileged accesses (e.g. the timer counter register).
>
> The registers can be grouped and aligned on page boundaries which would
> allow a kernel to grant access to certain functional subsets of the
> privileged control registers to user mode code via a va to pa mapping.

Yes, I get that (see my response to Mitch above). My comment is that
this should be documented so the user can see how to read, for example,
the CPUID register or equivalent)

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Wednesday, July 5, 2023 at 12:58:00 PM UTC-5, Stephen Fuld wrote:
> On 7/5/2023 10:13 AM, MitchAlsup wrote:
> > On Wednesday, July 5, 2023 at 10:32:41 AM UTC-5, Stephen Fuld wrote:
> >> On 7/3/2023 2:55 PM, MitchAlsup wrote:
> >>
> >> snip
> >>> All of the state inside a core can be accessed via memory mapped
> >>> control registers--including the registers. So, if a core stops talking
> >>> to the rest of the system, one can go in and look at what the core
> >>> was doing and the instant of where it stopped--remotely.
> >>> <
> >>> All other control registers {L2 cache, L3 cache, PCIe hostBridge,
> >>> devices, timers,...} are all memory mapped.
> >>> <
> >>> There is no CPUID-like instruction, all the information of each
> >>> core is accessible via configuration space on the capabilities
> >>> list of what smells like a PCIe header.
> >> How does a user program, which presumably has access to 64 bits of user
> >> address space, and can't modify its page table entries, get access to
> >> perform loads from the configuration space?
> > <
> > Why would a host not grant read access to the configuration stuff ??
<
> I presume it would, but if a particular address maps to a page in the
> configuration space, then the user loses access to the same addressed
> page in the user space.
<
My 66000 has 64-bit DRAM space, 64-bit MMI/O space, 64-bit configuration
space, and 64-bit ROM space, accessible from a 63-bit virtual address
space (user; super has 64-bit VAS).
<
Allocating a page where user can access core configuration RO data
takes away from user VA space, but from the 66-bit universal address
space, not enough to worry about.
<
< I don't think losing access to a few pages is a
> big deal, but it does mean the user doesn't really have a full 64 bit
> user address space. Again, not a big deal, but it should be documented.
<
User has a 63-bit VAS, supers have 64-bit VASs.
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
>On 7/5/2023 10:57 AM, Scott Lurndal wrote:
>> Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
>>> On 7/3/2023 2:55 PM, MitchAlsup wrote:
>>>
>>> snip
>>>
>>>> All of the state inside a core can be accessed via memory mapped
>>>> control registers--including the registers. So, if a core stops talking
>>>> to the rest of the system, one can go in and look at what the core
>>>> was doing and the instant of where it stopped--remotely.
>>>> <
>>>> All other control registers {L2 cache, L3 cache, PCIe hostBridge,
>>>> devices, timers,...} are all memory mapped.
>>>> <
>>>> There is no CPUID-like instruction, all the information of each
>>>> core is accessible via configuration space on the capabilities
>>>> list of what smells like a PCIe header.
>>>
>>> How does a user program, which presumably has access to 64 bits of user
>>> address space, and can't modify its page table entries, get access to
>>> perform loads from the configuration space?
>>
>> User mode code generally doesn't require access to system control
>> registers. ARMv8 has privileged controls that can open certain
>> registers up to non-privileged accesses (e.g. the timer counter register).
>>
>> The registers can be grouped and aligned on page boundaries which would
>> allow a kernel to grant access to certain functional subsets of the
>> privileged control registers to user mode code via a va to pa mapping.
>
>Yes, I get that (see my response to Mitch above). My comment is that
>this should be documented so the user can see how to read, for example,
>the CPUID register or equivalent)

Oddly enough, that's one of the registers that ARMv8 user mode code
_cannot_ read, and there's no control for privileged code to open it up
to user-mode code (although there are controls to trap OS accesses to
the hypervisor similar to VT-X and SVM).

Linux on ARMv8 (arm64) provides a list of CPU capabilities to
applications through the AUX vector on exec(2) - AT_HWCAP - that can be used as
run-time conditionals for specific sets of instructions such
as the scalable vector extension or scalable matrix extensions.

https://docs.kernel.org/arm64/elf_hwcaps.html

Since x86/amd allow user applications to directly invoke
the CPUID instruction, no AT_HWCAP is zero in those systems.

On 7/3/23 5:55 PM, MitchAlsup wrote:
> On Monday, July 3, 2023 at 12:14:37 PM UTC-5, BGB wrote:
[snip]
>> Generally, one needs to rebuild the code for each configuration.
>> Code built for small configurations will perform worse on bigger
>> configurations. Code built for wide configurations will not run on small
>> configurations.
> <
> This is exactly why my architecture allows for the HW to perform the narrow
> to wide transformations. There is 1 ISA model for everything from 1-wide
> in-order to 8-wide GBOoO. Code that runs optimally on the 1-wide machine
> is within spitting distance of running optimally on the GBOoO.

I would *guess* that if one cannot spit very far (<1% — at which
point _measuring_ the difference actually caused by using the same
binary might be challenging) this implementation constraint might
hinder exploiting some good cases.

Excluding the creativity effect of constraint, such a constraint
seems to require a non-negative (likely positive) cost.

The creativity effect (which is one advantage of formal poetry —
another is audience expectation) realistically applies as design
effort is not infinite.

If I adequately understand the general implementation proposals,
loop fission/fusion, loop interchange, and loop blocking choices
could differ based on the targeted microarchitecture (even just L1
cache size?). If a small loop body runs much faster on a small
implementation, loop fission might be better even with extra store
and load overhead to communicate with the second loop — presumably
with blocking to keep the communication within L1 — while a larger
implementation might run more than 1% faster with the loop not
split.

Doing more work to achieve higher performance seems a known
microarchitecture-aware optimization. In addition to loop
optimizations, higher-level algorithmic options might change which
binary code expression is "nearly best".

While the dynamic scheduling of a GBOoO implementation will tend
to mute differences, I suspect cases can be contrived with more
than a 5% performance difference.

[snip]> All of the state inside a core can be accessed via memory
mapped
> control registers--including the registers. So, if a core stops talking
> to the rest of the system, one can go in and look at what the core
> was doing and the instant of where it stopped--remotely.

What is the consistency model for memory-mapped registers both
within a thread and between cores?

I would guess that within a thread one would want strong
consistency which seems to require alias detection for every load
and store. For between threads, strong consistency also seems
desirable as the cost would typically not be significant (rare
use?) and the software benefit significant (already complex
reasoning required so any simplification might be appreciated).

This also seems to reduce the cost of polling for a receiver
thread (a register test being cheaper than a cache read and the
write is probably more "pushed" than "pulled").

(Address-based store "pushing" has been proposed. With last level
cache home nodes — for data placement and not just coherence
management — lower level evictions get pushed home, but this is
slow. Obviously replacement policy and early writeback could
adjust the timing.)

This might also present an opportunity similar to MONITOR/MWAIT
while avoiding the issues of using general memory. (MONITOR/MWAIT
can be one/any-to-many, which has some uses.)

Paul A. Clayton <paaronclayton@gmail.com> schrieb:

> If I adequately understand the general implementation proposals,
> loop fission/fusion, loop interchange, and loop blocking choices
> could differ based on the targeted microarchitecture (even just L1
> cache size?).

Very much so.

Very highly optimized code such as OpenBLAS sets the blocking
factors according to a table for the CPU it is running on,
to fit into the cache.

It would make sense to be able to query the cache sizes for
this kind of code.

"Paul A. Clayton" <paaronclayton@gmail.com> writes:
>On 7/3/23 5:55 PM, MitchAlsup wrote:

>
>[snip]> All of the state inside a core can be accessed via memory
>mapped
>> control registers--including the registers. So, if a core stops talking
>> to the rest of the system, one can go in and look at what the core
>> was doing and the instant of where it stopped--remotely.

The AArch64 Octeon chips have had this feature (architecturally defined)
since day one. It can be accessed by other cores, management processors
or via JTAG.

>
>What is the consistency model for memory-mapped registers both
>within a thread and between cores?

The memory mapped registers are generally used for debugging exceptional
conditions (like a hung core), and are not expected to be used in parallel
with other mechanisms (e.g. MSR/MRS in Aarch64).

On Monday, July 24, 2023 at 8:24:53 AM UTC-5, Paul A. Clayton wrote:
> On 7/3/23 5:55 PM, MitchAlsup wrote:
> > On Monday, July 3, 2023 at 12:14:37 PM UTC-5, BGB wrote:
> [snip]
> >> Generally, one needs to rebuild the code for each configuration.
> >> Code built for small configurations will perform worse on bigger
> >> configurations. Code built for wide configurations will not run on small
> >> configurations.
> > <
> > This is exactly why my architecture allows for the HW to perform the narrow
> > to wide transformations. There is 1 ISA model for everything from 1-wide
> > in-order to 8-wide GBOoO. Code that runs optimally on the 1-wide machine
> > is within spitting distance of running optimally on the GBOoO.
> I would *guess* that if one cannot spit very far (<1% — at which
> point _measuring_ the difference actually caused by using the same
> binary might be challenging) this implementation constraint might
> hinder exploiting some good cases.
>
> Excluding the creativity effect of constraint, such a constraint
> seems to require a non-negative (likely positive) cost.
<
Very little more than the GBOoO already requires.
>
> The creativity effect (which is one advantage of formal poetry —
> another is audience expectation) realistically applies as design
> effort is not infinite.
>
> If I adequately understand the general implementation proposals,
> loop fission/fusion, loop interchange, and loop blocking choices
> could differ based on the targeted microarchitecture (even just L1
> cache size?).
<
You do remember I have vectorization where each implementation
is allowed to decide how many lanes are present and perform
accordingly.
<
> If a small loop body runs much faster on a small
> implementation, loop fission might be better even with extra store
> and load overhead to communicate with the second loop — presumably
> with blocking to keep the communication within L1 — while a larger
> implementation might run more than 1% faster with the loop not
> split.
>
> Doing more work to achieve higher performance seems a known
> microarchitecture-aware optimization. In addition to loop
> optimizations, higher-level algorithmic options might change which
> binary code expression is "nearly best".
<
Algorithmic optimization is almost always better than compiler
optimization.
>
> While the dynamic scheduling of a GBOoO implementation will tend
> to mute differences, I suspect cases can be contrived with more
> than a 5% performance difference.
>
> [snip]> All of the state inside a core can be accessed via memory
> mapped
> > control registers--including the registers. So, if a core stops talking
> > to the rest of the system, one can go in and look at what the core
> > was doing and the instant of where it stopped--remotely.
<
> What is the consistency model for memory-mapped registers both
> within a thread and between cores?
<
MMI/O space is sequentially consistent.
Configuration space is strongly ordered.
>
> I would guess that within a thread one would want strong
> consistency which seems to require alias detection for every load
> and store. For between threads, strong consistency also seems
> desirable as the cost would typically not be significant (rare
> use?) and the software benefit significant (already complex
> reasoning required so any simplification might be appreciated).
<
The core has to be able to deal with an access at any clock edge,
but the system expects relatively few of these accesses. When
the system is running "normally" only core[k] accesses its
control registers[k]. There are interrupt<ing> tables used to do
IPIs and alert for scheduled tasks--these are in unCacheable
DRAM (which is cacheable in L3).
>
> This also seems to reduce the cost of polling for a receiver
> thread (a register test being cheaper than a cache read and the
> write is probably more "pushed" than "pulled").
>
> (Address-based store "pushing" has been proposed. With last level
> cache home nodes — for data placement and not just coherence
> management — lower level evictions get pushed home, but this is
> slow. Obviously replacement policy and early writeback could
> adjust the timing.)
>
> This might also present an opportunity similar to MONITOR/MWAIT
> while avoiding the issues of using general memory. (MONITOR/MWAIT
> can be one/any-to-many, which has some uses.)

poniedziałek, 3 lipca 2023 o 19:14:37 UTC+2 BGB napisał(a):
> One annoyance with my project is that I can't run a core that can run
> exactly the same code along the range of FPGA's I want to use:
> XC7S25: Can only run a simplistic RISC-like core (with no FPU or MMU)
> XC7S50: Currently running a 2-wide configuration with 32 GPRs
> Reduced features, eg: no 128-bit ALU ops in this case, ...
> XC7A100T: Can run 3-wide with 64 GPRs, no issue.
> Can also run 128-bit ALU ops, ...
>
>
>
> Generally, one needs to rebuild the code for each configuration.
> Code built for small configurations will perform worse on bigger
> configurations. Code built for wide configurations will not run on small
> configurations.
>
> There is a lot of "minor stuff" that may break when going between 32 and
> 64 GPRs.
> Running programs built for 64 GPRs on a kernel built in 32 GPR mode will
> tend to explode, ...
>
> I also don't necessarily want to have separate builds of of all the
> static libraries for each configuration, ...
>
>
> Though, this is partly inescapable as I have not come up with ideal ways
> to gloss over "sizeof(void *)" and "intptr" and some other issues (well,
> though there is an "__intptr" type whose main property in this case is
> to be the same width as "__sizeof(void *)" and similar).
>
> This part is in an "almost works" category, but code may act buggy or
> break in weird ways if linked against a library compiled for a different
> width (mostly because type-consistency in a lot of the existing codebase
> is a bit hit or miss, and the "gold standard" of using #ifdef for
> everything doesn't entirely work in this case).
>
>
> Though, my wonky compiler architecture has allowed a partial workaround
> for a limited set of cases (BGBCC specific):
> .ifarch feature_expr //ASM
> __declspec(ifarch(feature_expr)) //C declarations
> __ifarch(feature_expr) { ... } //C code
>
> Where, conventional #ifdef only works in the preprocessor stage, which
> can't deal with variability in the final configuration without a full
> recompile.
>
>
> For example, noting as my existing task scheduler code would explode
> with a 32 GPRs or without the XMOV (96-bit VA's) extension, I ended up
> needing to add some hackery:
> ...
> taskern=(TKPE_TaskInfoKern *)task->krnlptr;
> memcpy(
> taskern->ctx_regsave,
> __arch_isrsave,
> __ARCH_SIZEOF_REGSAVE__);
> __ifarch(!has_xgpr) //deal with 32 GPRs
> {
> taskern->ctx_regsave[TKPE_REGSAVE_GBR] =
> taskern->ctx_regsave[TKPE_REGSAVE_GBR_LO];
> taskern->ctx_regsave[TKPE_REGSAVE_LR] =
> taskern->ctx_regsave[TKPE_REGSAVE_LR_LO];
> taskern->ctx_regsave[TKPE_REGSAVE_SPC] =
> taskern->ctx_regsave[TKPE_REGSAVE_SPC_LO];
> taskern->ctx_regsave[TKPE_REGSAVE_EXSR] =
> taskern->ctx_regsave[TKPE_REGSAVE_EXSR_LO];
> }
> ...
> __ifarch(has_xmov) //only do this if we have XMOV
> {
> taskern->ctx_regsave[TKPE_REGSAVE_PCH]=__arch_pch;
> taskern->ctx_regsave[TKPE_REGSAVE_GBH]=__arch_gbh;
> }
> ...
>
> Where, it mostly deals with the 32 GPR config by copying some of the
> registers around so that they are in the correct places for the other
> code (with most of the kernel assuming 64 GPRs, but needing to wrangle
> it around when saving and restoring task contexts).
>
> While __ARCH_SIZEOF_REGSAVE__ looks like a normal preprocessor constant,
> it doesn't actually expand to a constant, but instead an "architecture
> defined variable" (which turns into a constant later on).
>
> Similarly, the "__arch_reg" variables allow exposing the underlying CPU
> control registers as-if they were C variables (though, the exact
> mechanism differs slightly in this case).
>
>
>
> Where, as noted, BGBCC doesn't handle static libraries by compiling
> directly to machine code, but instead first compiles to a stack-oriented
> bytecode format (RIL, along vaguely similar lines to JVM or .NET
> bytecode), which is then translated to 3-address-code and then machine
> code while compiling the final binary.
>
> So, in this case, if the "ifarch" remains flexible in the RIL code, then
> it can resolved once the RIL is translated to 3AC for the final
> compilation (this is typically also when things like "struct layout" and
> "sizeof(whatever)" is fully sorted out).
>
>
> RIL is kinda messy in terms of its design, and the whole image basically
> needs to be decoded in a linear fashion (it is not a conventional
> structured format, rather the entire image, including all the metadata,
> is expressed as a long linear stream of bytecode ops), but "basically
> works"... Most efforts to work on replacements have tended to stall though.
>
>
> For global declarations, it may cause the "global object" to be
> suppressed, where the compiler pretends as-if it did not exist.
>
> For code-blocks, it is more mundane:
> It is compiled mostly like a normal "if" block, except that the "if()"
> expression is resolved to a constant based on architectural feature
> flags and (may) allow the resulting basic-block to be either included or
> omitted from the binary (though, may not be sufficient to avoid the
> compiler from complaining about missing dependencies if the variables or
> functions used only exist along certain branches of the "ifarch" tree).
>
> These are mostly using normal C expression syntax, with some
> restrictions (one can't access normal runtime variables or call
> functions or similar; only use things that will be able to evaluate to a
> constant at compile time).
>
>
>
> But, the final binaries still end up being specific to a particular
> configuration, which is still kind of annoying.
>
> And, getting everything in place so that I can use an Arty S7-50 to
> drive a small robot around is being a lot more effort than ideal (mostly
> because this has led to a *lot* of these types of issues, as a lot of my
> code had implicitly drifted to assumptions of a "bigger" target, and I
> need to scale things back some and get things working again).
>
> Well, and in addition still need to write the robot control code and
> other stuff, still with some uncertainty about the whole "camera module"
> thing (maybe I should just get some IR or ultrasonic distance sensors?...).
>
> Granted, "easy path" in this case would be "just use a RasPi", but alas.
> My recent tinkering with neural nets was also partly related to this
> sub-case, as I had intended to try to use NNs for processing camera
> data, but this still seems like it is "pushing it".
>
> ...
>
>
> Any thoughts?...
It is a fun project, and you are complaining on excessive amount of fun...
In the real world there are goals and constraints.
You need to specify yours, set priorities, and take realistic approach.
It is hard to eat a cake and have it!
Time is your most precious resource.