Anton Ertl wrote:

> mitchalsup@aol.com (MitchAlsup1) writes:

> I don't think that it's plausible that the 88120, which would have
> appeared in the mid-1990s would perform as well or better than
> goldencove on this workload. My guess is that it would have had to
> undergo a silicon diet like the PPC620, probably even more so, because
> it was to appear earlier, which probably would have meant less
> transistors, which would have reduced the matrix300 IPC, and probably
> to a lesser amount, the Xlisp IPC.

> Also, the question is how fast the result would clock. The 88110 was
> available in 1992 at 50MHz, in the same year as the 200MHz 21064.
> When would the 88120 have been available at what clock rate? The

We were on schedule when I left in 1992 to be in silicon by 1995.
We were targeting 100 MHz.
We did not get far enough to know the die-size*.

(*) Motorola was offering us 0.5µ BiCMOS. SPICE simulations indicated
that the wire attached to the emitters was going to be suspect to
crystal migration due to current density. We (our project) was only
using the bipolars in sense amplifiers and not in normal gates.

> PPC620 was available in 1997 at up to 150MHz, while the Pentium II
> Klamath was available in 1997 at clock rates up to 300MHz, and the
> (in-order) 21164a was available at 600MHz; my guess is that the 21164a
> could also produce good matrix300 numbers.

> - anton

On Sun, 03 Mar 2024 16:33:45 GMT
anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

> Michael S <already5chosen@yahoo.com> writes:
> >I can't find information about Matrix300.
> >It seems to be part of SPEC89 FP suite, but spec.org does not provide
> >info about anything older than SPEC92.
> >Can you tell me what exactly does it do?
>
> It's 300x300 FP matrix multiply (not sure if single or double). There
> was a company that had a tool (famous at the time, but I don't
> remember the name) that could transform the original source code into
> a cache-blocked variant, which typically ran at the limits imposed by
> the FUs. Eventually everyone used that tool in their compiler to get
> good SPEC89 results. As a consequence, SPEC eliminated matrix300 in
> SPEC92.
>
> - anton

So, in today's world it would be something like "How fast can you do
DGEMM with 7 out of your 8 [SIMD] hands tied behind your back?"
Or, may be, more than 7 if your variant of AMX supports double
precision.
The challenge is, funny, but the answer not particularly useful.
But even in that not particularly useful answer IPC appears to be the
least useful part. Far worse than FLOPS/Hz.

Michael S <already5chosen@yahoo.com> writes:
>But even in that not particularly useful answer IPC appears to be the
>least useful part. Far worse than FLOPS/Hz.

Those were the days before SIMD, so IPC told you a little about
FLOPS/Hz. I think, though, that you look at it from the other end.
You are asking: Is that a number I want to know for evaluating DGEMM
performance on the 88120? But Mitch Alsup and the other people were
probably thinking: this uarchitecture can never exceed 6 IPC, so
getting 5.9 IPC on an actual SPEC CPU89 benchmark is pretty good. And
matrix300 with its mixture of FP adds, FP muls, loads, stores, address
arithmetic, and control, i.e., making relatively balanced use of many
FUs, is a pretty good benchmark for getting high numbers of this kind.

I guess that the 21164 also showed close to 4 IPC on the 4-wide 21164
on matrix300, while its 2 integer units would limit it to much lower
performance on, say, intmm.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Michael S wrote:

> On Sun, 03 Mar 2024 16:33:45 GMT
> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

>> Michael S <already5chosen@yahoo.com> writes:
>> >I can't find information about Matrix300.
>> >It seems to be part of SPEC89 FP suite, but spec.org does not provide
>> >info about anything older than SPEC92.
>> >Can you tell me what exactly does it do?
>>
>> It's 300x300 FP matrix multiply (not sure if single or double). There
>> was a company that had a tool (famous at the time, but I don't
>> remember the name) that could transform the original source code into
>> a cache-blocked variant, which typically ran at the limits imposed by
>> the FUs. Eventually everyone used that tool in their compiler to get
>> good SPEC89 results. As a consequence, SPEC eliminated matrix300 in
>> SPEC92.
>>
>> - anton

> So, in today's world it would be something like "How fast can you do
> DGEMM with 7 out of your 8 [SIMD] hands tied behind your back?"
> Or, may be, more than 7 if your variant of AMX supports double
> precision.

xGEMM supports transposes of the input matrixes; D stands for Double
precision, S stands for Single precision. Matrix300 used DGEMM.

SIMD would only support 2 of the 8 calls to DGEMM where the transposes
are not {'N', 'n', 'T', 't', 'C', or 'c'}. SIMD would do nothing for
the 6 transpose calls. It should also be noted that the transposed
matrixes have significantly worse cache behavior than the non trans-
posed version as each access is to a different cache line.

The problem is typical SIMD does not support the kinds of transposes
xGEMM performs. That is the problem is not the transposes, it is naïve
SIMD which is the problem.

So, postulate that one can SIMD the non-transposed loop and gain 4×.
The other 3 loops get 1× for an overall gain of <less than> 25%;
where the "less than" is due to the cache and TLB effects.

A TLB with as few as 24 entries FA gets 100% hit rate in the non-
transpose case, and poor performance on any (all) of the transpose
cases, in the dual transpose case, the TLB takes a miss every other
cache access. Here a 256-entry DM TLB gets 100% hit rate where a 64-
entry FA TLB is getting close to zero hit rate.

> The challenge is, funny, but the answer not particularly useful.

xGEMM is likely the second most used GB-math number crunching
subroutine in use--FFT <flavors> being the most used.

> But even in that not particularly useful answer IPC appears to be the
> least useful part. Far worse than FLOPS/Hz.

Correct, and this illustrates how times have changed. In 1985 Matrix300
would use * and + as separate instructions. The major loop consists of
2 LDs, 1*, 1+, 1 ST, and a ADD-CMP-BC which could be distributed over
the 4-way unrolled loop (in source code). Mc 88110 compiler would
produce a 24 instruction loop (non-transposed) and the Mc 88120 sim-
ulator would run this loop (including DRAM accesses (cache misses and
cache victims), and TLB table-walking) in 4 cycles. Today, FMAC is
<fused into> 1 instruction, saving instruction count and calculation
latency (8->5) allowing Matrix 300 to fit in a 78-instruction execution
window instead of a 96-instruction EW. {{But you still have to count
FMAC as 2 FLOPs.}}

My 66000, using VEC-LOOP the instruction count goes down again to 5 (from
6) per loop since LOOP is performing ADD-CMP-BC in a single instruction
and in a single clock.

mitchalsup@aol.com (MitchAlsup1) writes:
>SIMD would only support 2 of the 8 calls to DGEMM where the transposes
>are not {'N', 'n', 'T', 't', 'C', or 'c'}. SIMD would do nothing for
>the 6 transpose calls.

The following version is good for the non-transposed matrices (it's
not DGEMM, but the difference to DGEMM is left as exercise):

void matmul(double a[], double b[], double c[], size_t m, size_t n, size_t p)
{ size_t i,j,k;
double r;
memset(c,0,n*p*sizeof(double));
for (i=0; i<n; i++)
for (k=0; k<m; k++)
for (j=0; j<p; j++)
c[i*p+j] += a[i*m+k]*b[k*p+j];
}

But the loops are interchangeable, and the naive i,j,k order is bad
for SIMD as well as introducing a recurrence on c[i*p+j] on the inner
loop. Given the amount of parallelism inherent in matrix
multiplication, I would be surprised if one if transposing some or all
of the involved matrices prevented some loop order or other
transformation that would prevent SIMD. In the extreme case, you just
transpose an appropriate input matrix at the start, or the result at
the end, at O(n^2) effort (for n*n matrices), while the matrix
multiply itself takes O(n^3) effort.

But of course, for the 88120 that was a non-issue, because it did not
have SIMD.

>It should also be noted that the transposed
>matrixes have significantly worse cache behavior than the non trans-
>posed version as each access is to a different cache line.

That was fixed by the cache-blocking transformation that everybody
used, and which resulted in the elimination of matrix300 from SPEC92.

It was not the cache sizes, which could easily have been addressed by
modifying matrix300 into, say, matrix2000.

>My 66000, using VEC-LOOP the instruction count goes down again to 5 (from
>6) per loop since LOOP is performing ADD-CMP-BC in a single instruction
>and in a single clock.

If the programmer (or compiler) for My66000 does not process the
elements in the favoured order, it will not perform particularly good
for arbitrary transpositions, either. I don't think that you want to
perform these program transformations in hardware, do you?

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Sun, 03 Mar 2024 22:22:37 GMT
anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

> Michael S <already5chosen@yahoo.com> writes:
> >But even in that not particularly useful answer IPC appears to be the
> >least useful part. Far worse than FLOPS/Hz.
>
>
> I guess that the 21164 also showed close to 4 IPC on the 4-wide 21164
> on matrix300, while its 2 integer units would limit it to much lower
> performance on, say, intmm.
>
> - anton

I don't know about specific case of matrix300 and what transformations
are allowed by SPEC rules and what not, but if I were tasked with
writing generic DGEMM for Alpha 21164 with maximal performance on
non-small relatively square matrices as a amin goal, then I'd start
with something like that:

// main_loop_3x6 - multiply 3 raws of A[][]
// by 6 columns of B[][] assuming C-language order
void innermost_loop_3x6(
const double* A, int lda,
const double* B, int ldb,
double* C, int ldc,
int n)
{ const double* A0 = A;
const double* A1 = A0 + lda;
const double* A2 = A1 + lda;
double acc00 = 0, acc10 = 0, acc20 = 0;
double acc01 = 0, acc11 = 0, acc21 = 0;
double acc02 = 0, acc12 = 0, acc22 = 0;
double acc03 = 0, acc13 = 0, acc23 = 0;
double acc04 = 0, acc14 = 0, acc24 = 0;
double acc05 = 0, acc15 = 0, acc25 = 0;

for (int i = 0; i < n; ++i) {
double a0 = A0[i];
double a1 = A1[i];
double a2 = A2[i];
double b;
b = B[0]; acc00 += a0 * b; acc10 += a1 * b; acc20 += a2 * b;
b = B[1]; acc01 += a0 * b; acc11 += a1 * b; acc21 += a2 * b;
b = B[2]; acc02 += a0 * b; acc12 += a1 * b; acc22 += a2 * b;
b = B[3]; acc03 += a0 * b; acc13 += a1 * b; acc23 += a2 * b;
b = B[4]; acc04 += a0 * b; acc14 += a1 * b; acc24 += a2 * b;
b = B[5]; acc05 += a0 * b; acc15 += a1 * b; acc25 += a2 * b;
B += ldb;
}

double* C0 = C;
double* C1 = C0 + ldc;
double* C2 = C1 + ldc;
C0[0] += acc00; C1[0] += acc10; C2[0] += acc20;
C0[1] += acc01; C1[1] += acc11; C2[1] += acc21;
C0[2] += acc02; C1[2] += acc12; C2[2] += acc22;
C0[3] += acc03; C1[3] += acc13; C2[3] += acc23;
C0[4] += acc04; C1[4] += acc14; C2[4] += acc24;
C0[5] += acc05; C1[5] += acc15; C2[5] += acc25;
}

The loop consists of 9 loads, 4 pointer updates,
1 counter decrement, 1 conditional branch, 18 DP multiplies and 18
DP adds. 51 total instructions.
Ideally, it will run in 18 clocks, for IPC = 2.83. Realistically on
real hardware with cache misses etc it will take 20-23 clock and IPC
would be proportionally lower.
What is my point? My point is that I expect "medium-IPC" kernel like
above to achieve higher FLOPS (== better performance) then simpler,
smaller kernel with IPC in excess of 3.5.

Michael S <already5chosen@yahoo.com> writes:
>On Sun, 03 Mar 2024 22:22:37 GMT
>anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>
>> Michael S <already5chosen@yahoo.com> writes:
>> >But even in that not particularly useful answer IPC appears to be the
>> >least useful part. Far worse than FLOPS/Hz.
>>
>>
>> I guess that the 21164 also showed close to 4 IPC on the 4-wide 21164
>> on matrix300, while its 2 integer units would limit it to much lower
>> performance on, say, intmm.
>>
>> - anton
>
>I don't know about specific case of matrix300 and what transformations
>are allowed by SPEC rules and what not, but if I were tasked with
>writing generic DGEMM for Alpha 21164 with maximal performance on
>non-small relatively square matrices as a amin goal, then I'd start
>with something like that:
>
>// main_loop_3x6 - multiply 3 raws of A[][]
>// by 6 columns of B[][] assuming C-language order
....
>The loop consists of 9 loads, 4 pointer updates,
>1 counter decrement, 1 conditional branch, 18 DP multiplies and 18
>DP adds. 51 total instructions.
>Ideally, it will run in 18 clocks, for IPC = 2.83.

Given that starting 18 FP multiplies and 18 FP additions takes 18
cycles, that is optimal. But you unrolled more than is necessary to
achieve 2FlOPC (FP operations per cycle). With less unrolling, you
could have achieved the same 2FLOPC and of course you would see higher
IPC. And as Mitch Alsup explains, his 5.9 IPC was for a non-unrolled
loop.

>What is my point? My point is that I expect "medium-IPC" kernel like
>above to achieve higher FLOPS (== better performance) then simpler,
>smaller kernel with IPC in excess of 3.5.

These days, with power limits resulting in lower clocks for programs
that do more work, yes, I guess that you will see better FLOPS from
variants that execute fewer instructions. But in the 90s, CPUs ran at
their rated clock rate no matter what, and a 21164 would run a variant
that does 2 FLOPc at the same speed as any other 2 FLOPC variant,
whether that variant performs 0.83 non-flop instructions/cycle or 1.9
non-flop instructions/cycle.

But yes, 5.9 IPC on matrix300 shows little about the matrix multiply
performance. Still, I think that the point is that there are many
hurdles that might result in a lower IPC (for code where only 6IPC
means 2FLOPC), the fact that they achieved 5.9 indicates that they
managed to lower the hurdles a lot; true, it would be better if they
could have shown it with code where 6IPC is more meaningful.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Anton Ertl wrote:

> Michael S <already5chosen@yahoo.com> writes:
>>On Sun, 03 Mar 2024 22:22:37 GMT
>>anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>>
>>> Michael S <already5chosen@yahoo.com> writes:
>>> >But even in that not particularly useful answer IPC appears to be the
>>> >least useful part. Far worse than FLOPS/Hz.
>>>
>>>
>>> I guess that the 21164 also showed close to 4 IPC on the 4-wide 21164
>>> on matrix300, while its 2 integer units would limit it to much lower
>>> performance on, say, intmm.
>>>
>>> - anton
>>
>>I don't know about specific case of matrix300 and what transformations
>>are allowed by SPEC rules and what not, but if I were tasked with
>>writing generic DGEMM for Alpha 21164 with maximal performance on
>>non-small relatively square matrices as a amin goal, then I'd start
>>with something like that:
>>
>>// main_loop_3x6 - multiply 3 raws of A[][]
>>// by 6 columns of B[][] assuming C-language order
> ....
>>The loop consists of 9 loads, 4 pointer updates,
>>1 counter decrement, 1 conditional branch, 18 DP multiplies and 18
>>DP adds. 51 total instructions.
>>Ideally, it will run in 18 clocks, for IPC = 2.83.

> Given that starting 18 FP multiplies and 18 FP additions takes 18
> cycles, that is optimal. But you unrolled more than is necessary to
> achieve 2FlOPC (FP operations per cycle). With less unrolling, you
> could have achieved the same 2FLOPC and of course you would see higher
> IPC. And as Mitch Alsup explains, his 5.9 IPC was for a non-unrolled
> loop.

>>What is my point? My point is that I expect "medium-IPC" kernel like
>>above to achieve higher FLOPS (== better performance) then simpler,
>>smaller kernel with IPC in excess of 3.5.

> These days, with power limits resulting in lower clocks for programs
> that do more work, yes, I guess that you will see better FLOPS from
> variants that execute fewer instructions. But in the 90s, CPUs ran at
> their rated clock rate no matter what, and a 21164 would run a variant
> that does 2 FLOPc at the same speed as any other 2 FLOPC variant,
> whether that variant performs 0.83 non-flop instructions/cycle or 1.9
> non-flop instructions/cycle.

> But yes, 5.9 IPC on matrix300 shows little about the matrix multiply
> performance. Still, I think that the point is that there are many
> hurdles that might result in a lower IPC (for code where only 6IPC
> means 2FLOPC), the fact that they achieved 5.9 indicates that they
> managed to lower the hurdles a lot; true, it would be better if they
> could have shown it with code where 6IPC is more meaningful.

The processor for which that IPC was stated had a 16KB L1 DM Cache
with 4 banks used twice per cycle and a 16 byte line, so DGEMM was
<essentially> always taking cache misses (every other cycle). Most
of the performance of the overall design came down to 3 things::
The DRAM memory system which could start 2 new accesses every cycle
1 RD 1 WT; The zero cycle branch mispredict repair, and
The short 6-cycle pipeline from Fetch to Retire.

And most of this centered around what we called the conditional
cache--a RoB for memory <if you will>--a place where ST could be
placed and LDs could access but would not get written to L1 if
the instruction packet could not retire (mispredict or exception).

No processor today is doing any of these (well maybe My 66600...)
The BOOM RISC-V processor has a 7 stage front-end and 3 cycle branch
mispredict repair...

> - anton

MitchAlsup1 wrote:
> EricP wrote:
>> My only exception handler that is triggered with any regularity is
>> page fault (assuming a hardware table walker so no TLB miss exceptions),
>> and it typically invokes a handler with many thousands of instructions
>> so prefetching that code a few clocks earlier won't make any difference.
>
> If you use it often enough it will still be in your cache when you next
> need it. {I don't remember exactly who told me this, but it was one of
> the original MIPS (the company not Stanford) guys}; so you don't need to
> prefetch it.

That has been my rule-of-thumb for lookup tables replacing logic: If the
table is small enough and used often enough that it could make a
significant difference to the total runtime, then it will also stay in
cache nearly all the time.

if it does get evicted between uses most of the time, then it simply
wasn't that important.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On Mon, 04 Mar 2024 18:18:35 GMT
anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

> Michael S <already5chosen@yahoo.com> writes:
> >On Sun, 03 Mar 2024 22:22:37 GMT
> >anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
> >
> >> Michael S <already5chosen@yahoo.com> writes:
> >> >But even in that not particularly useful answer IPC appears to be
> >> >the least useful part. Far worse than FLOPS/Hz.
> >>
> >>
> >> I guess that the 21164 also showed close to 4 IPC on the 4-wide
> >> 21164 on matrix300, while its 2 integer units would limit it to
> >> much lower performance on, say, intmm.
> >>
> >> - anton
> >
> >I don't know about specific case of matrix300 and what
> >transformations are allowed by SPEC rules and what not, but if I
> >were tasked with writing generic DGEMM for Alpha 21164 with maximal
> >performance on non-small relatively square matrices as a amin goal,
> >then I'd start with something like that:
> >
> >// main_loop_3x6 - multiply 3 raws of A[][]
> >// by 6 columns of B[][] assuming C-language order
> ...
> >The loop consists of 9 loads, 4 pointer updates,
> >1 counter decrement, 1 conditional branch, 18 DP multiplies and 18
> >DP adds. 51 total instructions.
> >Ideally, it will run in 18 clocks, for IPC = 2.83.
>
> Given that starting 18 FP multiplies and 18 FP additions takes 18
> cycles, that is optimal. But you unrolled more than is necessary to
> achieve 2FlOPC (FP operations per cycle). With less unrolling, you
> could have achieved the same 2FLOPC and of course you would see higher
> IPC. And as Mitch Alsup explains, his 5.9 IPC was for a non-unrolled
> loop.
>
> >What is my point? My point is that I expect "medium-IPC" kernel like
> >above to achieve higher FLOPS (== better performance) then simpler,
> >smaller kernel with IPC in excess of 3.5.
>
> These days, with power limits resulting in lower clocks for programs
> that do more work, yes, I guess that you will see better FLOPS from
> variants that execute fewer instructions. But in the 90s, CPUs ran at
> their rated clock rate no matter what, and a 21164 would run a variant
> that does 2 FLOPc at the same speed as any other 2 FLOPC variant,
> whether that variant performs 0.83 non-flop instructions/cycle or 1.9
> non-flop instructions/cycle.
>
> But yes, 5.9 IPC on matrix300 shows little about the matrix multiply
> performance. Still, I think that the point is that there are many
> hurdles that might result in a lower IPC (for code where only 6IPC
> means 2FLOPC), the fact that they achieved 5.9 indicates that they
> managed to lower the hurdles a lot; true, it would be better if they
> could have shown it with code where 6IPC is more meaningful.
>
> - anton

On Mon, 04 Mar 2024 18:18:35 GMT
anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

> Michael S <already5chosen@yahoo.com> writes:
> >On Sun, 03 Mar 2024 22:22:37 GMT
> >anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
> >
> >> Michael S <already5chosen@yahoo.com> writes:
> >> >But even in that not particularly useful answer IPC appears to be
> >> >the least useful part. Far worse than FLOPS/Hz.
> >>
> >>
> >> I guess that the 21164 also showed close to 4 IPC on the 4-wide
> >> 21164 on matrix300, while its 2 integer units would limit it to
> >> much lower performance on, say, intmm.
> >>
> >> - anton
> >
> >I don't know about specific case of matrix300 and what
> >transformations are allowed by SPEC rules and what not, but if I
> >were tasked with writing generic DGEMM for Alpha 21164 with maximal
> >performance on non-small relatively square matrices as a amin goal,
> >then I'd start with something like that:
> >
> >// main_loop_3x6 - multiply 3 raws of A[][]
> >// by 6 columns of B[][] assuming C-language order
> ...
> >The loop consists of 9 loads, 4 pointer updates,
> >1 counter decrement, 1 conditional branch, 18 DP multiplies and 18
> >DP adds. 51 total instructions.
> >Ideally, it will run in 18 clocks, for IPC = 2.83.
>
> Given that starting 18 FP multiplies and 18 FP additions takes 18
> cycles, that is optimal. But you unrolled more than is necessary to
> achieve 2FlOPC (FP operations per cycle). With less unrolling, you
> could have achieved the same 2FLOPC and of course you would see higher
> IPC. And as Mitch Alsup explains, his 5.9 IPC was for a non-unrolled
> loop.
>
> >What is my point? My point is that I expect "medium-IPC" kernel like
> >above to achieve higher FLOPS (== better performance) then simpler,
> >smaller kernel with IPC in excess of 3.5.
>
> These days, with power limits resulting in lower clocks for programs
> that do more work, yes, I guess that you will see better FLOPS from
> variants that execute fewer instructions. But in the 90s, CPUs ran at
> their rated clock rate no matter what, and a 21164 would run a variant
> that does 2 FLOPc at the same speed as any other 2 FLOPC variant,
> whether that variant performs 0.83 non-flop instructions/cycle or 1.9
> non-flop instructions/cycle.
>

In 90-x CPUs had other reasons to minimize the # of instructions and
esp. the # of load instructions per task. E.g. too few banks in L1D
cache, so the cache that in theory supports two accesses per clock in
practice is closer to 1. E.g. very few hits served under miss. E.g. low
associativity. E.g. theoretically 4-wide instruction Fetch/Decode that
in practice delivers 4 decoded instructions only when all inner planets
in solar system are aligned.
According to my understanding, 21164 being speed racer suffered from
that sort of problems more than most competitors.

> But yes, 5.9 IPC on matrix300 shows little about the matrix multiply
> performance. Still, I think that the point is that there are many
> hurdles that might result in a lower IPC (for code where only 6IPC
> means 2FLOPC), the fact that they achieved 5.9 indicates that they
> managed to lower the hurdles a lot; true, it would be better if they
> could have shown it with code where 6IPC is more meaningful.
>
> - anton

Michael S wrote:

> On Mon, 04 Mar 2024 18:18:35 GMT
> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

>> These days, with power limits resulting in lower clocks for programs
>> that do more work, yes, I guess that you will see better FLOPS from
>> variants that execute fewer instructions. But in the 90s, CPUs ran at
>> their rated clock rate no matter what, and a 21164 would run a variant
>> that does 2 FLOPc at the same speed as any other 2 FLOPC variant,
>> whether that variant performs 0.83 non-flop instructions/cycle or 1.9
>> non-flop instructions/cycle.
>>

> In 90-x CPUs had other reasons to minimize the # of instructions and

Everyone Always have excellent reasons to minimize the number of instructions.

Over in CISC-land, it takes fewer instructions to get the job done.
Over in RISC-land, the instructions run in fewer nanoseconds.

The critical term in performance is::

seconds/program = instructions/program × cycles/instruction × seconds/cycle

RISCs tend to get instructions/program wrong and cycles/instruction right,
while
CISCs tend to get instructions/program right and cycles/instruction wrong.

I happen to believe that between RISC and CISC is a realm where one needs
fewer instructions but sacrifices essentially nothing in the frequency
department.

My 66000 tends to have only 10% more instructions than VAX while RISC-V
tends to have 50% more instructions than VAX--My 66000 needs only 71% the
instruction count as RISC-V.

> esp. the # of load instructions per task. E.g. too few banks in L1D
> cache, so the cache that in theory supports two accesses per clock in
> practice is closer to 1.

CISCs generally have a 45%-50% memory reference density, while
RISCs generally have a 30%-35% memory reference density. So, CISCs tend
to run into the cache banking wall at 2 IPC, while RISCs delay that wall
until 3 IPC.

> E.g. very few hits served under miss.

Accesses are correlated, so this is to be expected. The real question is
whether you can still perform with miss under miss !! Even if you don't
take hits, you can still get the next request out "there" sooner. Sooner
saves latency.

> E.g. low associativity.

Associativity costs power and area.

> E.g. theoretically 4-wide instruction Fetch/Decode that
> in practice delivers 4 decoded instructions only when all inner planets
> in solar system are aligned.

A 4-wide instruction fetch yields only 2.5 instructions per random access.
This is just std math:: (1+2+3+4)/4 = 2.5

But access to good predication means up to 1/3rd of all short branches can
be avoided. Few ISAs have access to "good" predication. Here, a good solution
for predication, drives the random 4-wide fetch access to deliver 3.25 instruc-
tion per fetch; a 50% increase.

> According to my understanding, 21164 being speed racer suffered from
> that sort of problems more than most competitors.

Because it was wider and faster it was more dependent on "everything
working well all the time" and the fact that it was high frequency
than others means all its bad cache behavior got multiplied by the
latency to deeper levels of the memory hierarchy !

On 3/4/2024 4:05 PM, MitchAlsup1 wrote:
> Michael S wrote:
>
>> On Mon, 04 Mar 2024 18:18:35 GMT
>> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>
>>> These days, with power limits resulting in lower clocks for programs
>>> that do more work, yes, I guess that you will see better FLOPS from
>>> variants that execute fewer instructions.  But in the 90s, CPUs ran at
>>> their rated clock rate no matter what, and a 21164 would run a variant
>>> that does 2 FLOPc at the same speed as any other 2 FLOPC variant,
>>> whether that variant performs 0.83 non-flop instructions/cycle or 1.9
>>> non-flop instructions/cycle.
>>>
>
>> In 90-x CPUs had other reasons to minimize the # of instructions and
>
> Everyone Always have excellent reasons to minimize the number of
> instructions.
>
> Over in CISC-land, it takes fewer instructions to get the job done.
> Over in RISC-land, the instructions run in fewer nanoseconds.
>
> The critical term in performance is::
>
> seconds/program = instructions/program × cycles/instruction × seconds/cycle
>
> RISCs tend to get instructions/program wrong and cycles/instruction
> right, while
> CISCs tend to get instructions/program right and cycles/instruction wrong.
>
> I happen to believe that between RISC and CISC is a realm where one needs
> fewer instructions but sacrifices essentially nothing in the frequency
> department.
>
> My 66000 tends to have only 10% more instructions than VAX while RISC-V
> tends to have 50% more instructions than VAX--My 66000 needs only 71%
> the instruction count as RISC-V.
>
>> esp. the # of load instructions per task. E.g. too few banks in L1D
>> cache, so the cache that in theory supports two accesses per clock in
>> practice is closer to 1.
>
> CISCs generally have a 45%-50% memory reference density, while
> RISCs generally have a 30%-35% memory reference density.

If those percentages are number of loads & stores divided by total
instruction count, isn't this just a restatement of your previous point
that CISCs need fewer instructions to do the job? i.e. the *time*
between loads or stores is the same for RISCs and CISCs?

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 3/4/24 3:52 AM, Anton Ertl wrote:
[snip]
> That was fixed by the cache-blocking transformation that everybody
> used, and which resulted in the elimination of matrix300 from SPEC92.

I seem to recall that art from SPEC CPU 2000 had a similar issue
where array of structures to structure of arrays compiler
transformation significantly improved performance.

Stephen Fuld wrote:

> On 3/4/2024 4:05 PM, MitchAlsup1 wrote:
>> Michael S wrote:
>>
>>> On Mon, 04 Mar 2024 18:18:35 GMT
>>> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>>
>>>> These days, with power limits resulting in lower clocks for programs
>>>> that do more work, yes, I guess that you will see better FLOPS from
>>>> variants that execute fewer instructions.  But in the 90s, CPUs ran at
>>>> their rated clock rate no matter what, and a 21164 would run a variant
>>>> that does 2 FLOPc at the same speed as any other 2 FLOPC variant,
>>>> whether that variant performs 0.83 non-flop instructions/cycle or 1.9
>>>> non-flop instructions/cycle.
>>>>
>>
>>> In 90-x CPUs had other reasons to minimize the # of instructions and
>>
>> Everyone Always have excellent reasons to minimize the number of
>> instructions.
>>
>> Over in CISC-land, it takes fewer instructions to get the job done.
>> Over in RISC-land, the instructions run in fewer nanoseconds.
>>
>> The critical term in performance is::
>>
>> seconds/program = instructions/program × cycles/instruction × seconds/cycle
>>
>> RISCs tend to get instructions/program wrong and cycles/instruction
>> right, while
>> CISCs tend to get instructions/program right and cycles/instruction wrong.
>>
>> I happen to believe that between RISC and CISC is a realm where one needs
>> fewer instructions but sacrifices essentially nothing in the frequency
>> department.
>>
>> My 66000 tends to have only 10% more instructions than VAX while RISC-V
>> tends to have 50% more instructions than VAX--My 66000 needs only 71%
>> the instruction count as RISC-V.
>>
>>> esp. the # of load instructions per task. E.g. too few banks in L1D
>>> cache, so the cache that in theory supports two accesses per clock in
>>> practice is closer to 1.
>>
>> CISCs generally have a 45%-50% memory reference density, while
>> RISCs generally have a 30%-35% memory reference density.

> If those percentages are number of loads & stores divided by total
> instruction count, isn't this just a restatement of your previous point
> that CISCs need fewer instructions to do the job? i.e. the *time*
> between loads or stores is the same for RISCs and CISCs?

Not when you include various other facts::
CISCs tend to have fewer registers
CISCs tend to have LD-OPs and some have LD-OP-STs
Both of the above give the compiler the illusion that inbound memory
references are less expensive than a typical LD because you get the
LD, and you don't have to waste a precious register. Thus there are
more memory references--but RISC compilers have taught us that more
registers beats LD-OPs--pipeline designers have taught us that thin-
ner pipelines perform better--both stand against LD-OPs and LD-OP-STs.

VAX went so far as to allow any operand and any result to be memory
{Most of us now believe this was a massive overstep.}

CISCs really do perform more memory references--not by as much as the
above statistics imply, but significantly more memory references.

On 3/4/2024 6:33 PM, MitchAlsup1 wrote:
> Stephen Fuld wrote:
>
>> On 3/4/2024 4:05 PM, MitchAlsup1 wrote:

snip

>>> CISCs generally have a 45%-50% memory reference density, while
>>> RISCs generally have a 30%-35% memory reference density.
>
>> If those percentages are number of loads & stores divided by total
>> instruction count, isn't this just a restatement of your previous
>> point that CISCs need fewer instructions to do the job?  i.e. the
>> *time* between loads or stores is the same for RISCs and CISCs?
>
>
> Not when you include various other facts::
> CISCs tend to have fewer registers
> CISCs tend to have LD-OPs and some have LD-OP-STs
> Both of the above give the compiler the illusion that inbound memory
> references are less expensive than a typical LD because you get the
> LD, and you don't have to waste a precious register. Thus there are
> more memory references--but RISC compilers have taught us that more
> registers beats LD-OPs--pipeline designers have taught us that thin-
> ner pipelines perform better--both stand against LD-OPs and LD-OP-STs.
>
> VAX went so far as to allow any operand and any result to be memory
> {Most of us now believe this was a massive overstep.}
>
> CISCs really do perform more memory references--not by as much as the
> above statistics imply, but significantly more memory references.

Interesting. Thanks.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

mitchalsup@aol.com (MitchAlsup1) writes:
>Stephen Fuld wrote:
>
>> On 3/4/2024 4:05 PM, MitchAlsup1 wrote:

>>>
>>> CISCs generally have a 45%-50% memory reference density, while
>>> RISCs generally have a 30%-35% memory reference density.
>
>> If those percentages are number of loads & stores divided by total
>> instruction count, isn't this just a restatement of your previous point
>> that CISCs need fewer instructions to do the job? i.e. the *time*
>> between loads or stores is the same for RISCs and CISCs?
>
>
>Not when you include various other facts::
>CISCs tend to have fewer registers

How much of that is because active CISC architectures
are forty or fifty years old?

Would a modern, designed from scratch, CISC architecture
still restrict the number of registers?

If memory access ever becomes as fast a register access,
all bets will be off...

Scott Lurndal wrote:

> mitchalsup@aol.com (MitchAlsup1) writes:
>>Stephen Fuld wrote:
>>
>>> On 3/4/2024 4:05 PM, MitchAlsup1 wrote:

>>>>
>>>> CISCs generally have a 45%-50% memory reference density, while
>>>> RISCs generally have a 30%-35% memory reference density.
>>
>>> If those percentages are number of loads & stores divided by total
>>> instruction count, isn't this just a restatement of your previous point
>>> that CISCs need fewer instructions to do the job? i.e. the *time*
>>> between loads or stores is the same for RISCs and CISCs?
>>
>>
>>Not when you include various other facts::
>>CISCs tend to have fewer registers

> How much of that is because active CISC architectures
> are forty or fifty years old?

The problems of encoding remain as relevant today as 50 years ago.
But the things one wants the ISA to do are larger today than 50 YA.
Those encodings with LD-OPs are pretty much restricted to 16 registers
(16 base registers) and here you still have OpCode mapping difficulties.

If you give up on LD-OPs to gain register count, you are already in the
RISC-camp.

> Would a modern, designed from scratch, CISC architecture
> still restrict the number of registers?

I wanted to do a 64-bit VAX minus the indirect address modes and
give it 32 registers. Never got around to it though.

My experience with My 66000 indicates one can get vanishingly close
to VAX instruction density (and count) with a RISC ISA done right.

> If memory access ever becomes as fast a register access,
> all bets will be off...

It won't, and never has.

mitchalsup@aol.com (MitchAlsup1) writes:
>Scott Lurndal wrote:

>> If memory access ever becomes as fast a register access,
>> all bets will be off...
>
>It won't, and never has.

There were a number of historic implementations where the
registers were actually stored in low memory.

Scott Lurndal wrote:

> mitchalsup@aol.com (MitchAlsup1) writes:
>>Scott Lurndal wrote:

>>> If memory access ever becomes as fast a register access,
>>> all bets will be off...
>>
>>It won't, and never has.

> There were a number of historic implementations where the
> registers were actually stored in low memory.

Yes, but that is making registers as slow as memory,
not making memory as fast as registers.

scott@slp53.sl.home (Scott Lurndal) writes:
>Would a modern, designed from scratch, CISC architecture
>still restrict the number of registers?

Designed from scratch? AVX-512 supports 32 xmm/ymm/zmm registers.
Intel APX will support 32 GPRs.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

mitchalsup@aol.com (MitchAlsup1) writes:
>Scott Lurndal wrote:
>
>> mitchalsup@aol.com (MitchAlsup1) writes:
>>>Scott Lurndal wrote:
>
>>>> If memory access ever becomes as fast a register access,
>>>> all bets will be off...
>>>
>>>It won't, and never has.
>
>> There were a number of historic implementations where the
>> registers were actually stored in low memory.
>
>
>Yes, but that is making registers as slow as memory,
> not making memory as fast as registers.

With something like SRAM, with less than 1ns latency, then. With that
you might completely eliminate registers and do everything
memory-to-memory. The small gain lost by eliminating
registers will likely be offset by fewer instructions
to execute. Sufficient for most desktop users, surely.

That's assuming SRAM can scale at reasonable cost due
to some future technology breakthrough (or some other
future non-volatile memory technology with favorable
access timing).

Scott Lurndal wrote:

> mitchalsup@aol.com (MitchAlsup1) writes:
>>Scott Lurndal wrote:
>>
>>> mitchalsup@aol.com (MitchAlsup1) writes:
>>>>Scott Lurndal wrote:
>>
>>>>> If memory access ever becomes as fast a register access,
>>>>> all bets will be off...
>>>>
>>>>It won't, and never has.
>>
>>> There were a number of historic implementations where the
>>> registers were actually stored in low memory.
>>
>>
>>Yes, but that is making registers as slow as memory,
>> not making memory as fast as registers.

> With something like SRAM, with less than 1ns latency, then. With that
> you might completely eliminate registers and do everything
> memory-to-memory. The small gain lost by eliminating
> registers will likely be offset by fewer instructions
> to execute. Sufficient for most desktop users, surely.

Registers are 200ps, while on-die SRAM can cycle at 200ps
you still have to pay the latency to route address-bits
from AGEN to the SRAM arrays, and route the data back,
while one can access a register and run through the
forwarding logic in the same cycle. So, you are essentially
comparing something that costs ½* cycle to one that costs
1¼ cycles.

(*) maybe ¾-cycle on a GB physical RF.

> That's assuming SRAM can scale at reasonable cost due
> to some future technology breakthrough (or some other
> future non-volatile memory technology with favorable
> access timing).

I have lived in FAB environments where I was told "SRAM
will approach DRAM densities* in a generation or two, too.
Still, yet to happen.

(*) They thought they had to keep the DRAM capacitor above
a certain amount of fF to retain the 8ms (-16ms) refresh
rates. What they really had to do was control leakage !

On Tue, 05 Mar 2024 18:10:06 GMT
scott@slp53.sl.home (Scott Lurndal) wrote:

> mitchalsup@aol.com (MitchAlsup1) writes:
> >Scott Lurndal wrote:
> >
> >> mitchalsup@aol.com (MitchAlsup1) writes:
> >>>Scott Lurndal wrote:
> >
> >>>> If memory access ever becomes as fast a register access,
> >>>> all bets will be off...
> >>>
> >>>It won't, and never has.
> >
> >> There were a number of historic implementations where the
> >> registers were actually stored in low memory.
> >
> >
> >Yes, but that is making registers as slow as memory,
> > not making memory as fast as registers.
>
> With something like SRAM, with less than 1ns latency, then. With that
> you might completely eliminate registers and do everything
> memory-to-memory. The small gain lost by eliminating
> registers will likely be offset by fewer instructions
> to execute. Sufficient for most desktop users, surely.
>
> That's assuming SRAM can scale at reasonable cost due
> to some future technology breakthrough

1ns latency means the size of your SRAM array is 256KB at best.
More realistically 128 KB.
The only possible breakthrough I can see at this front is 3D
working much better than anticipated (and better than how well it works
for 3D NAND flash). But even that can improve capacity by factor of 200
at best, more realistically by factor of 100.
So, in the best possible future scenario, given all benefits of doubt
1ns latency requirement limits the size of your SRAM to ~50 MB.
Make it at least 5 ns instead of 1 then you can start dreaming.
Still more for benefit of your grandchildren rather than for yourself.

> (or some other
> future non-volatile memory technology with favorable
> access timing).

On Tue, 5 Mar 2024 19:28:01 +0000
mitchalsup@aol.com (MitchAlsup1) wrote:
>
> I have lived in FAB environments where I was told "SRAM
> will approach DRAM densities* in a generation or two, too.
> Still, yet to happen.
>
> (*) They thought they had to keep the DRAM capacitor above
> a certain amount of fF to retain the 8ms (-16ms) refresh
> rates. What they really had to do was control leakage !

According to what I read on RWT forum, in recent 3-4 years the table
was turned completely - SRAM-to-DRAM area ratio is growing rather than
shrinking. Slowly, of course, nowadays everything is slow.