On 6/8/2023 10:00 AM, MitchAlsup wrote:
> On Wednesday, June 7, 2023 at 4:42:36 PM UTC-5, Brett wrote:

snip

>> With the M2 machines Apple has dumped DIMMs and brought RAM onto the CPU
>> carrier, quadrupling the dram bandwidth.
> <
> And makes upgrading DRAM size impossible.

I don't know how much better performance the quadrupling of DRAM
bandwidth provides, but if it is at all significant, I would bet that
the average Mac customer would prefer better performance to DRAM size
upgradeability.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
>I don't know how much better performance the quadrupling of DRAM
>bandwidth provides

That depends on the application. You can get some idea by looking at
the 4-channel (3990X) vs. 8-channel (3995WX) results on
<https://www.anandtech.com/print/16478/64-cores-of-rendering-madness-the-amd-threadripper-pro-3995wx-review>.

And you can find an explicit 2-channel vs. 8-channel comparison in
<https://www.anandtech.com/print/16482/lenovo-thinkstation-p620-review-a-vehicle-for-threadripper-pro>.
In 25 out of 105 tests, the performance of the 8-channel configuration
was more than a factor 1.03 compared to the 2-channel configuration.

And note that the latter results were with 64-core CPUs with 51.2 GB/s
or 204.8GB/s memory bandwidth, wheras the M2 Max has 12 (8P+4E) cores
with (according to
<https://www.apple.com/mz/newsroom/2023/06/apple-introduces-m2-ultra/>)
400GB/s memory bandwidth and the M2 Ultra has 24 (16P+8E) cores with
800GB/s memory bandwidth. I expect that, as far as CPU performance
goes, both could do with a quarter of the bandwidth, and very few
applications would see a measurable difference; things may be
different for stuff that uses the GPU. If more CPU performance is
desired and the application is parallelizable, there are good chances
that a Threadripper 5995WX box (more than twice the cores) beats an M2
Ultra box (four times the memory bandwidth).

>but if it is at all significant, I would bet that
>the average Mac customer would prefer better performance to DRAM size
>upgradeability.

Why? Are Macs slow? I expect that my collegue would have preferred
to be able to upgrade the RAM of his laptop over having more memory
bandwidth.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Saturday, June 10, 2023 at 2:13:29 AM UTC-6, Anton Ertl wrote:
> Stephen Fuld <sf...@alumni.cmu.edu.invalid> writes:

> >but if it is at all significant, I would bet that
> >the average Mac customer would prefer better performance to DRAM size
> >upgradeability.

> Why? Are Macs slow?

I think it is rather that Macintosh computers, in general, have very limited
or no upgrade options, and thus the average Mac customer is not concerned
too much about upgradeability - those who are, are PC customers instead.

John Savard

In article <1ca69920-e709-4aac-871e-e406d6372a54n@googlegroups.com>,
MitchAlsup@aol.com (MitchAlsup) wrote:

> On Wednesday, June 7, 2023 at 4:42:36_PM UTC-5, Brett wrote:
> > With the M2 machines Apple has dumped DIMMs and brought RAM onto
> > the CPU carrier, quadrupling the dram bandwidth.
> And makes upgrading DRAM size impossible.

M-series Macs do not have spinning disks, only SSDs, and fast ones at
that. They have quite limited RAM, but they can do demand paging very
quickly. This works pretty well in my limited experience of using them
for software building and testing.

John

On 6/10/2023 12:43 AM, Anton Ertl wrote:
> Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
>> I don't know how much better performance the quadrupling of DRAM
>> bandwidth provides
>
> That depends on the application. You can get some idea by looking at
> the 4-channel (3990X) vs. 8-channel (3995WX) results on
> <https://www.anandtech.com/print/16478/64-cores-of-rendering-madness-the-amd-threadripper-pro-3995wx-review>.
>
> And you can find an explicit 2-channel vs. 8-channel comparison in
> <https://www.anandtech.com/print/16482/lenovo-thinkstation-p620-review-a-vehicle-for-threadripper-pro>.
> In 25 out of 105 tests, the performance of the 8-channel configuration
> was more than a factor 1.03 compared to the 2-channel configuration.
>
> And note that the latter results were with 64-core CPUs with 51.2 GB/s
> or 204.8GB/s memory bandwidth, wheras the M2 Max has 12 (8P+4E) cores
> with (according to
> <https://www.apple.com/mz/newsroom/2023/06/apple-introduces-m2-ultra/>)
> 400GB/s memory bandwidth and the M2 Ultra has 24 (16P+8E) cores with
> 800GB/s memory bandwidth. I expect that, as far as CPU performance
> goes, both could do with a quarter of the bandwidth, and very few
> applications would see a measurable difference; things may be
> different for stuff that uses the GPU. If more CPU performance is
> desired and the application is parallelizable, there are good chances
> that a Threadripper 5995WX box (more than twice the cores) beats an M2
> Ultra box (four times the memory bandwidth).
>
>> but if it is at all significant, I would bet that
>> the average Mac customer would prefer better performance to DRAM size
>> upgradeability.
>
> Why? Are Macs slow?

I am not saying that, but my understanding is that a big market for Macs
is for graphic design/animation, etc. where memory bandwidth is often a
limiting factor, and faster is always better, and often noticeable.

> I expect that my collegue would have preferred
> to be able to upgrade the RAM of his laptop over having more memory
> bandwidth.

While I don't doubt that, from my understanding, your colleague may not
be representative of the "typical" Mac user. Of course YMMV, or perhaps
more accurately YCMMV. :-)

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 6/3/23 12:52 PM, MitchAlsup wrote:
[snip]
> On my GBOoO designs, the CAMs only look at 1 of the
> 6 tag-busses; yes all 6 tag-busses go to every CAM,
> but during INSERT the renamer tells the CAM which FU
> will produce the result, and so there is a 6-way mux from
> the tag-busses to the CAM, and the CAM only looks at a
> single bus.

It seems that for very large schedulers one might be able to use
something like set associativity where some scheduler entries only
listen to certain classes of results. (For smaller schedulers
"conflict misses" would excessively shrink utilization.) One might
include a "fully [or more highly] associative victim cache" to
reduce the impact of bad cases.

This would effectively be a hardwired filter/router.

(If each dynamic operand had a different scheduler entry and
operations with multiple dynamic operands somehow merged the ready
signals, such might reduce the number of CAMs needed as well as
possibly facilitate N-operand operations without a matrix
scheduler. I suspect such would have severe routing issues, but
perhaps constraints could ease those issues without losing most of
the advantages — if those advantages actually exist. Last arriving
prediction can reduce all operations [except perhaps a few that
are hard to predict] to one operand for scheduling, but adds
prediction/recovery overhead.)

Depending on rename-time FU choice would constrain scheduling.
While this would work well when each functional unit has a
dedicated scheduler, I have read that "unified" schedulers pair
execution ports in a scrambled/overlaid manner to provide the
utilization advantages of a unified scheduler with the routing
advantages of localized schedulers. For such a design, the
specific port would not be known at rename/insertion (though a
choice of only two is still easier than a fully general checking).
Even with such a design one could filter by the scheduler unit
rather than the execution port (FU).

(Variable latency also seems to introduce complexity. If a port
has operations that take one cycle or two cycles, two results
could be available from that port at the same time.)

(I wonder if one could use scheduler limited associativity with
something like overlaid skewed associativity to increase entry
utilization while providing advantages of limited associativity.
Managing routing/locality concerns seems challenging.)

In general, there is often information available earlier to help
filter decisions and specializations can often reduce resource
requirements without detracting excessively from the utilization
advantages of generality.

On Friday, June 9, 2023 at 7:13:06 PM UTC+3, Anton Ertl wrote:
> Michael S <already...@yahoo.com> writes:
> >So, you say, I can upgrade E-2176G based development server to
> >128 GB without losing bandwidth or latency?=20
> >By now, with 16GB DIMMs, it appears to operate at max rate (2666 MT/s),=20
> >but I am not sure about latency.
> I am just saying that you can upgrade it to 128GB. However, if you
> now use 4 two-sided 16GB DIMMs,

Yes, that's what we have now.

> I don't expect bandwidth or latency to
> get worse if you change to 4 32GB DIMMs. If you have only 2 DIMMs, or
> 4 one-sided DIMMs, you may see worse numbers, but for a development
> server I don't expect it to translate into a noticable slowdown.
>

Not software development. FPGA.
FPGA P&R tools appear to be quite sensitive to main DRAM latency. Probably,
also to DRAM bandwidth achievable by single core. But not sensitive to all cores
bandwidth. I.e. quite different, if not to say, opposite, to to compilation of large
SW project.

> We have Xeon E-2388G servers with 128GB here, and what dmidecode tells
> me about one of the DIMMs is:
>
> Speed: 3200 MT/s
> Configured Memory Speed: 2933 MT/s

E-2388G is Rocket Lake. Much newer than our Coffee Lake.
Better cores, but slower L2. Not sure how it effect FPGA tools.

> - anton

Thank you.
I didn't pay attention that 32 GB DDR4 UDIMMs finally materialized.

> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On Friday, June 9, 2023 at 8:00:34 PM UTC+3, Scott Lurndal wrote:
> an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> >Michael S <already...@yahoo.com> writes:
> >>So, you say, I can upgrade E-2176G based development server to
> >>128 GB without losing bandwidth or latency?=20
> >>By now, with 16GB DIMMs, it appears to operate at max rate (2666 MT/s),=20
> >>but I am not sure about latency.
> >
> >I am just saying that you can upgrade it to 128GB. However, if you
> >now use 4 two-sided 16GB DIMMs, I don't expect bandwidth or latency to
> >get worse if you change to 4 32GB DIMMs. If you have only 2 DIMMs, or
> >4 one-sided DIMMs, you may see worse numbers, but for a development
> >server I don't expect it to translate into a noticable slowdown.
> >
> >We have Xeon E-2388G servers with 128GB here, and what dmidecode tells
> >me about one of the DIMMs is:
> >
> > Speed: 3200 MT/s
> > Configured Memory Speed: 2933 MT/s
> Our development machine has 380GB, DDR4, all at the
> same speed as yours using 48 DIMMs.
>
> $ grep DIMM /tmp/dmi.decode | wc -l
> 48
> model name : Intel(R) Xeon(R) Gold 6246R CPU @ 3.40GHz

Those servers have much higher DRAM latency than Xeon E.
Also, according to Anandetch benchmarks (back from times when Andrei and
Ian still worked there) the bandwidth, available to individual core, is at best half
of what available on Xeon E.
Of course, for big SW compilation all that insignificant, what matters is mostly
total bandwidth on all cores and the size of LLC cache, both of which are a lot
better than on Xeon-E. But for FPGA development, our small inexpensive server
probably beats yours big and costly box by wide margin. The only problem is,
that 64 GB is not enough to run two Stratix-10 compilations in parallel.
And sometimes, not very often, we want to do just that.

On Saturday, June 10, 2023 at 3:18:52 PM UTC-5, Michael S wrote:
> On Friday, June 9, 2023 at 8:00:34 PM UTC+3, Scott Lurndal wrote:
> > an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> > >Michael S <already...@yahoo.com> writes:
> > >>So, you say, I can upgrade E-2176G based development server to
> > >>128 GB without losing bandwidth or latency?=20
> > >>By now, with 16GB DIMMs, it appears to operate at max rate (2666 MT/s),=20
> > >>but I am not sure about latency.
> > >
> > >I am just saying that you can upgrade it to 128GB. However, if you
> > >now use 4 two-sided 16GB DIMMs, I don't expect bandwidth or latency to
> > >get worse if you change to 4 32GB DIMMs. If you have only 2 DIMMs, or
> > >4 one-sided DIMMs, you may see worse numbers, but for a development
> > >server I don't expect it to translate into a noticable slowdown.
> > >
> > >We have Xeon E-2388G servers with 128GB here, and what dmidecode tells
> > >me about one of the DIMMs is:
> > >
> > > Speed: 3200 MT/s
> > > Configured Memory Speed: 2933 MT/s
> > Our development machine has 380GB, DDR4, all at the
> > same speed as yours using 48 DIMMs.
> >
> > $ grep DIMM /tmp/dmi.decode | wc -l
> > 48
> > model name : Intel(R) Xeon(R) Gold 6246R CPU @ 3.40GHz
<
> Those servers have much higher DRAM latency than Xeon E.
<
Servers benefit from closed page mode, too; whereas application processors
benefit from open page mode DRAM access. Lower aggregate latency favors
servers, lower minimum latency favors application processing.
<
Once you get away from:: "I want this thread to run as fast as possible"
and go to "I want all threads to run in aggregate as fast as possible",
lots of things want to be utilized and configured differently.
<
> Also, according to Anandetch benchmarks (back from times when Andrei and
> Ian still worked there) the bandwidth, available to individual core, is at best half
> of what available on Xeon E.
<
SUN 10,000 (64 processor) servers had 3× the latency of the 4 processor
servers, yet could perform database stuff more than 16× faster than the
4 processor servers. {Circa 2000}
<
> Of course, for big SW compilation all that insignificant, what matters is mostly
> total bandwidth on all cores and the size of LLC cache, both of which are a lot
> better than on Xeon-E. But for FPGA development, our small inexpensive server
> probably beats yours big and costly box by wide margin. The only problem is,
> that 64 GB is not enough to run two Stratix-10 compilations in parallel.
> And sometimes, not very often, we want to do just that.
<
That is what I am getting at:: at one end you are latency bound, at the other
you are throughput bound. I might note that GPUs often average (AVERAGE)
400 clocks to memory*, but with 10s of thousands of threads available, they
dramatically outperform non-WARP-like ISAs even with horribly slow (by CPU
standards) memory.
<
(*) cache hits included

On Sunday, June 11, 2023 at 4:18:54 AM UTC+3, MitchAlsup wrote:
> On Saturday, June 10, 2023 at 3:18:52 PM UTC-5, Michael S wrote:
> > On Friday, June 9, 2023 at 8:00:34 PM UTC+3, Scott Lurndal wrote:
> > > an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> > > >Michael S <already...@yahoo.com> writes:
> > > >>So, you say, I can upgrade E-2176G based development server to
> > > >>128 GB without losing bandwidth or latency?=20
> > > >>By now, with 16GB DIMMs, it appears to operate at max rate (2666 MT/s),=20
> > > >>but I am not sure about latency.
> > > >
> > > >I am just saying that you can upgrade it to 128GB. However, if you
> > > >now use 4 two-sided 16GB DIMMs, I don't expect bandwidth or latency to
> > > >get worse if you change to 4 32GB DIMMs. If you have only 2 DIMMs, or
> > > >4 one-sided DIMMs, you may see worse numbers, but for a development
> > > >server I don't expect it to translate into a noticable slowdown.
> > > >
> > > >We have Xeon E-2388G servers with 128GB here, and what dmidecode tells
> > > >me about one of the DIMMs is:
> > > >
> > > > Speed: 3200 MT/s
> > > > Configured Memory Speed: 2933 MT/s
> > > Our development machine has 380GB, DDR4, all at the
> > > same speed as yours using 48 DIMMs.
> > >
> > > $ grep DIMM /tmp/dmi.decode | wc -l
> > > 48
> > > model name : Intel(R) Xeon(R) GoldXeon(R) Gold 6246R CPU @ 3.40GHz
> <
> > Those servers have much higher DRAM latency than Xeon E.
> <
> Servers benefit from closed page mode, too; whereas application processors
> benefit from open page mode DRAM access. Lower aggregate latency favors
> servers, lower minimum latency favors application processing.

I am not so sure that it is as true for 3 y.o. Xeon 6246R as it was for 20 y.o.
DDR1-based Opteron. I mean, not sure that closed page mode is still beneficial.

DDR4 has 16 banks vs 2 in DDR1. The number of R-DIMMs per channel is a little
lower than in olden times (3 vs 4?), but today each server-grade DIMM has at least
2 ranks and sometimes 4, while back than it was 1 or sometimes 2. So # average
ranks per channel increased, too. And there are 6 logical channels instead of
one (it seems to me, those Opterons were running a pair of DIMMs in lockstep,
as a single logical channel, but may be I am confusing them with contemporary
Xeons ?).
On top of that, queues in the memory controllers are probably at least 8 times
deeper, but likely more than 8 times, so controller has much better visibility into
what is coming in the near future. All that makes "smart" open page policy more
attractive even in most serverish of server scenarios. Especially so for likes of
6246R that has modest # of cores (16).

When I say "smart" open page policy I mean that page is closed either when MC
sees that the next access to bank in question is directed to different row or when
request queue holds no accesses to this bank and the time to next refresh is below
judiciously chosen threshold. Otherwise page kept open.

Brett <ggtgp@yahoo.com> schrieb:

> With the M2 machines Apple has dumped DIMMs and brought RAM onto the CPU
> carrier, quadrupling the dram bandwidth.

I would _love_ to see a standard OpenFOAM benchmarks on these machines.

CFD is traditionally memory-bound; twice the memory bandwidth
translates to roughly twice the performance.

Even at the inflated prices of Apple hardware, having n/4 Apple
M2 machines in a cluster could actually be more economical than
having n x86 machines.

Something similar for POWER10, but I assume that the prices are
even more astronomical there.

On Sunday, June 11, 2023 at 12:42:06 PM UTC-5, Thomas Koenig wrote:
> Brett <gg...@yahoo.com> schrieb:
> > With the M2 machines Apple has dumped DIMMs and brought RAM onto the CPU
> > carrier, quadrupling the dram bandwidth.
> I would _love_ to see a standard OpenFOAM benchmarks on these machines.
>
> CFD is traditionally memory-bound; twice the memory bandwidth
> translates to roughly twice the performance.
<
True for 3D CFD not so true for 1D CFD.
<
When you have to touch 1M nodes of 64-bytes each × every iteration step,
memory bandwidth becomes the entire game.
<
Also FFT:: no mater how you break the thing up, at some point in the
calculation, every complex number gets multiplied by every other
complex number in the data set undergoing transformation. Nothing
caches can do for you here (except become as big as the data set....)
>
> Even at the inflated prices of Apple hardware, having n/4 Apple
> M2 machines in a cluster could actually be more economical than
> having n x86 machines.
>
> Something similar for POWER10, but I assume that the prices are
> even more astronomical there.