Vir Campestris <vir.campestris@invalid.invalid> wrote:
> This is not the right group for this - but I don't know where is.
> Suggestions on a postcard please...

I'm crossposting this to comp.arch, where they may have some ideas.

> For reasons I won't go into I've been writing some code to evaluate
> memory performance on my AMD Ryzen 5 3400G.
>
> It says in the stuff I've found that each core has an 8-way set
> associative L1 data cache of 128k (and an L1 instruction cache); an L2
> cache of 512k, also set associative; and there's an L3 cache of 4MB.
>
> To measure the performance I have three nested loops.
>
> The innermost one goes around a loop incrementing a series of 64 bit
> memory locations. The length of the series is set by the outermost loop.
>
> The middle one repeats the innermost loop so that the number of memory
> accesses is constant regardless of the series length.
>
> The outermost one sets the series length. It starts at 1, and doubles it
> each time.
>
> I _thought_ what would happen is that as I increase the length of the
> series after a while the data won't fit in the cache, and I'll see a
> sudden slowdown.
>
> What I actually see is:
>
> With a series length of 56 to 128 bytes I get the highest speed.
>
> With a series length of 500B to 1.5MB, I get a consistent speed of about
> 2/3 the highest speed.
>
> Once the series length exceeds 1.5MB the speed drops, and is consistent
> from then on. That I can see is main memory speed, and is about 40% of
> the highest.
>
> OK so far.
>
> But...
> Series length 8B is about the same as the 56 to 128 speed. Series length
> 16B is a bit less. Series length 32 is a lot less. Not as slow as main
> memory, but not much more than half the peak speed. My next step up is
> the peak speed. Series length 144 to 448 is slower still - slower in
> fact than the main memory speed.
>
> WTF?
>
> I can post the code (C++, but not very complex) if that would help.

For 'series length 8B/16B/32B' do you mean 8 bytes? ie 8B is a single 64
bit word transferred?

What instruction sequences are being generated for the 8/16/32/64 byte
loops? I'm wondering if the compiler is using different instructions,
eg using MMX, SSE, AVX to do the operations. Maybe they are having
different caching behaviour?

It would help if you could tell us the compiler and platform you're using,
including version.

Theo

Theo wrote:

> Vir Campestris <vir.campestris@invalid.invalid> wrote:
>> This is not the right group for this - but I don't know where is.
>> Suggestions on a postcard please...

> I'm crossposting this to comp.arch, where they may have some ideas.

>> For reasons I won't go into I've been writing some code to evaluate
>> memory performance on my AMD Ryzen 5 3400G.
>>
>> It says in the stuff I've found that each core has an 8-way set
>> associative L1 data cache of 128k (and an L1 instruction cache); an L2
>> cache of 512k, also set associative; and there's an L3 cache of 4MB.
>>
>> To measure the performance I have three nested loops.
>>
>> The innermost one goes around a loop incrementing a series of 64 bit
>> memory locations. The length of the series is set by the outermost loop.
>>
>> The middle one repeats the innermost loop so that the number of memory
>> accesses is constant regardless of the series length.
>>
>> The outermost one sets the series length. It starts at 1, and doubles it
>> each time.
>>
>> I _thought_ what would happen is that as I increase the length of the
>> series after a while the data won't fit in the cache, and I'll see a
>> sudden slowdown.
>>
>> What I actually see is:
>>
>> With a series length of 56 to 128 bytes I get the highest speed.
>>
>> With a series length of 500B to 1.5MB, I get a consistent speed of about
>> 2/3 the highest speed.
>>
>> Once the series length exceeds 1.5MB the speed drops, and is consistent
>> from then on. That I can see is main memory speed, and is about 40% of
>> the highest.
>>
>> OK so far.
>>
>> But...
>> Series length 8B is about the same as the 56 to 128 speed. Series length
>> 16B is a bit less. Series length 32 is a lot less. Not as slow as main
>> memory, but not much more than half the peak speed. My next step up is
>> the peak speed. Series length 144 to 448 is slower still - slower in
>> fact than the main memory speed.
>>
>> WTF?
>>
>> I can post the code (C++, but not very complex) if that would help.

> For 'series length 8B/16B/32B' do you mean 8 bytes? ie 8B is a single 64
> bit word transferred?

> What instruction sequences are being generated for the 8/16/32/64 byte
> loops? I'm wondering if the compiler is using different instructions,
> eg using MMX, SSE, AVX to do the operations. Maybe they are having
> different caching behaviour?

> It would help if you could tell us the compiler and platform you're using,
> including version.

> Theo

Can we see the code ??

Can you present a table of the timing results ??

On 20/12/2023 18:08, Theo wrote:
> Vir Campestris <vir.campestris@invalid.invalid> wrote:
>> This is not the right group for this - but I don't know where is.
>> Suggestions on a postcard please...
>
> I'm crossposting this to comp.arch, where they may have some ideas.
>
<snip>
>
> For 'series length 8B/16B/32B' do you mean 8 bytes? ie 8B is a single 64
> bit word transferred?
>
Yes. My system has a 64 bit CPU and 64 bit main memory.

> What instruction sequences are being generated for the 8/16/32/64 byte
> loops? I'm wondering if the compiler is using different instructions,
> eg using MMX, SSE, AVX to do the operations. Maybe they are having
> different caching behaviour?
>
It's running the same loop for each time, but with different values for
the loop sizes.

> It would help if you could tell us the compiler and platform you're using,
> including version.
>

g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Which of course tells you I'm running Ubuntu!

On 20/12/2023 18:58, MitchAlsup wrote:
>
> Can we see the code ??
>
> Can you present a table of the timing results ??

I've run this with more detailed increments on the line size, but here
are my results for powers of 2.

Size 1 gave 3.82242e+09 bytes/second.
Size 2 gave 3.80533e+09 bytes/second.
Size 4 gave 2.68017e+09 bytes/second.
Size 8 gave 2.33751e+09 bytes/second.
Size 16 gave 2.18424e+09 bytes/second.
Size 32 gave 2.10243e+09 bytes/second.
Size 64 gave 1.99371e+09 bytes/second.
Size 128 gave 1.98475e+09 bytes/second.
Size 256 gave 2.01653e+09 bytes/second.
Size 512 gave 2.00884e+09 bytes/second.
Size 1024 gave 2.02713e+09 bytes/second.
Size 2048 gave 2.01803e+09 bytes/second.
Size 4096 gave 3.26472e+09 bytes/second.
Size 8192 gave 3.85126e+09 bytes/second.
Size 16384 gave 3.85377e+09 bytes/second.
Size 32768 gave 3.85293e+09 bytes/second.
Size 65536 gave 2.06793e+09 bytes/second.
Size 131072 gave 2.06845e+09 bytes/second.

The code will continue, but the results are roughly stable for larger sizes.

The code I have put in a signature block; there's no point in risking
someone posting it again. I've commented it, but no doubt not in all the
right places! I'd be interested to know what results other people get.

Thanks
Andy
--
#include <chrono>
#include <iostream>
#include <vector>

int main()
{ // If your computer is much slower or faster than mine
// you may need to adjust this value.
constexpr size_t NextCount = 1 << 28;

std::vector<uint64_t> CacheStore(NextCount, 0);

// Get a raw pointer to the vector.
// On my machine (Ubuntu, g++) this improves
// performance. Using vector's operator[]
// results in a function call.
uint64_t *CachePtr = &CacheStore[0];

// SetSize is the count of the uint64_t items to be tested.
// I assume that when this is too big for a cache the data
// will overflow to the next level.
// Each loop it doubles in size. I've run it with smaller
// increments too, and the behaviour
// is still confusing.
for (auto SetSize = 1; SetSize < NextCount; SetSize<<=1)
{
size_t loopcount = 0;
size_t j = NextCount / SetSize;
auto start = std::chrono::steady_clock::now();

// The outer loop repeats enough times so that the data
// written by the inner loops of various sizes is
// approximately constant.
for (size_t k = 0; k < j; ++k)
{
// The inner loop modifies data
// within a set of words.
for (size_t l = 0; l < SetSize; ++l)
{
// read-modify-write some data.
// Comment this out
// to confirm that the looping is not
// the cause of the anomaly
++CachePtr[l];

// this counts the actual number
// of memory accesses.
// rounding errors means that for
// different SetSize values
// the count is not completely
// consistent.
++loopcount;
}
}

// Work out how long the loops took in microseconds,
// then scale to seconds
auto delta =
std::chrono::duration_cast<std::chrono::microseconds>
(std::chrono::steady_clock::now() - start).count()
/ 1e6;

// calculate how many bytes per second, and print.
std::cout << "Size " << SetSize << " gave "
<< (double)loopcount * (double)sizeof(uint64_t) /
delta << " bytes/second." << std::endl;
}

return 0;
}

Vir Campestris <vir.campestris@invalid.invalid> writes:
>On 20/12/2023 18:08, Theo wrote:
>> Vir Campestris <vir.campestris@invalid.invalid> wrote:
>>> This is not the right group for this - but I don't know where is.
>>> Suggestions on a postcard please...
>>
>> I'm crossposting this to comp.arch, where they may have some ideas.
>>
><snip>
>>
>> For 'series length 8B/16B/32B' do you mean 8 bytes? ie 8B is a single 64
>> bit word transferred?
>>
>Yes. My system has a 64 bit CPU and 64 bit main memory.

Surely your main memory is just a sequence of 8-bit bytes (+ECC).

On 21/12/2023 14:56, Scott Lurndal wrote:
> Vir Campestris <vir.campestris@invalid.invalid> writes:
>> On 20/12/2023 18:08, Theo wrote:
>>> Vir Campestris <vir.campestris@invalid.invalid> wrote:
>>>> This is not the right group for this - but I don't know where is.
>>>> Suggestions on a postcard please...
>>>
>>> I'm crossposting this to comp.arch, where they may have some ideas.
>>>
>> <snip>
>>>
>>> For 'series length 8B/16B/32B' do you mean 8 bytes? ie 8B is a single 64
>>> bit word transferred?
>>>
>> Yes. My system has a 64 bit CPU and 64 bit main memory.
>
> Surely your main memory is just a sequence of 8-bit bytes (+ECC).
>

AIUI I have two DIMMs, each of which has a 32-bit bus. I'm not up on
hardware these days, but it used to be if you wanted to write a byte
into your 16-bit memory you had to read both bytes, then write one back.

And

On Thu, 21 Dec 2023 15:32:10 +0000
Vir Campestris <vir.campestris@invalid.invalid> wrote:

> On 21/12/2023 14:56, Scott Lurndal wrote:
> > Vir Campestris <vir.campestris@invalid.invalid> writes:
> >> On 20/12/2023 18:08, Theo wrote:
> >>> Vir Campestris <vir.campestris@invalid.invalid> wrote:
> >>>> This is not the right group for this - but I don't know where is.
> >>>> Suggestions on a postcard please...
> >>>
> >>> I'm crossposting this to comp.arch, where they may have some
> >>> ideas.
> >> <snip>
> >>>
> >>> For 'series length 8B/16B/32B' do you mean 8 bytes? ie 8B is a
> >>> single 64 bit word transferred?
> >>>
> >> Yes. My system has a 64 bit CPU and 64 bit main memory.
> >
> > Surely your main memory is just a sequence of 8-bit bytes (+ECC).
> >
>
> AIUI I have two DIMMs, each of which has a 32-bit bus. I'm not up on
> hardware these days, but it used to be if you wanted to write a byte
> into your 16-bit memory you had to read both bytes, then write one
> back.
>
> And

DIMMs have 64-bit data buses. Both these days and previous millennium.
Now, these days DDR5 DIMM splits 64-bit data bus into pair of
independent 32-bit channels, but the total is still 64 bits.

That's a physical perspective. From logical perspective, DDR3 and DDR4
bits exchange data with controller in 512-bit chunks. On DDR5 DIMM each
channel exchanges data with controller in 512-bit chunks.

From signaling perspective, it is still possible (at least on non-ECC
gear) to tell to memory to write just 8 bits out of 512 and to ignore
the rest. In PC-class hardware this ability is used very rarely if used
at all.

Vir Campestris wrote:

> Scott Lurndal wrote:
>
>> Surely your main memory is just a sequence of 8-bit bytes (+ECC).
>
> AIUI I have two DIMMs, each of which has a 32-bit bus. I'm not up on
> hardware these days, but it used to be if you wanted to write a byte
> into your 16-bit memory you had to read both bytes, then write one back.

I thought intel x64 machines work in cache lines of 64 bytes per memory
transaction ... maybe AMD processors are different?

Vir Campestris wrote:

> On 20/12/2023 18:58, MitchAlsup wrote:
> >
> > Can we see the code ??
> >
> > Can you present a table of the timing results ??

> I've run this with more detailed increments on the line size, but here
> are my results for powers of 2.

{ A
> Size 1 gave 3.82242e+09 bytes/second.
> Size 2 gave 3.80533e+09 bytes/second.
> Size 4 gave 2.68017e+09 bytes/second.
> Size 8 gave 2.33751e+09 bytes/second.
> Size 16 gave 2.18424e+09 bytes/second.
> Size 32 gave 2.10243e+09 bytes/second.
> Size 64 gave 1.99371e+09 bytes/second.
> Size 128 gave 1.98475e+09 bytes/second.
> Size 256 gave 2.01653e+09 bytes/second.
> Size 512 gave 2.00884e+09 bytes/second.
> Size 1024 gave 2.02713e+09 bytes/second.
> Size 2048 gave 2.01803e+09 bytes/second.
} A
{ B
> Size 4096 gave 3.26472e+09 bytes/second.
> Size 8192 gave 3.85126e+09 bytes/second.
> Size 16384 gave 3.85377e+09 bytes/second.
> Size 32768 gave 3.85293e+09 bytes/second.
> Size 65536 gave 2.06793e+09 bytes/second.
> Size 131072 gave 2.06845e+09 bytes/second.
} B

A) Here we have a classical sequence pipelines often encounter where
a simple loop starts off fast 4 bytes/cycle and progressively slows
down by a factor of 2 (2 bytes per cycle) over some interval of
complexity (size). What is important is that factor of 2 something
that took 1 cycle early starts taking 2 cycles later on.

B) here we have a second classical sequence where the performance at
some boundary (4096) reverts back to the 1 cycle pipeline of performance
only to degrade again (in basically the same sequence as A). {{Side
note: size=4096 has "flutter" in the stairstep. size={8192..32768}
has peak performance--probably something to do with sets in the cache
and 4096 is the size of a page (TLB effects)}}.
I suspect the flutter has something to do with your buffer crossing
a page boundary.

> The code will continue, but the results are roughly stable for larger sizes.

> The code I have put in a signature block; there's no point in risking
> someone posting it again. I've commented it, but no doubt not in all the
> right places! I'd be interested to know what results other people get.

> Thanks
> Andy

Michael S wrote:

> On Thu, 21 Dec 2023 15:32:10 +0000
> Vir Campestris <vir.campestris@invalid.invalid> wrote:

>> On 21/12/2023 14:56, Scott Lurndal wrote:
>> > Vir Campestris <vir.campestris@invalid.invalid> writes:
>> >> On 20/12/2023 18:08, Theo wrote:
>> >>> Vir Campestris <vir.campestris@invalid.invalid> wrote:
>> >>>> This is not the right group for this - but I don't know where is.
>> >>>> Suggestions on a postcard please...
>> >>>
>> >>> I'm crossposting this to comp.arch, where they may have some
>> >>> ideas.
>> >> <snip>
>> >>>
>> >>> For 'series length 8B/16B/32B' do you mean 8 bytes? ie 8B is a
>> >>> single 64 bit word transferred?
>> >>>
>> >> Yes. My system has a 64 bit CPU and 64 bit main memory.
>> >
>> > Surely your main memory is just a sequence of 8-bit bytes (+ECC).
>> >
>>
>> AIUI I have two DIMMs, each of which has a 32-bit bus. I'm not up on
>> hardware these days, but it used to be if you wanted to write a byte
>> into your 16-bit memory you had to read both bytes, then write one
>> back.
>>
>> And

> DIMMs have 64-bit data buses. Both these days and previous millennium.
> Now, these days DDR5 DIMM splits 64-bit data bus into pair of
> independent 32-bit channels, but the total is still 64 bits.

All DDR DIMMs have 64-bit busses (200 pins on the DIMM).
Some SDR (pre 2000) DIMMs had 32-bit busses, some ancient laptop memory
carriers had 32-bit busses.

> That's a physical perspective. From logical perspective, DDR3 and DDR4
> bits exchange data with controller in 512-bit chunks. On DDR5 DIMM each
> channel exchanges data with controller in 512-bit chunks.

Note:: 512-bis is 64-bytes.

> From signaling perspective, it is still possible (at least on non-ECC
> gear) to tell to memory to write just 8 bits out of 512 and to ignore
> the rest. In PC-class hardware this ability is used very rarely if used
> at all.

Never justified when CPU uses a cache. So, only unCacheable (sized)
requests can use this. AMD memory controllers hid this from the DRAMs
so we at least had the opportunity to recompute ECC on ECC carrying
DIMMs. MC would ask DRC for the line of data, check (and repair) ECC
then write the unCacheable data, and place the written data in the
outbound memory queue with its new ECC.

On 12/21/2023 10:05 AM, Andy Burns wrote:
> Vir Campestris wrote:
>
>> Scott Lurndal wrote:
>>
>>> Surely your main memory is just a sequence of 8-bit bytes (+ECC).
>>
>> AIUI I have two DIMMs, each of which has a 32-bit bus. I'm not up on
>> hardware these days, but it used to be if you wanted to write a byte
>> into your 16-bit memory you had to read both bytes, then write one back.
>
> I thought intel x64 machines work in cache lines of 64 bytes per memory
> transaction ... maybe AMD processors are different?
>
>
>

Remember when Intel had a false sharing problem when they had 128 cache
lines split into two 64 regions? Iirc, it was on some of their first
hyperthreaded processors. A work around from intel was to offset threads
using alloca... I remember it.

On Thu, 21 Dec 2023 18:36:42 +0000
mitchalsup@aol.com (MitchAlsup) wrote:

> Michael S wrote:
>
> > On Thu, 21 Dec 2023 15:32:10 +0000
> > Vir Campestris <vir.campestris@invalid.invalid> wrote:
>
> >> On 21/12/2023 14:56, Scott Lurndal wrote:
> >> > Vir Campestris <vir.campestris@invalid.invalid> writes:
> >> >> On 20/12/2023 18:08, Theo wrote:
> >> >>> Vir Campestris <vir.campestris@invalid.invalid> wrote:
> >> >>>> This is not the right group for this - but I don't know where
> >> >>>> is. Suggestions on a postcard please...
> >> >>>
> >> >>> I'm crossposting this to comp.arch, where they may have some
> >> >>> ideas.
> >> >> <snip>
> >> >>>
> >> >>> For 'series length 8B/16B/32B' do you mean 8 bytes? ie 8B is a
> >> >>> single 64 bit word transferred?
> >> >>>
> >> >> Yes. My system has a 64 bit CPU and 64 bit main memory.
> >> >
> >> > Surely your main memory is just a sequence of 8-bit bytes (+ECC).
> >> >
> >>
> >> AIUI I have two DIMMs, each of which has a 32-bit bus. I'm not up
> >> on hardware these days, but it used to be if you wanted to write a
> >> byte into your 16-bit memory you had to read both bytes, then
> >> write one back.
> >>
> >> And
>
> > DIMMs have 64-bit data buses. Both these days and previous
> > millennium. Now, these days DDR5 DIMM splits 64-bit data bus into
> > pair of independent 32-bit channels, but the total is still 64
> > bits.
>
> All DDR DIMMs have 64-bit busses (200 pins on the DIMM).
> Some SDR (pre 2000) DIMMs had 32-bit busses,

Are you sure? My impression always was that the word DIMM was invented
to clearly separate 64-bit DIMMs from 32-bit SIMMs.

> some ancient laptop
> memory carriers had 32-bit busses.

But were they called just DIMM or SO-DIMM?
The later indeed had 72-pin variant with 32-bit bus.

>
> > That's a physical perspective. From logical perspective, DDR3 and
> > DDR4 bits exchange data with controller in 512-bit chunks. On DDR5
> > DIMM each channel exchanges data with controller in 512-bit chunks.
> >
>
> Note:: 512-bis is 64-bytes.
>

Which *not* co-incidentally matches cache line size of majority of x86
CPUs.

> > From signaling perspective, it is still possible (at least on
> > non-ECC gear) to tell to memory to write just 8 bits out of 512 and
> > to ignore the rest. In PC-class hardware this ability is used very
> > rarely if used at all.
>
> Never justified when CPU uses a cache. So, only unCacheable (sized)
> requests can use this. AMD memory controllers hid this from the DRAMs
> so we at least had the opportunity to recompute ECC on ECC carrying
> DIMMs. MC would ask DRC for the line of data, check (and repair) ECC
> then write the unCacheable data, and place the written data in the
> outbound memory queue with its new ECC.

Michael S wrote:

> On Thu, 21 Dec 2023 18:36:42 +0000
> mitchalsup@aol.com (MitchAlsup) wrote:

>> Michael S wrote:
>>
>> > On Thu, 21 Dec 2023 15:32:10 +0000
>> > Vir Campestris <vir.campestris@invalid.invalid> wrote:
>>
>> >> On 21/12/2023 14:56, Scott Lurndal wrote:
>> >> > Vir Campestris <vir.campestris@invalid.invalid> writes:
>> >> >> On 20/12/2023 18:08, Theo wrote:
>> >> >>> Vir Campestris <vir.campestris@invalid.invalid> wrote:
>> >> >>>> This is not the right group for this - but I don't know where
>> >> >>>> is. Suggestions on a postcard please...
>> >> >>>
>> >> >>> I'm crossposting this to comp.arch, where they may have some
>> >> >>> ideas.
>> >> >> <snip>
>> >> >>>
>> >> >>> For 'series length 8B/16B/32B' do you mean 8 bytes? ie 8B is a
>> >> >>> single 64 bit word transferred?
>> >> >>>
>> >> >> Yes. My system has a 64 bit CPU and 64 bit main memory.
>> >> >
>> >> > Surely your main memory is just a sequence of 8-bit bytes (+ECC).
>> >> >
>> >>
>> >> AIUI I have two DIMMs, each of which has a 32-bit bus. I'm not up
>> >> on hardware these days, but it used to be if you wanted to write a
>> >> byte into your 16-bit memory you had to read both bytes, then
>> >> write one back.
>> >>
>> >> And
>>
>> > DIMMs have 64-bit data buses. Both these days and previous
>> > millennium. Now, these days DDR5 DIMM splits 64-bit data bus into
>> > pair of independent 32-bit channels, but the total is still 64
>> > bits.
>>
>> All DDR DIMMs have 64-bit busses (200 pins on the DIMM).
>> Some SDR (pre 2000) DIMMs had 32-bit busses,

> Are you sure? My impression always was that the word DIMM was invented
> to clearly separate 64-bit DIMMs from 32-bit SIMMs.

Dual In-Line Memory Module means they have pins on both sides of the
board where pins make contact with the plug. They put pins on both
sides so they could get ~200 pins {Vdd, Gnd, lock, reset, control,
and data} this was in the early 1990s.

>> some ancient laptop
>> memory carriers had 32-bit busses.

> But were they called just DIMM or SO-DIMM?
> The later indeed had 72-pin variant with 32-bit bus.

That sounds right.

>>
>> > That's a physical perspective. From logical perspective, DDR3 and
>> > DDR4 bits exchange data with controller in 512-bit chunks. On DDR5
>> > DIMM each channel exchanges data with controller in 512-bit chunks.
>> >
>>
>> Note:: 512-bis is 64-bytes.
>>

> Which *not* co-incidentally matches cache line size of majority of x86
> CPUs.

Given that x86 at the time the Std committee was doing the first one
represented 90% of all computers being sold, and the people on the
committee wanting to keep it that way, is unsurprising.

>> > From signaling perspective, it is still possible (at least on
>> > non-ECC gear) to tell to memory to write just 8 bits out of 512 and
>> > to ignore the rest. In PC-class hardware this ability is used very
>> > rarely if used at all.
>>
>> Never justified when CPU uses a cache. So, only unCacheable (sized)
>> requests can use this. AMD memory controllers hid this from the DRAMs
>> so we at least had the opportunity to recompute ECC on ECC carrying
>> DIMMs. MC would ask DRC for the line of data, check (and repair) ECC
>> then write the unCacheable data, and place the written data in the
>> outbound memory queue with its new ECC.

On 12/21/2023 1:20 PM, Chris M. Thomasson wrote:
> On 12/21/2023 10:05 AM, Andy Burns wrote:
>> Vir Campestris wrote:
>>
>>> Scott Lurndal wrote:
>>>
>>>> Surely your main memory is just a sequence of 8-bit bytes (+ECC).
>>>
>>> AIUI I have two DIMMs, each of which has a 32-bit bus. I'm not up on
>>> hardware these days, but it used to be if you wanted to write a byte
>>> into your 16-bit memory you had to read both bytes, then write one back.
>>
>> I thought intel x64 machines work in cache lines of 64 bytes per
>> memory transaction ... maybe AMD processors are different?
>>
>>
>>
>
> Remember when Intel had a false sharing problem when they had 128 cache
> lines split into two 64 regions? Iirc, it was on some of their first
> hyperthreaded processors. A work around from intel was to offset threads
> using alloca... I remember it.

It was a nightmare, however, at least the workaround did help a bit wrt
performance.

On 21/12/2023 18:30, MitchAlsup wrote:
>> I've run this with more detailed increments on the line size, but here
>> are my results for powers of 2.
>
> { A
>> Size 1 gave 3.82242e+09 bytes/second.
>> Size 2 gave 3.80533e+09 bytes/second.
>> Size 4 gave 2.68017e+09 bytes/second.
>> Size 8 gave 2.33751e+09 bytes/second.
>> Size 16 gave 2.18424e+09 bytes/second.
>> Size 32 gave 2.10243e+09 bytes/second.
>> Size 64 gave 1.99371e+09 bytes/second.
>> Size 128 gave 1.98475e+09 bytes/second.
>> Size 256 gave 2.01653e+09 bytes/second.
>> Size 512 gave 2.00884e+09 bytes/second.
>> Size 1024 gave 2.02713e+09 bytes/second.
>> Size 2048 gave 2.01803e+09 bytes/second.
> } A
> { B
>> Size 4096 gave 3.26472e+09 bytes/second.
>> Size 8192 gave 3.85126e+09 bytes/second.
>> Size 16384 gave 3.85377e+09 bytes/second.
>> Size 32768 gave 3.85293e+09 bytes/second.
>> Size 65536 gave 2.06793e+09 bytes/second.
>> Size 131072 gave 2.06845e+09 bytes/second.
> } B
>
> A) Here we have a classical sequence pipelines often encounter where
> a simple loop starts off fast 4 bytes/cycle and progressively slows
> down by a factor of 2 (2 bytes per cycle) over some interval of
> complexity (size). What is important is that factor of 2 something
> that took 1 cycle early starts taking 2 cycles later on.
>
> B) here we have a second classical sequence where the performance at
> some boundary (4096) reverts back to the 1 cycle pipeline of performance
> only to degrade again (in basically the same sequence as A). {{Side
> note: size=4096 has "flutter" in the stairstep. size={8192..32768}
> has peak performance--probably something to do with sets in the cache
> and 4096 is the size of a page (TLB effects)}}.
> I suspect the flutter has something to do with your buffer crossing a
> page boundary.

I ran it with the memory access commented out. When I do that the
overall time is consistent regardless of how many times it goes around
the inner loop, and how many times the outer one.

But pipelines are funny things.

64k x 64 bit words is I think my L3 cache size. It's not very big on my
CPU, they had to fit a GPU on the die.

I'd be interested to see the results from anyone else's computer.

Andy

In comp.arch Vir Campestris <vir.campestris@invalid.invalid> wrote:
> I'd be interested to see the results from anyone else's computer.

$ neofetch --off
OS: Kubuntu 23.10 x86_64
Host: 20QBS03N00 ThinkPad X1 Titanium Gen 1
Kernel: 6.5.0-14-generic
CPU: 11th Gen Intel i5-1130G7 (8) @ 4.000GHz
GPU: Intel Tiger Lake-UP4 GT2 [Iris Xe Graphics]
Memory: 4436MiB / 15704MiB
$ gcc --version
gcc version 13.2.0 (Ubuntu 13.2.0-4ubuntu3)

$ g++ -o cache cache.c
$ ./cache
Size 1 gave 4.13643e+09 bytes/second.
Size 2 gave 4.79971e+09 bytes/second.
Size 4 gave 4.87535e+09 bytes/second.
Size 8 gave 4.8321e+09 bytes/second.
Size 16 gave 4.71703e+09 bytes/second.
Size 32 gave 3.89488e+09 bytes/second.
Size 64 gave 4.02976e+09 bytes/second.
Size 128 gave 4.15832e+09 bytes/second.
Size 256 gave 4.19562e+09 bytes/second.
Size 512 gave 4.08511e+09 bytes/second.
Size 1024 gave 4.0796e+09 bytes/second.
Size 2048 gave 4.11983e+09 bytes/second.
Size 4096 gave 4.06869e+09 bytes/second.
Size 8192 gave 4.06807e+09 bytes/second.
Size 16384 gave 4.06217e+09 bytes/second.
Size 32768 gave 4.06067e+09 bytes/second.
Size 65536 gave 4.04791e+09 bytes/second.
Size 131072 gave 4.06143e+09 bytes/second.
Size 262144 gave 4.04301e+09 bytes/second.
Size 524288 gave 4.03872e+09 bytes/second.
Size 1048576 gave 3.97715e+09 bytes/second.
Size 2097152 gave 3.97609e+09 bytes/second.
Size 4194304 gave 3.98361e+09 bytes/second.
Size 8388608 gave 3.98617e+09 bytes/second.
Size 16777216 gave 3.98376e+09 bytes/second.
Size 33554432 gave 3.98504e+09 bytes/second.
Size 67108864 gave 3.98726e+09 bytes/second.
Size 134217728 gave 3.99495e+09 bytes/second.

Theo wrote:
> $ g++ -o cache cache.c
> $ ./cache
{ A
> Size 1 gave 4.13643e+09 bytes/second.
> Size 2 gave 4.79971e+09 bytes/second.
> Size 4 gave 4.87535e+09 bytes/second.
> Size 8 gave 4.8321e+09 bytes/second.
> Size 16 gave 4.71703e+09 bytes/second.
} A
{ B
> Size 32 gave 3.89488e+09 bytes/second.
> Size 64 gave 4.02976e+09 bytes/second.
> Size 128 gave 4.15832e+09 bytes/second.
> Size 256 gave 4.19562e+09 bytes/second.
> Size 512 gave 4.08511e+09 bytes/second.
> Size 1024 gave 4.0796e+09 bytes/second.
> Size 2048 gave 4.11983e+09 bytes/second.
> Size 4096 gave 4.06869e+09 bytes/second.
> Size 8192 gave 4.06807e+09 bytes/second.
> Size 16384 gave 4.06217e+09 bytes/second.
> Size 32768 gave 4.06067e+09 bytes/second.
> Size 65536 gave 4.04791e+09 bytes/second.
> Size 131072 gave 4.06143e+09 bytes/second.
> Size 262144 gave 4.04301e+09 bytes/second.
> Size 524288 gave 4.03872e+09 bytes/second.
> Size 1048576 gave 3.97715e+09 bytes/second.
> Size 2097152 gave 3.97609e+09 bytes/second.
> Size 4194304 gave 3.98361e+09 bytes/second.
> Size 8388608 gave 3.98617e+09 bytes/second.
> Size 16777216 gave 3.98376e+09 bytes/second.
> Size 33554432 gave 3.98504e+09 bytes/second.
> Size 67108864 gave 3.98726e+09 bytes/second.
> Size 134217728 gave 3.99495e+09 bytes/second.
} B

The B group are essentially 4.0 with noise.
The A group are essentially 4.8 with a startup flutter.
A 20% down or 25% up change at 32::64.

In comp.arch MitchAlsup <mitchalsup@aol.com> wrote:
> Theo wrote:
> > $ g++ -o cache cache.c
> > $ ./cache
> { A
> > Size 1 gave 4.13643e+09 bytes/second.
> > Size 2 gave 4.79971e+09 bytes/second.
> > Size 4 gave 4.87535e+09 bytes/second.
> > Size 8 gave 4.8321e+09 bytes/second.
> > Size 16 gave 4.71703e+09 bytes/second.
> } A
> { B
> > Size 32 gave 3.89488e+09 bytes/second.
....
> > Size 134217728 gave 3.99495e+09 bytes/second.
> } B
>
> The B group are essentially 4.0 with noise.
> The A group are essentially 4.8 with a startup flutter.
> A 20% down or 25% up change at 32::64.

The nearest machine I have to the OP is this:

OS: Ubuntu 20.04.6 LTS x86_64
Host: 1U4LW-X570/2L2T
Kernel: 5.4.0-167-generic
CPU: AMD Ryzen 7 5800X (16) @ 3.800GHz
GPU: 29:00.0 ASPEED Technology, Inc. ASPEED Graphics Family
Memory: 3332MiB / 128805MiB

gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0

Size 1 gave 6.22043e+09 bytes/second.
Size 2 gave 6.35674e+09 bytes/second.
Size 4 gave 4.14766e+09 bytes/second.
Size 8 gave 5.4239e+09 bytes/second.
Size 16 gave 6.01113e+09 bytes/second.
Size 32 gave 7.75976e+09 bytes/second.
Size 64 gave 8.7972e+09 bytes/second.
Size 128 gave 9.71523e+09 bytes/second.
Size 256 gave 9.91644e+09 bytes/second.
Size 512 gave 1.00179e+10 bytes/second.
Size 1024 gave 1.0065e+10 bytes/second.
Size 2048 gave 9.78508e+09 bytes/second.
Size 4096 gave 9.76764e+09 bytes/second.
Size 8192 gave 9.86537e+09 bytes/second.
Size 16384 gave 9.90053e+09 bytes/second.
Size 32768 gave 9.91552e+09 bytes/second.
Size 65536 gave 9.84556e+09 bytes/second.
Size 131072 gave 9.78442e+09 bytes/second.
Size 262144 gave 9.80282e+09 bytes/second.
Size 524288 gave 9.81447e+09 bytes/second.
Size 1048576 gave 9.81981e+09 bytes/second.
Size 2097152 gave 9.81456e+09 bytes/second.
Size 4194304 gave 9.70057e+09 bytes/second.
Size 8388608 gave 9.55507e+09 bytes/second.
Size 16777216 gave 9.44032e+09 bytes/second.
Size 33554432 gave 9.33896e+09 bytes/second.
Size 67108864 gave 9.28529e+09 bytes/second.
Size 134217728 gave 9.25213e+09 bytes/second.

which seems to have more flutter in group A. I get similar results in a VM
running on a AMD Ryzen 9 5950X (32) @ 3.393GHz.

Theo

Theo wrote:

> In comp.arch MitchAlsup <mitchalsup@aol.com> wrote:
>>

> gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0

{ A
> Size 1 gave 6.22043e+09 bytes/second.
> Size 2 gave 6.35674e+09 bytes/second.
> Size 4 gave 4.14766e+09 bytes/second.
> Size 8 gave 5.4239e+09 bytes/second.
> Size 16 gave 6.01113e+09 bytes/second.
> Size 32 gave 7.75976e+09 bytes/second.
> Size 64 gave 8.7972e+09 bytes/second.
} A
{ B
> Size 128 gave 9.71523e+09 bytes/second.
> Size 256 gave 9.91644e+09 bytes/second.
> Size 512 gave 1.00179e+10 bytes/second.
> Size 1024 gave 1.0065e+10 bytes/second.
> Size 2048 gave 9.78508e+09 bytes/second.
> Size 4096 gave 9.76764e+09 bytes/second.
> Size 8192 gave 9.86537e+09 bytes/second.
> Size 16384 gave 9.90053e+09 bytes/second.
> Size 32768 gave 9.91552e+09 bytes/second.
> Size 65536 gave 9.84556e+09 bytes/second.
> Size 131072 gave 9.78442e+09 bytes/second.
> Size 262144 gave 9.80282e+09 bytes/second.
> Size 524288 gave 9.81447e+09 bytes/second.
> Size 1048576 gave 9.81981e+09 bytes/second.
> Size 2097152 gave 9.81456e+09 bytes/second.
} B
{ C
> Size 4194304 gave 9.70057e+09 bytes/second.
> Size 8388608 gave 9.55507e+09 bytes/second.
> Size 16777216 gave 9.44032e+09 bytes/second.
> Size 33554432 gave 9.33896e+09 bytes/second.
> Size 67108864 gave 9.28529e+09 bytes/second.
> Size 134217728 gave 9.25213e+09 bytes/second.
} C

> which seems to have more flutter in group A. I get similar results in a VM
> running on a AMD Ryzen 9 5950X (32) @ 3.393GHz.

A has some kind of startup irregularity: down then up.
B is essentially constant
C is slowly degrading (maybe from TLB effects}

> Theo

On 22/12/2023 21:58, Theo wrote:
<snip>
>
> which seems to have more flutter in group A. I get similar results
in a VM
> running on a AMD Ryzen 9 5950X (32) @ 3.393GHz.
>
> Theo
Thank you for those. Neither of them show the odd thing I was seeing -
but I compiled with -ofast. Does that make a difference?

Andy

Vir Campestris <vir.campestris@invalid.invalid> schrieb:
> On 22/12/2023 21:58, Theo wrote:
><snip>
> >
> > which seems to have more flutter in group A. I get similar results
> in a VM
> > running on a AMD Ryzen 9 5950X (32) @ 3.393GHz.
> >
> > Theo
> Thank you for those. Neither of them show the odd thing I was seeing -
> but I compiled with -ofast. Does that make a difference?

That would put it in an executable named "fast" :-)

-march=native -mtune=native might make more of a difference, depending
on how good the compiler's model of your hardware is.

On 23/12/2023 23:50, Thomas Koenig wrote:
> Vir Campestris <vir.campestris@invalid.invalid> schrieb:
>> On 22/12/2023 21:58, Theo wrote:
>> <snip>
>>>
>>> which seems to have more flutter in group A. I get similar results
>> in a VM
>>> running on a AMD Ryzen 9 5950X (32) @ 3.393GHz.
>>>
>>> Theo
>> Thank you for those. Neither of them show the odd thing I was seeing -
>> but I compiled with -ofast. Does that make a difference?
>
> That would put it in an executable named "fast" :-)

Some programs are really fussy about capitalisation!

>
> -march=native -mtune=native might make more of a difference, depending
> on how good the compiler's model of your hardware is.

For gcc on modern x86-64 processors (IME at least), the difference
between -O0 and -O1 is often large, but the difference between -O1 and
higher levels usually makes far less difference. The processors
themselves do a good job of things like instruction scheduling and
register renaming at runtime, and are designed to be good for running
weakly optimised code. I find it makes a bigger difference on
processors that don't do as much at run-time, such as microcontroller cores.

But the "-march=native" can make a very big difference, especially if it
means the compiler can use SIMD or other advanced instructions. (You
don't need "-mtune" if you have "march", as it is implied by "-march" -
you only need both if you want to make a build that will run on many
x86-64 variants but is optimised for one particular one.) And the
"-march=native" benefits go well with some of the higher optimisations -
thus it is good to combine "-march=native" with "-Ofast" or "-O2".

In practice, things also vary dramatically according to the type of program.

On 25/12/2023 14:56, David Brown wrote:
> On 23/12/2023 23:50, Thomas Koenig wrote:
>> Vir Campestris <vir.campestris@invalid.invalid> schrieb:
>>> On 22/12/2023 21:58, Theo wrote:
>>> <snip>
>>>>
>>>> which seems to have more flutter in group A.  I get similar results
>>> in a VM
>>>> running on a AMD Ryzen 9 5950X (32) @ 3.393GHz.
>>>>
>>>> Theo
>>> Thank you for those. Neither of them show the odd thing I was seeing -
>>> but I compiled with -ofast. Does that make a difference?
>>
>> That would put it in an executable named "fast" :-)

YKWIM :P
>
> Some programs are really fussy about capitalisation!
>
>>
>> -march=native -mtune=native might make more of a difference, depending
>> on how good the compiler's model of your hardware is.
>
> For gcc on modern x86-64 processors (IME at least), the difference
> between -O0 and -O1 is often large, but the difference between -O1 and
> higher levels usually makes far less difference.  The processors
> themselves do a good job of things like instruction scheduling and
> register renaming at runtime, and are designed to be good for running
> weakly optimised code.  I find it makes a bigger difference on
> processors that don't do as much at run-time, such as microcontroller
> cores.
>
> But the "-march=native" can make a very big difference, especially if it
> means the compiler can use SIMD or other advanced instructions.  (You
> don't need "-mtune" if you have "march", as it is implied by "-march" -
> you only need both if you want to make a build that will run on many
> x86-64 variants but is optimised for one particular one.)  And the
> "-march=native" benefits go well with some of the higher optimisations -
> thus it is good to combine "-march=native" with "-Ofast" or "-O2".
>
> In practice, things also vary dramatically according to the type of
> program.
>

mtune=native _does_ make a difference. Somewhat to my surprise it makes
it _slower_ - the biggest difference being about 5% with a size of 128k
UINT64.

I checked O0 (really slow) O1 (a lot faster) O3 (quite a bit faster too)
Ofast (mostly slightly faster than O3 when I use a capital O) and O3
march=native.

The pipeline thing made me try something different - instead of
incrementing a value I used std::copy to copy each word to the next one.

That rose to a peak rate of about 64MB/S with a size of 32k, dropped
sharply to 45MB/s and then showed a steady decline to 40MB/s at 256k. It
then dropped sharply to 10MB/S for all larger sizes.

A much more sensible result!

Andy

On 27/12/2023 15:20, Vir Campestris wrote:
> On 25/12/2023 14:56, David Brown wrote:
>> On 23/12/2023 23:50, Thomas Koenig wrote:
>>> Vir Campestris <vir.campestris@invalid.invalid> schrieb:
>>>> On 22/12/2023 21:58, Theo wrote:
>>>> <snip>
>>>>>
>>>>> which seems to have more flutter in group A.  I get similar results
>>>> in a VM
>>>>> running on a AMD Ryzen 9 5950X (32) @ 3.393GHz.
>>>>>
>>>>> Theo
>>>> Thank you for those. Neither of them show the odd thing I was seeing -
>>>> but I compiled with -ofast. Does that make a difference?
>>>
>>> That would put it in an executable named "fast" :-)
>
> YKWIM :P
>>
>> Some programs are really fussy about capitalisation!
>>
>>>
>>> -march=native -mtune=native might make more of a difference, depending
>>> on how good the compiler's model of your hardware is.
>>
>> For gcc on modern x86-64 processors (IME at least), the difference
>> between -O0 and -O1 is often large, but the difference between -O1 and
>> higher levels usually makes far less difference.  The processors
>> themselves do a good job of things like instruction scheduling and
>> register renaming at runtime, and are designed to be good for running
>> weakly optimised code.  I find it makes a bigger difference on
>> processors that don't do as much at run-time, such as microcontroller
>> cores.
>>
>> But the "-march=native" can make a very big difference, especially if
>> it means the compiler can use SIMD or other advanced instructions.
>> (You don't need "-mtune" if you have "march", as it is implied by
>> "-march" - you only need both if you want to make a build that will
>> run on many x86-64 variants but is optimised for one particular one.)
>> And the "-march=native" benefits go well with some of the higher
>> optimisations - thus it is good to combine "-march=native" with
>> "-Ofast" or "-O2".
>>
>> In practice, things also vary dramatically according to the type of
>> program.
>>
>
> mtune=native _does_ make a difference. Somewhat to my surprise it makes
> it _slower_ - the biggest difference being about 5% with a size of 128k
> UINT64.
>
> I checked O0 (really slow) O1 (a lot faster) O3 (quite a bit faster too)
> Ofast (mostly slightly faster than O3 when I use a capital O) and O3
> march=native.
>

"-Ofast" is like -O3, but additionally allows the compiler to "bend" the
rules somewhat. For example, it allows optimisations of stores that
might cause data races if another thread can see the same data, and it
enables "-ffast-math" which treats floating point as somewhat
approximate and always finite, rather than following IEEE rules
strictly. If you are okay with this, then "-Ofast" is fine, but you
should read the documentation to check that you are happy with it.

While you are there, you can see a number of other optimisation flags
that are not enabled by any -O sets. These may or may not help,
depending on your source code.

> The pipeline thing made me try something different - instead of
> incrementing a value I used std::copy to copy each word to the next one.
>
> That rose to a peak rate of about 64MB/S with a size of 32k, dropped
> sharply to 45MB/s and then showed a steady decline to 40MB/s at 256k. It
> then dropped sharply to 10MB/S for all larger sizes.
>
> A much more sensible result!
>
> Andy

On Wed, 27 Dec 2023 14:20:08 +0000
Vir Campestris <vir.campestris@invalid.invalid> wrote:

> On 25/12/2023 14:56, David Brown wrote:
> > On 23/12/2023 23:50, Thomas Koenig wrote:
> >> Vir Campestris <vir.campestris@invalid.invalid> schrieb:
> >>> On 22/12/2023 21:58, Theo wrote:
> >>> <snip>
> >>>>
> >>>> which seems to have more flutter in group A.  I get similar
> >>>> results
> >>> in a VM
> >>>> running on a AMD Ryzen 9 5950X (32) @ 3.393GHz.
> >>>>
> >>>> Theo
> >>> Thank you for those. Neither of them show the odd thing I was
> >>> seeing - but I compiled with -ofast. Does that make a difference?
> >>>
> >>
> >> That would put it in an executable named "fast" :-)
>
> YKWIM :P
> >
> > Some programs are really fussy about capitalisation!
> >
> >>
> >> -march=native -mtune=native might make more of a difference,
> >> depending on how good the compiler's model of your hardware is.
> >
> > For gcc on modern x86-64 processors (IME at least), the difference
> > between -O0 and -O1 is often large, but the difference between -O1
> > and higher levels usually makes far less difference.  The
> > processors themselves do a good job of things like instruction
> > scheduling and register renaming at runtime, and are designed to be
> > good for running weakly optimised code.  I find it makes a bigger
> > difference on processors that don't do as much at run-time, such as
> > microcontroller cores.
> >
> > But the "-march=native" can make a very big difference, especially
> > if it means the compiler can use SIMD or other advanced
> > instructions.  (You don't need "-mtune" if you have "march", as it
> > is implied by "-march" - you only need both if you want to make a
> > build that will run on many x86-64 variants but is optimised for
> > one particular one.)  And the "-march=native" benefits go well with
> > some of the higher optimisations - thus it is good to combine
> > "-march=native" with "-Ofast" or "-O2".
> >
> > In practice, things also vary dramatically according to the type of
> > program.
> >
>
> mtune=native _does_ make a difference. Somewhat to my surprise it
> makes it _slower_ - the biggest difference being about 5% with a size
> of 128k UINT64.
>
> I checked O0 (really slow) O1 (a lot faster) O3 (quite a bit faster
> too) Ofast (mostly slightly faster than O3 when I use a capital O)
> and O3 march=native.
>
> The pipeline thing made me try something different - instead of
> incrementing a value I used std::copy to copy each word to the next
> one.
>
> That rose to a peak rate of about 64MB/S with a size of 32k, dropped
> sharply to 45MB/s and then showed a steady decline to 40MB/s at 256k.
> It then dropped sharply to 10MB/S for all larger sizes.
>
> A much more sensible result!

GB rather than MB, hopefully

>
> Andy

Can you tell us what are you trying to achieve?
Is it a microbenchmark intended to improve your understanding of Zen1
internals?
Or part of the code that you want to run as fast as possible?
If the later, what exactly do you want to be done by the code?

On 27/12/2023 16:38, Michael S wrote:
> On Wed, 27 Dec 2023 14:20:08 +0000
> Vir Campestris <vir.campestris@invalid.invalid> wrote:
>>
>> That rose to a peak rate of about 64MB/S with a size of 32k, dropped
>> sharply to 45MB/s and then showed a steady decline to 40MB/s at 256k.
>> It then dropped sharply to 10MB/S for all larger sizes.
>>
>> A much more sensible result!
>
> GB rather than MB, hopefully
>
>>
>> Andy
>
> Can you tell us what are you trying to achieve?
> Is it a microbenchmark intended to improve your understanding of Zen1
> internals?
> Or part of the code that you want to run as fast as possible?
> If the later, what exactly do you want to be done by the code?
>

It is a microbenchmark.

I was trying to understand cache performance on my system.

Over in comp.lang.c++ you'll find a thread about Sieve or Eratosthenes.

I and another poster are trying to optimise it. Not for any good reason
of course... but I was just curious about some of the results I was getting.

Andy