I've knocked up a little utility program to try to work out some
performance figures for my CPU.

It's an AMD Ryzen™ 5 3400G. It says on the spec it has:
4MB L3 cache
2MB L2 cache
384kb L1 cache

What I do is to xor a location in memory in an array many times.
The size of the area I xor over is set by a mask on the store index.
The words in the store are 64 bit.

A C++ fragment is this. I can post the whole thing if it would help.

// Calculate a bit mask for the entire store
Word mask = storeWordCount - 1;

Stopwatch s;
s.start();
while (1) // until break when mask runs out
{ for (size_t index = 0; index < storeWordCount; ++index)
{
// read and write a word in store.
Raw[index & mask] ^= index;
}
s.lap(mask); // records the current time

if (mask == 0) break; // Stop if we've run out of mask

mask >>= 1; // shrink the mask
}

As you can see it starts with a large mask (in fact for a whole GB) and
halves it as it goes around.

All looks fine at first. I get about 8GB per second with a large mask,
at 4MB it goes up to 15GB/s, at 8MB up to 23. It holds that as the mask
gets smaller. No apparent effect when it gets under the L1 cache size.

But...
When the mask is very small (3) it slows to 18GB/s. With 1 it halves
again, and with zero (so it only operates on the same word over and
over) it's half again. A fifth of the size with a large block.

Something odd is happening here when I hammer the same location (32
bytes and on down) so that it's slower. Yet this ought to be in the L1
data cache.

A late thought was to replace that ^= index with something that reads
the memory only, or that writes it only, instead of doing a
read-modify-write cycle. That gives me much faster performance with
writes than reads. And neither read only, nor write only, show this odd
slow down with small masks.

What am I missing?

Thanks
Andy

Vir Campestris <vir.campestris@invalid.invalid> writes:
> for (size_t index = 0; index < storeWordCount; ++index)
> {
> // read and write a word in store.
> Raw[index & mask] ^= index;
> }
....
>When the mask is very small (3) it slows to 18GB/s. With 1 it halves
>again, and with zero (so it only operates on the same word over and
>over) it's half again. A fifth of the size with a large block.
>
>Something odd is happening here when I hammer the same location (32
>bytes and on down) so that it's slower. Yet this ought to be in the L1
>data cache.
>
>A late thought was to replace that ^= index with something that reads
>the memory only, or that writes it only, instead of doing a
>read-modify-write cycle. That gives me much faster performance with
>writes than reads. And neither read only, nor write only, show this odd
>slow down with small masks.
>
>What am I missing?

When you do

raw[0] ^= index;

in every step you read the result of the pervious iteration, xor it,
and store it again. This means that you have one chain of RMW data
dependences, with one RMW per iteration. On the Zen2 (which your
3400G has), this requires 8 cycles (see column H of
<http://www.complang.tuwien.ac.at/anton/memdep/>). With mask=1, you
get 2 chains, each with one 8-cycle RMW every second iteration, so you
need 4 cycles per iteration (see my column C). With mask=3, you get 4
chains and 2 cycles per iteration. Looking at my results, I would
expect another doubling with mask=7, but maybe your loop is running
into resource limits at that point (mine does 4 RMWs per iteration).

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Tue, 30 Jan 2024 16:36:17 +0000
Vir Campestris <vir.campestris@invalid.invalid> wrote:

> I've knocked up a little utility program to try to work out some
> performance figures for my CPU.
>
> It's an AMD Ryzen™ 5 3400G. It says on the spec it has:
> 4MB L3 cache
> 2MB L2 cache
> 384kb L1 cache
>

That's for the whole chip and it includes L1I caches.
For individual core and excluding L1I the numbers are:
4MB L3 cache
512 KB L2 cache
32 KB L1D cache

> What I do is to xor a location in memory in an array many times.
> The size of the area I xor over is set by a mask on the store index.
> The words in the store are 64 bit.
>
> A C++ fragment is this. I can post the whole thing if it would help.
>
> // Calculate a bit mask for the entire store
> Word mask = storeWordCount - 1;
>
> Stopwatch s;
> s.start();
> while (1) // until break when mask runs out
> {
> for (size_t index = 0; index < storeWordCount; ++index)
> {
> // read and write a word in store.
> Raw[index & mask] ^= index;
> }
> s.lap(mask); // records the current time
>
> if (mask == 0) break; // Stop if we've run out of mask
>
> mask >>= 1; // shrink the mask
> }
>
> As you can see it starts with a large mask (in fact for a whole GB)
> and halves it as it goes around.
>
> All looks fine at first. I get about 8GB per second with a large
> mask, at 4MB it goes up to 15GB/s, at 8MB up to 23. It holds that as
> the mask gets smaller. No apparent effect when it gets under the L1
> cache size.
>
> But...
> When the mask is very small (3) it slows to 18GB/s. With 1 it halves
> again, and with zero (so it only operates on the same word over and
> over) it's half again. A fifth of the size with a large block.
>
> Something odd is happening here when I hammer the same location (32
> bytes and on down) so that it's slower. Yet this ought to be in the
> L1 data cache.
>
> A late thought was to replace that ^= index with something that reads
> the memory only, or that writes it only, instead of doing a
> read-modify-write cycle. That gives me much faster performance with
> writes than reads. And neither read only, nor write only, show this
> odd slow down with small masks.
>
> What am I missing?
>
> Thanks
> Andy

First, I'd look at generated asm.
If compiler was doing a good job then at mask <= 4095 (32 KB) you should
see slightly less than 1 iteration of the loop per cycle, i.e. assuming
4.2 GHz clock, approximately 30 GB/s.
Since you see less, it's a sign that compiler did less than perfect job.
Try to help it with manual loop unrolling.

As to the problem with lower performance at very small masks, it's
expected. CPU tries to execute loads speculatively out of order under
assumption that they don't alias with preceding stores. So actual loads
runs few loop iterations ahead of the stores. We can't say for sure how
many iterations ahead, but 7 to 10 iterations sounds like a good guess.
When your mask=7 (32 bytes) then aliasing starts to happen. On old
primitive CPUs, like Pentium 4, it causes massive slowdown, because
those early loads has to be replayed after rather significant delay
of about 20 cycles (length of pipeline). Your Zen1+ CPU is much smarter,
it detects that things are no good and stops wild speculations. So, you
don't see huge slowdown. But without speculation every load starts only
after all stores that preceded it in program order were either
committed into L1D cache or their address was checked against the
speculative load address and no aliasing was found. Since you see only
mild slowdown, it seems that the later is done rather effectively and
your CPU is still able to run loads speculatively, but now only 2 or 3
steps ahead, which is not enough to get the same performance as before.

Vir Campestris wrote:

> As you can see it starts with a large mask (in fact for a whole GB) and
> halves it as it goes around.

> All looks fine at first. I get about 8GB per second with a large mask,
> at 4MB it goes up to 15GB/s, at 8MB up to 23. It holds that as the mask
> gets smaller. No apparent effect when it gets under the L1 cache size.

The execution window is apparently able to absorb the latency of L3 miss,
and stream L3->L1 accesses.

Anton answered the question regarding small masks.

On Tue, 30 Jan 2024 20:11:42 +0000
mitchalsup@aol.com (MitchAlsup1) wrote:

> Vir Campestris wrote:
>
> > As you can see it starts with a large mask (in fact for a whole GB)
> > and halves it as it goes around.
>
> > All looks fine at first. I get about 8GB per second with a large
> > mask, at 4MB it goes up to 15GB/s, at 8MB up to 23. It holds that
> > as the mask gets smaller. No apparent effect when it gets under the
> > L1 cache size.
>
> The execution window is apparently able to absorb the latency of L3
> miss, and stream L3->L1 accesses.
>

That sounds unlikely. L3 latency is too big to be covered by execution
window. Much more likely they have adequate HW prefetch from L3 to L2
and may be (less likely) even to L1D.

> Anton answered the question regarding small masks.

Vir Campestris wrote:
> I've knocked up a little utility program to try to work out some
> performance figures for my CPU.
>
> It's an AMD Ryzenâ„¢ 5 3400G. It says on the spec it has:
> 4MB L3 cache
> 2MB L2 cache
> 384kb L1 cache
>
> What I do is to xor a location in memory in an array many times.
> The size of the area I xor over is set by a mask on the store index.
> The words in the store are 64 bit.
>
> A C++ fragment is this. I can post the whole thing if it would help.
>
> // Calculate a bit mask for the entire store
> Word mask = storeWordCount - 1;
>
> Stopwatch s;
> s.start();
> while (1)       // until break when mask runs out
> {
>         for (size_t index = 0; index < storeWordCount; ++index)
>         {
>                 // read and write a word in store.
>                 Raw[index & mask] ^= index;
>         }
>         s.lap(mask);            // records the current time
>
>         if (mask == 0) break;   // Stop if we've run out of mask
>
>         mask >>= 1;             // shrink the mask
> }
>
> As you can see it starts with a large mask (in fact for a whole GB) and
> halves it as it goes around.
>
> All looks fine at first. I get about 8GB per second with a large mask,
> at 4MB it goes up to 15GB/s, at 8MB up to 23. It holds that as the mask
> gets smaller. No apparent effect when it gets under the L1 cache size.
>
> But...
> When the mask is very small (3) it slows to 18GB/s. With 1 it halves
> again, and with zero (so it only operates on the same word over and
> over) it's half again. A fifth of the size with a large block.
>
> Something odd is happening here when I hammer the same location (32
> bytes and on down) so that it's slower. Yet this ought to be in the L1
> data cache.
>
> A late thought was to replace that ^= index with something that reads
> the memory only, or that writes it only, instead of doing a
> read-modify-write cycle. That gives me much faster performance with
> writes than reads. And neither read only, nor write only, show this odd
> slow down with small masks.

Mitch, Anton and Michael have already answered, I just want to add that
we have one additional potential factor:

Rowhammer protection:

It is possible that the pattern of re-XORing the same or a small number
of locations over and over could trigger a pattern detector which was
designed to mitigate against Rowhammer.

OTOH, this would much more easily be handled with memory range based
coalescing of write operations in the last level cache, right?

I.e. for normal (write combining) memory, it would (afaik) be legal to
delay the actual writes to RAM for a significant time, long enough to
merge multiple memory writes.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

Terje Mathisen <terje.mathisen@tmsw.no> writes:
>Rowhammer protection:
>
>It is possible that the pattern of re-XORing the same or a small number=20
>of locations over and over could trigger a pattern detector which was=20
>designed to mitigate against Rowhammer.

I don't think that memory controller designers have actually
implemented Rowhammer protection: I would expect that the processor
manufacturers would have bregged about that if they had. They have
not. And even RAM manufacturers have stopped mentioning anything
about Rowhammer in their specs. It seems that all hardware
manufacturers have decided that Rowhammer is something that will just
disappear from public knowledge (and therefore from what they have to
deal with) if they just ignore it long enough. It appears that they
are right.

They seem to take the same approach wrt Spectre-family attacks. In
that case, however, new variants appear all the time, so maybe the
approach won't work here.

However, in the present case "the same small number of locations" is
not hammered, because a small number of memory locations fits into the
cache in the adjacent access pattern that this test uses, and all
writes will just be to the cache.

>OTOH, this would much more easily be handled with memory range based=20
>coalescing of write operations in the last level cache, right?

We have had write-back caches (at the L2 or L1 level, and certainly at
the LLC level) since the later 486 years.

>I.e. for normal (write combining) memory

Normal memory is write-back. AFAIK write combining is for stuff like
graphics card memory.

>it would (afaik) be legal to=20
>delay the actual writes to RAM for a significant time, long enough to=20
>merge multiple memory writes.

And this is what actually happens, through the magic of write-back
caches.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Wed, 31 Jan 2024 07:59:41 +0100
Terje Mathisen <terje.mathisen@tmsw.no> wrote:

> Vir Campestris wrote:
> > I've knocked up a little utility program to try to work out some
> > performance figures for my CPU.
> >
> > It's an AMD Ryzenâ„¢ 5 3400G. It says on the spec it has:
> > 4MB L3 cache
> > 2MB L2 cache
> > 384kb L1 cache
> >
> > What I do is to xor a location in memory in an array many times.
> > The size of the area I xor over is set by a mask on the store index.
> > The words in the store are 64 bit.
> >
> > A C++ fragment is this. I can post the whole thing if it would help.
> >
> > // Calculate a bit mask for the entire store
> > Word mask = storeWordCount - 1;
> >
> > Stopwatch s;
> > s.start();
> > while (1)       // until break when mask runs out
> > {
> >         for (size_t index = 0; index < storeWordCount; ++index)
> >         {
> >                 // read and write a word in store.
> >                 Raw[index & mask] ^= index;
> >         }
> >         s.lap(mask);            // records the current time
> >
> >         if (mask == 0) break;   // Stop if we've run out of mask
> >
> >         mask >>= 1;             // shrink the mask
> > }
> >
> > As you can see it starts with a large mask (in fact for a whole GB)
> > and halves it as it goes around.
> >
> > All looks fine at first. I get about 8GB per second with a large
> > mask, at 4MB it goes up to 15GB/s, at 8MB up to 23. It holds that
> > as the mask gets smaller. No apparent effect when it gets under the
> > L1 cache size.
> >
> > But...
> > When the mask is very small (3) it slows to 18GB/s. With 1 it
> > halves again, and with zero (so it only operates on the same word
> > over and over) it's half again. A fifth of the size with a large
> > block.
> >
> > Something odd is happening here when I hammer the same location (32
> > bytes and on down) so that it's slower. Yet this ought to be in the
> > L1 data cache.
> >
> > A late thought was to replace that ^= index with something that
> > reads the memory only, or that writes it only, instead of doing a
> > read-modify-write cycle. That gives me much faster performance with
> > writes than reads. And neither read only, nor write only, show this
> > odd slow down with small masks.
>
> Mitch, Anton and Michael have already answered, I just want to add
> that we have one additional potential factor:
>
> Rowhammer protection:
>
> It is possible that the pattern of re-XORing the same or a small
> number of locations over and over could trigger a pattern detector
> which was designed to mitigate against Rowhammer.
>
> OTOH, this would much more easily be handled with memory range based
> coalescing of write operations in the last level cache, right?
>
> I.e. for normal (write combining) memory, it would (afaik) be legal
> to delay the actual writes to RAM for a significant time, long enough
> to merge multiple memory writes.
>
> Terje
>
>

I have very little to add to very good response by Anton.
That little addition is: the most if not all Rowhammer POC examples rely
on CLFLUSH. That's what the manual says about it:
"Executions of the CLFLUSH instruction are ordered with respect to each
other and with respect to writes, locked read-modify-write
instructions, fence instructions, and executions of CLFLUSHOPT to the
same cache line.1 They are not ordered with respect to executions of
CLFLUSHOPT to different cache lines."

By now, it seems obvious that making CLFLUSH instruction non-privilaged
and pretty much non-restricted by memory range/page attributes was a
mistake, but that mistake can't be fixed without breaking things.
Considering that CLFLUSH exists since very early 2000s, it is
understandable.
IIRC, ARMv8 did the same mistake a decade later. It is less
understandable.

Michael S <already5chosen@yahoo.com> writes:
>On Wed, 31 Jan 2024 07:59:41 +0100
>Terje Mathisen <terje.mathisen@tmsw.no> wrote:
>
>
>By now, it seems obvious that making CLFLUSH instruction non-privilaged
>and pretty much non-restricted by memory range/page attributes was a
>mistake, but that mistake can't be fixed without breaking things.
>Considering that CLFLUSH exists since very early 2000s, it is
>understandable.
>IIRC, ARMv8 did the same mistake a decade later. It is less
>understandable.

ARMv8 has a control bit that can be set to allow EL0 access
to the DC system instructions. By default it is a privileged
instruction. It is up to the operating software to enable
it for user-mode code.

Michael S <already5chosen@yahoo.com> writes:
>I have very little to add to very good response by Anton.
>That little addition is: the most if not all Rowhammer POC examples rely
>on CLFLUSH. That's what the manual says about it:
>"Executions of the CLFLUSH instruction are ordered with respect to each
>other and with respect to writes, locked read-modify-write
>instructions, fence instructions, and executions of CLFLUSHOPT to the
>same cache line.1 They are not ordered with respect to executions of
>CLFLUSHOPT to different cache lines."
>
>By now, it seems obvious that making CLFLUSH instruction non-privilaged
>and pretty much non-restricted by memory range/page attributes was a
>mistake, but that mistake can't be fixed without breaking things.
>Considering that CLFLUSH exists since very early 2000s, it is
>understandable.
>IIRC, ARMv8 did the same mistake a decade later. It is less
>understandable.

Ideally caches are fully transparent microarchitecture, then you don't
need stuff like CLFLUSH. My guess is that CLFLUSH is there for
getting DRAM up-to-date for DMA from I/O devices. An alternative
would be to let the memory controller remember which lines are
modified, and if the I/O device asks for that line, get the up-to-date
data from the cache line using the cache-consistency protocol. This
would turn CLFLUSH into a noop (at least as far as writing to DRAM is
concerned, the ordering constraints may still be relevant), so there
is a way to fix this mistake (if it is one).

However, AFAIK this is insufficient for fixing Rowhammer. Caches have
relatively limited associativity, up to something like 16-way
set-associativity, so if you write to the same set 17 times, you are
guaranteed to miss the cache. With 3 levels of cache you may need 49
accesses (probably less), but I expect that the resulting DRAM
accesses to a cache line are still not rare enough that Rowhammer
cannot happen.

The first paper on Rowhammer already outlined how the memory
controller could count how often adjacent DRAM rows are accessed and
thus weaken the row under consideration. This approach needs a little
adjustment for Double Rowhammer and not immediately neighbouring rows,
but otherwise seems to me to be the way to go. With autorefresh in
the DRAM devices these days, the DRAM manufacturers could implement
this on their own, without needing to coordinate with memory
controller designers. But apparently they think that the customers
don't care, so they can save the expense.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Anton Ertl wrote:

> Michael S <already5chosen@yahoo.com> writes:
>>I have very little to add to very good response by Anton.
>>That little addition is: the most if not all Rowhammer POC examples rely
>>on CLFLUSH. That's what the manual says about it:
>>"Executions of the CLFLUSH instruction are ordered with respect to each
>>other and with respect to writes, locked read-modify-write
>>instructions, fence instructions, and executions of CLFLUSHOPT to the
>>same cache line.1 They are not ordered with respect to executions of
>>CLFLUSHOPT to different cache lines."
>>
>>By now, it seems obvious that making CLFLUSH instruction non-privilaged
>>and pretty much non-restricted by memory range/page attributes was a
>>mistake, but that mistake can't be fixed without breaking things.
>>Considering that CLFLUSH exists since very early 2000s, it is
>>understandable.
>>IIRC, ARMv8 did the same mistake a decade later. It is less
>>understandable.

> Ideally caches are fully transparent microarchitecture, then you don't
> need stuff like CLFLUSH. My guess is that CLFLUSH is there for
> getting DRAM up-to-date for DMA from I/O devices.

I have wondered for a while about why device access is not to coherent
space. If it were so, then no CFLUSH functionality is needed, I/O can
just read/write an address and always get the freshest copy. {{Maybe
not the device itself, but the PCIe Root could translate from device
access space to memory access space (coherent).}}
> An alternative
> would be to let the memory controller remember which lines are
> modified, and if the I/O device asks for that line, get the up-to-date
> data from the cache line using the cache-consistency protocol. This
> would turn CLFLUSH into a noop (at least as far as writing to DRAM is
> concerned, the ordering constraints may still be relevant), so there
> is a way to fix this mistake (if it is one).

> However, AFAIK this is insufficient for fixing Rowhammer.

If L3 (LLC) is not a processor cache but a great big read/write buffer
for DRAM, then Rowhammering is significantly harder to accomplish.
> Caches have
> relatively limited associativity, up to something like 16-way
> set-associativity, so if you write to the same set 17 times, you are
> guaranteed to miss the cache. With 3 levels of cache you may need 49
> accesses (probably less), but I expect that the resulting DRAM
> accesses to a cache line are still not rare enough that Rowhammer
> cannot happen.

Rowhammer happens when you beat on the same cache line multiple times
{causing a charge sharing problem on the word lines. Every time you cause
the DRAM to precharge (deActivate) you lose the count on how many times
you have to bang on the same word line to disrupt the stored cells.

So, the trick is to detect the RowHammering and insert refresh commands.

> The first paper on Rowhammer already outlined how the memory
> controller could count how often adjacent DRAM rows are accessed and
> thus weaken the row under consideration. This approach needs a little
> adjustment for Double Rowhammer and not immediately neighbouring rows,
> but otherwise seems to me to be the way to go. With autorefresh in
> the DRAM devices these days, the DRAM manufacturers could implement
> this on their own, without needing to coordinate with memory
> controller designers. But apparently they think that the customers
> don't care, so they can save the expense.

> - anton

On Wed, 31 Jan 2024 17:17:21 GMT
anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

> Michael S <already5chosen@yahoo.com> writes:
> >I have very little to add to very good response by Anton.
> >That little addition is: the most if not all Rowhammer POC examples
> >rely on CLFLUSH. That's what the manual says about it:
> >"Executions of the CLFLUSH instruction are ordered with respect to
> >each other and with respect to writes, locked read-modify-write
> >instructions, fence instructions, and executions of CLFLUSHOPT to the
> >same cache line.1 They are not ordered with respect to executions of
> >CLFLUSHOPT to different cache lines."
> >
> >By now, it seems obvious that making CLFLUSH instruction
> >non-privilaged and pretty much non-restricted by memory range/page
> >attributes was a mistake, but that mistake can't be fixed without
> >breaking things. Considering that CLFLUSH exists since very early
> >2000s, it is understandable.
> >IIRC, ARMv8 did the same mistake a decade later. It is less
> >understandable.
>
> Ideally caches are fully transparent microarchitecture, then you don't
> need stuff like CLFLUSH. My guess is that CLFLUSH is there for
> getting DRAM up-to-date for DMA from I/O devices. An alternative
> would be to let the memory controller remember which lines are
> modified, and if the I/O device asks for that line, get the up-to-date
> data from the cache line using the cache-consistency protocol.

Considering that CLFLUSH was introduced by Intel in year 2000 or 2001
and that at that time all Intel's PCI/AGP root hubs were already fully
I/O-coherent for several years, I find your theory unlikely.

Myself, I don't know the original reason, but I do know a use case
where CLFLUSH, while not strictly necessary, simplifies things greatly
- entering deep sleep state in which CPU caches are powered down and
DRAM put in self-refresh mode.

Of course, this particular use case does not require *non-priviledged*
CLFLUSH, so obviously Intel had different reason.

> This
> would turn CLFLUSH into a noop (at least as far as writing to DRAM is
> concerned, the ordering constraints may still be relevant), so there
> is a way to fix this mistake (if it is one).
>
> However, AFAIK this is insufficient for fixing Rowhammer. Caches have
> relatively limited associativity, up to something like 16-way
> set-associativity, so if you write to the same set 17 times, you are
> guaranteed to miss the cache. With 3 levels of cache you may need 49
> accesses (probably less), but I expect that the resulting DRAM
> accesses to a cache line are still not rare enough that Rowhammer
> cannot happen.
>

Original RH required very high hammering rate that certainly can't be
achieved by playing with associativity of L3 cache.

Newer multiside hammering probably can do it in theory, but it would be
very difficult in practice.

Today we have yet another variant called RowPress that bypasses TRR
mitigation more reliably than mult-rate RH. I think this one would be
practically impossible without CLFLUSH., esp. when system under attack
carries other DRAM accesses in parallel with attackers code.

> The first paper on Rowhammer already outlined how the memory
> controller could count how often adjacent DRAM rows are accessed and
> thus weaken the row under consideration. This approach needs a little
> adjustment for Double Rowhammer and not immediately neighbouring rows,
> but otherwise seems to me to be the way to go.

IMHO, all thise solutions are pure fantasy, because memory controller
does not even know which rows are physically adjacent. POC authors
typically run lengthy tests in order to figure it out.

> With autorefresh in
> the DRAM devices these days, the DRAM manufacturers could implement
> this on their own, without needing to coordinate with memory
> controller designers. But apparently they think that the customers
> don't care, so they can save the expense.
>
> - anton

They cared enough to implement the simplest of proposed solutions - TRR.
Yes, it was quickly found insufficient, but at least there was a
demonstration of good intentions.

Michael S wrote:

> On Wed, 31 Jan 2024 17:17:21 GMT
> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

>> Michael S <already5chosen@yahoo.com> writes:
>> >I have very little to add to very good response by Anton.
>> >That little addition is: the most if not all Rowhammer POC examples
>> >rely on CLFLUSH. That's what the manual says about it:
>> >"Executions of the CLFLUSH instruction are ordered with respect to
>> >each other and with respect to writes, locked read-modify-write
>> >instructions, fence instructions, and executions of CLFLUSHOPT to the
>> >same cache line.1 They are not ordered with respect to executions of
>> >CLFLUSHOPT to different cache lines."
>> >
>> >By now, it seems obvious that making CLFLUSH instruction
>> >non-privilaged and pretty much non-restricted by memory range/page
>> >attributes was a mistake, but that mistake can't be fixed without
>> >breaking things. Considering that CLFLUSH exists since very early
>> >2000s, it is understandable.
>> >IIRC, ARMv8 did the same mistake a decade later. It is less
>> >understandable.
>>
>> Ideally caches are fully transparent microarchitecture, then you don't
>> need stuff like CLFLUSH. My guess is that CLFLUSH is there for
>> getting DRAM up-to-date for DMA from I/O devices. An alternative
>> would be to let the memory controller remember which lines are
>> modified, and if the I/O device asks for that line, get the up-to-date
>> data from the cache line using the cache-consistency protocol.

> Considering that CLFLUSH was introduced by Intel in year 2000 or 2001
> and that at that time all Intel's PCI/AGP root hubs were already fully
> I/O-coherent for several years, I find your theory unlikely.

> Myself, I don't know the original reason, but I do know a use case
> where CLFLUSH, while not strictly necessary, simplifies things greatly
> - entering deep sleep state in which CPU caches are powered down and
> DRAM put in self-refresh mode.

> Of course, this particular use case does not require *non-priviledged*
> CLFLUSH, so obviously Intel had different reason.

There was no assumption that this could result in a side channel or
attack vector at the time of its non-privileged inclusion. Afterwards
there was no reason to make it privileged until 2017 and by then the
ability to do anything about it has vanished.

Me, personally, I see this as a violation of the cache is there to
reduce memory latency principle and thereby improve performance.

>> This
>> would turn CLFLUSH into a noop (at least as far as writing to DRAM is
>> concerned, the ordering constraints may still be relevant), so there
>> is a way to fix this mistake (if it is one).
>>
>> However, AFAIK this is insufficient for fixing Rowhammer. Caches have
>> relatively limited associativity, up to something like 16-way
>> set-associativity, so if you write to the same set 17 times, you are
>> guaranteed to miss the cache. With 3 levels of cache you may need 49
>> accesses (probably less), but I expect that the resulting DRAM
>> accesses to a cache line are still not rare enough that Rowhammer
>> cannot happen.
>>

> Original RH required very high hammering rate that certainly can't be
> achieved by playing with associativity of L3 cache.

> Newer multiside hammering probably can do it in theory, but it would be
> very difficult in practice.

The problem here is the fact that DRAMs do not use linear decoders, so
address X and address X+1 do not necessarily shared paired word lines.
The word lines could be as far as ½ the block away from each other.

The DRAM decoders are faster and smaller when there is a grey-like-code
imposed on the logical-address to physical-word-line. This also happens
in SRAM decoders. Going back and looking at the most used logical to
physical mapping shows that while X and X+1 can (occasionally) be side
by side, X, X+1 and X+2 should never be 3 words lines in a row.

> Today we have yet another variant called RowPress that bypasses TRR
> mitigation more reliably than mult-rate RH. I think this one would be
> practically impossible without CLFLUSH., esp. when system under attack
> carries other DRAM accesses in parallel with attackers code.

>> The first paper on Rowhammer already outlined how the memory
>> controller could count how often adjacent DRAM rows are accessed and
>> thus weaken the row under consideration. This approach needs a little
>> adjustment for Double Rowhammer and not immediately neighbouring rows,
>> but otherwise seems to me to be the way to go.

> IMHO, all thise solutions are pure fantasy, because memory controller
> does not even know which rows are physically adjacent.

Different DIMMs and even different DRAMs on the same DIMM may not
share that correspondence. {There is a lot of bit line and a little
word line repair done at the tester.}

> POC authors
> typically run lengthy tests in order to figure it out.

>> With autorefresh in
>> the DRAM devices these days, the DRAM manufacturers could implement
>> this on their own, without needing to coordinate with memory
>> controller designers. But apparently they think that the customers
>> don't care, so they can save the expense.
>>
>> - anton

> They cared enough to implement the simplest of proposed solutions - TRR.
> Yes, it was quickly found insufficient, but at least there was a
> demonstration of good intentions.

Michael S <already5chosen@yahoo.com> wrote:
>
> By now, it seems obvious that making CLFLUSH instruction non-privilaged
> and pretty much non-restricted by memory range/page attributes was a
> mistake, but that mistake can't be fixed without breaking things.
> Considering that CLFLUSH exists since very early 2000s, it is
> understandable.
> IIRC, ARMv8 did the same mistake a decade later. It is less
> understandable.

For Arm, with its non-coherent data and instruction caches, you need
some way to flush dcache to the point of unification in order to make
instruction changes visible. Also, regardless of icache coherence, when
using non-volatile memory you need an efficient way to flush dcache to
the point of peristence. You need that in order to make sure that a
transaction has been written to a log.

With the latter, you could restrict dcache flushes to pages with a
particular non-volatile attribute. I don't think there's anything you
can do about the former, short of simply making i- and d-cache
coherent. Which is a good idea, but not everyone does it.

Andrew.

On Thu, 01 Feb 2024 09:39:13 +0000
aph@littlepinkcloud.invalid wrote:

> Michael S <already5chosen@yahoo.com> wrote:
> >
> > By now, it seems obvious that making CLFLUSH instruction
> > non-privilaged and pretty much non-restricted by memory range/page
> > attributes was a mistake, but that mistake can't be fixed without
> > breaking things. Considering that CLFLUSH exists since very early
> > 2000s, it is understandable.
> > IIRC, ARMv8 did the same mistake a decade later. It is less
> > understandable.
>
> For Arm, with its non-coherent data and instruction caches, you need
> some way to flush dcache to the point of unification in order to make
> instruction changes visible. Also, regardless of icache coherence,
> when using non-volatile memory you need an efficient way to flush
> dcache to the point of peristence. You need that in order to make
> sure that a transaction has been written to a log.
>
> With the latter, you could restrict dcache flushes to pages with a
> particular non-volatile attribute. I don't think there's anything you
> can do about the former, short of simply making i- and d-cache
> coherent.

For the later, privileged flush instruction sounds sufficient.

For the former, ARMv8 appears to have a special instruction (or you can
call it a special variant of DC instruction) - Clean by virtual address
to point of unification (DC CVAU). This instruction alone would not
make RH attack much easier. The problem is that privilagability of this
instruction controlled by the same bit as privilagability of two much
more dangerous variations of DC (DC CVAC and DC CIVAC).

> Which is a good idea, but not everyone does it.
>
> Andrew.

Neoverse N1 had it. I don't know about the rest of Neoverse series.

Anton Ertl wrote:
> Michael S <already5chosen@yahoo.com> writes:
>> I have very little to add to very good response by Anton.
>> That little addition is: the most if not all Rowhammer POC examples rely
>> on CLFLUSH. That's what the manual says about it:
>> "Executions of the CLFLUSH instruction are ordered with respect to each
>> other and with respect to writes, locked read-modify-write
>> instructions, fence instructions, and executions of CLFLUSHOPT to the
>> same cache line.1 They are not ordered with respect to executions of
>> CLFLUSHOPT to different cache lines."
>>
>> By now, it seems obvious that making CLFLUSH instruction non-privilaged
>> and pretty much non-restricted by memory range/page attributes was a
>> mistake, but that mistake can't be fixed without breaking things.
>> Considering that CLFLUSH exists since very early 2000s, it is
>> understandable.
>> IIRC, ARMv8 did the same mistake a decade later. It is less
>> understandable.
>
> Ideally caches are fully transparent microarchitecture, then you don't
> need stuff like CLFLUSH. My guess is that CLFLUSH is there for
> getting DRAM up-to-date for DMA from I/O devices. An alternative
> would be to let the memory controller remember which lines are
> modified, and if the I/O device asks for that line, get the up-to-date
> data from the cache line using the cache-consistency protocol. This
> would turn CLFLUSH into a noop (at least as far as writing to DRAM is
> concerned, the ordering constraints may still be relevant), so there
> is a way to fix this mistake (if it is one).

The text in Intel Vol 1 Architecture manual indicates they viewed all
these cache control instruction PREFETCH, CLFLUSH, and CLFLUSHOPT
as part of SSE for use by graphics applications that want to take
manual control of their caching and minimize cache pollution.

Note that the non-temporal move instructions MOVNTxx were also part of
that SSE bunch and could also be used to force a write to DRAM.

Michael S wrote:
> On Wed, 31 Jan 2024 17:17:21 GMT
> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>
>> Michael S <already5chosen@yahoo.com> writes:
>>> I have very little to add to very good response by Anton.
>>> That little addition is: the most if not all Rowhammer POC examples
>>> rely on CLFLUSH. That's what the manual says about it:
>>> "Executions of the CLFLUSH instruction are ordered with respect to
>>> each other and with respect to writes, locked read-modify-write
>>> instructions, fence instructions, and executions of CLFLUSHOPT to the
>>> same cache line.1 They are not ordered with respect to executions of
>>> CLFLUSHOPT to different cache lines."
>>>
>>> By now, it seems obvious that making CLFLUSH instruction
>>> non-privilaged and pretty much non-restricted by memory range/page
>>> attributes was a mistake, but that mistake can't be fixed without
>>> breaking things. Considering that CLFLUSH exists since very early
>>> 2000s, it is understandable.
>>> IIRC, ARMv8 did the same mistake a decade later. It is less
>>> understandable.
>> Ideally caches are fully transparent microarchitecture, then you don't
>> need stuff like CLFLUSH. My guess is that CLFLUSH is there for
>> getting DRAM up-to-date for DMA from I/O devices. An alternative
>> would be to let the memory controller remember which lines are
>> modified, and if the I/O device asks for that line, get the up-to-date
>> data from the cache line using the cache-consistency protocol.
>
> Considering that CLFLUSH was introduced by Intel in year 2000 or 2001
> and that at that time all Intel's PCI/AGP root hubs were already fully
> I/O-coherent for several years, I find your theory unlikely.
>
> Myself, I don't know the original reason, but I do know a use case
> where CLFLUSH, while not strictly necessary, simplifies things greatly
> - entering deep sleep state in which CPU caches are powered down and
> DRAM put in self-refresh mode.

CLFLUSH wouldn't be useful for that as it flushes for a virtual address.
It also allows all sorts reorderings that you don't want to think about
during a (possibly emergency) cache sync.

The privileged WBINVD and WBNOINVD instructions are intended for that.
It sounds like they basically halt the core for the duration of the
write back of all modified lines.

On Thu, 01 Feb 2024 09:05:19 -0500
EricP <ThatWouldBeTelling@thevillage.com> wrote:

> Anton Ertl wrote:
> > Michael S <already5chosen@yahoo.com> writes:
> >> I have very little to add to very good response by Anton.
> >> That little addition is: the most if not all Rowhammer POC
> >> examples rely on CLFLUSH. That's what the manual says about it:
> >> "Executions of the CLFLUSH instruction are ordered with respect to
> >> each other and with respect to writes, locked read-modify-write
> >> instructions, fence instructions, and executions of CLFLUSHOPT to
> >> the same cache line.1 They are not ordered with respect to
> >> executions of CLFLUSHOPT to different cache lines."
> >>
> >> By now, it seems obvious that making CLFLUSH instruction
> >> non-privilaged and pretty much non-restricted by memory range/page
> >> attributes was a mistake, but that mistake can't be fixed without
> >> breaking things. Considering that CLFLUSH exists since very early
> >> 2000s, it is understandable.
> >> IIRC, ARMv8 did the same mistake a decade later. It is less
> >> understandable.
> >
> > Ideally caches are fully transparent microarchitecture, then you
> > don't need stuff like CLFLUSH. My guess is that CLFLUSH is there
> > for getting DRAM up-to-date for DMA from I/O devices. An
> > alternative would be to let the memory controller remember which
> > lines are modified, and if the I/O device asks for that line, get
> > the up-to-date data from the cache line using the cache-consistency
> > protocol. This would turn CLFLUSH into a noop (at least as far as
> > writing to DRAM is concerned, the ordering constraints may still be
> > relevant), so there is a way to fix this mistake (if it is one).
>
> The text in Intel Vol 1 Architecture manual indicates they viewed all
> these cache control instruction PREFETCH, CLFLUSH, and CLFLUSHOPT
> as part of SSE for use by graphics applications that want to take
> manual control of their caching and minimize cache pollution.
>
> Note that the non-temporal move instructions MOVNTxx were also part of
> that SSE bunch and could also be used to force a write to DRAM.
>

According to Wikipedia, CLFLUSH was not introduced with SSE.
It was introduced together with SSE2, but formally is not part of it.
CLFLUSHOPT came much, much, much later and was likely related to Optane
DIMMs aspirations of late 2010s.

On 2/1/2024 6:05 AM, EricP wrote:
> Anton Ertl wrote:
>> Michael S <already5chosen@yahoo.com> writes:
>>> I have very little to add to very good response by Anton.
>>> That little addition is: the most if not all Rowhammer POC examples rely
>>> on CLFLUSH. That's what the manual says about it:
>>> "Executions of the CLFLUSH instruction are ordered with respect to each
>>> other and with respect to writes, locked read-modify-write
>>> instructions, fence instructions, and executions of CLFLUSHOPT to the
>>> same cache line.1 They are not ordered with respect to executions of
>>> CLFLUSHOPT to different cache lines."
>>>
>>> By now, it seems obvious that making CLFLUSH instruction non-privilaged
>>> and pretty much non-restricted by memory range/page attributes was a
>>> mistake, but that mistake can't be fixed without breaking things.
>>> Considering that CLFLUSH exists since very early 2000s, it is
>>> understandable.
>>> IIRC, ARMv8 did the same mistake a decade later. It is less
>>> understandable.
>>
>> Ideally caches are fully transparent microarchitecture, then you don't
>> need stuff like CLFLUSH.  My guess is that CLFLUSH is there for
>> getting DRAM up-to-date for DMA from I/O devices.  An alternative
>> would be to let the memory controller remember which lines are
>> modified, and if the I/O device asks for that line, get the up-to-date
>> data from the cache line using the cache-consistency protocol.  This
>> would turn CLFLUSH into a noop (at least as far as writing to DRAM is
>> concerned, the ordering constraints may still be relevant), so there
>> is a way to fix this mistake (if it is one).
>
> The text in Intel Vol 1 Architecture manual indicates they viewed all
> these cache control instruction PREFETCH, CLFLUSH, and CLFLUSHOPT
> as part of SSE for use by graphics applications that want to take
> manual control of their caching and minimize cache pollution.
>
> Note that the non-temporal move instructions MOVNTxx were also part of
> that SSE bunch and could also be used to force a write to DRAM.
>
>
>

Then there are the LFENCE, SFENCE and MFENCE for write back memory.
Non-temporal stores, iirc.

On 2/1/2024 12:48 PM, Chris M. Thomasson wrote:
> On 2/1/2024 6:05 AM, EricP wrote:
>> Anton Ertl wrote:
>>> Michael S <already5chosen@yahoo.com> writes:
>>>> I have very little to add to very good response by Anton.
>>>> That little addition is: the most if not all Rowhammer POC examples
>>>> rely
>>>> on CLFLUSH. That's what the manual says about it:
>>>> "Executions of the CLFLUSH instruction are ordered with respect to each
>>>> other and with respect to writes, locked read-modify-write
>>>> instructions, fence instructions, and executions of CLFLUSHOPT to the
>>>> same cache line.1 They are not ordered with respect to executions of
>>>> CLFLUSHOPT to different cache lines."
>>>>
>>>> By now, it seems obvious that making CLFLUSH instruction non-privilaged
>>>> and pretty much non-restricted by memory range/page attributes was a
>>>> mistake, but that mistake can't be fixed without breaking things.
>>>> Considering that CLFLUSH exists since very early 2000s, it is
>>>> understandable.
>>>> IIRC, ARMv8 did the same mistake a decade later. It is less
>>>> understandable.
>>>
>>> Ideally caches are fully transparent microarchitecture, then you don't
>>> need stuff like CLFLUSH.  My guess is that CLFLUSH is there for
>>> getting DRAM up-to-date for DMA from I/O devices.  An alternative
>>> would be to let the memory controller remember which lines are
>>> modified, and if the I/O device asks for that line, get the up-to-date
>>> data from the cache line using the cache-consistency protocol.  This
>>> would turn CLFLUSH into a noop (at least as far as writing to DRAM is
>>> concerned, the ordering constraints may still be relevant), so there
>>> is a way to fix this mistake (if it is one).
>>
>> The text in Intel Vol 1 Architecture manual indicates they viewed all
>> these cache control instruction PREFETCH, CLFLUSH, and CLFLUSHOPT
>> as part of SSE for use by graphics applications that want to take
>> manual control of their caching and minimize cache pollution.
>>
>> Note that the non-temporal move instructions MOVNTxx were also part of
>> that SSE bunch and could also be used to force a write to DRAM.
>>
>>
>>
>
> Then there are the LFENCE, SFENCE and MFENCE for write back memory.
> Non-temporal stores, iirc.

Oops, non-write back memory! IIRC. Sorry.

Michael S <already5chosen@yahoo.com> wrote:
> On Thu, 01 Feb 2024 09:39:13 +0000
> aph@littlepinkcloud.invalid wrote:
>
>> Michael S <already5chosen@yahoo.com> wrote:
>> >
>> > By now, it seems obvious that making CLFLUSH instruction
>> > non-privilaged and pretty much non-restricted by memory range/page
>> > attributes was a mistake, but that mistake can't be fixed without
>> > breaking things. Considering that CLFLUSH exists since very early
>> > 2000s, it is understandable.
>> > IIRC, ARMv8 did the same mistake a decade later. It is less
>> > understandable.
>>
>> For Arm, with its non-coherent data and instruction caches, you need
>> some way to flush dcache to the point of unification in order to make
>> instruction changes visible. Also, regardless of icache coherence,
>> when using non-volatile memory you need an efficient way to flush
>> dcache to the point of peristence. You need that in order to make
>> sure that a transaction has been written to a log.
>>
>> With the latter, you could restrict dcache flushes to pages with a
>> particular non-volatile attribute. I don't think there's anything you
>> can do about the former, short of simply making i- and d-cache
>> coherent.
>
> For the later, privileged flush instruction sounds sufficient.

Does it? You're trying for hight throughput, and a full system call
wouldn't help with that. And besides, if userspace can ask kernel to
do something on its behalf, you haven't added any security by making
it privileged.

> For the former, ARMv8 appears to have a special instruction (or you can
> call it a special variant of DC instruction) - Clean by virtual address
> to point of unification (DC CVAU). This instruction alone would not
> make RH attack much easier. The problem is that privilagability of this
> instruction controlled by the same bit as privilagability of two much
> more dangerous variations of DC (DC CVAC and DC CIVAC).

Ah, thanks.

Andrew.

MitchAlsup wrote:
> Anton Ertl wrote:
>
>
> Rowhammer happens when you beat on the same cache line multiple times
> {causing a charge sharing problem on the word lines. Every time you cause
> the DRAM to precharge (deActivate) you lose the count on how many times
> you have to bang on the same word line to disrupt the stored cells.
>
> So, the trick is to detect the RowHammering and insert refresh commands.

It's not just the immediately physically adjacent rows -
I think I read that the effect falls off for up to +-3 rows away.

Also it may be data dependent - 0's bleed into adjacent 1's and 1's into 0's.

And the threshold when it triggers has been changing as drams become more
dense. In 2014 when this was first encountered it took 139K activations.
By 2020 that was down to 4.8K.

So figuring out how much a row has been damaged is complicated,
and the window for detecting it is getting smaller.

MitchAlsup wrote:
> Michael S wrote:
>
>> Original RH required very high hammering rate that certainly can't be
>> achieved by playing with associativity of L3 cache.
>
>> Newer multiside hammering probably can do it in theory, but it would be
>> very difficult in practice.
>
> The problem here is the fact that DRAMs do not use linear decoders, so
> address X and address X+1 do not necessarily shared paired word lines.
> The word lines could be as far as ½ the block away from each other.
>
> The DRAM decoders are faster and smaller when there is a grey-like-code
> imposed on the logical-address to physical-word-line. This also happens
> in SRAM decoders. Going back and looking at the most used logical to
> physical mapping shows that while X and X+1 can (occasionally) be side
> by side, X, X+1 and X+2 should never be 3 words lines in a row.

A 16 Gb dram with 8kb rows has 2^21 = 2 million rows.
So having a counter for each row is impractical.

I was wondering if each row could have "canary" bit,
a specially weakened bit that always flips early.
This would also intrinsically handle the cases of effects
falling off over the +-3 adjacent rows.

Then a giant 2 million input OR gate would tell us if any row's
canary had flipped.

EricP <ThatWouldBeTelling@thevillage.com> schrieb:

> Then a giant 2 million input OR gate would tell us if any row's
> canary had flipped.

That would look... interesting.

How are large OR gates actually constructed? I would assume that an
eight-input OR gate could look something like

nand(nor(a,b),nor(c,d),nor(e,f),nor(g,h))

which would reduce the number of inputs by a factor of 2^3, so
seven layers of these OR gates would be needed.

Wiring would be interesting as well...

Thomas Koenig wrote:

> EricP <ThatWouldBeTelling@thevillage.com> schrieb:

>> Then a giant 2 million input OR gate would tell us if any row's
>> canary had flipped.

> That would look... interesting.

> How are large OR gates actually constructed? I would assume that an
> eight-input OR gate could look something like

> nand(nor(a,b),nor(c,d),nor(e,f),nor(g,h))

Close, but NANDs come with 4-inputs and NORs come with 3*, so you get
a 3×4 = 12:1 reduction per pair of stages.

2985984->248832->20736->1728->144->12->1

> which would reduce the number of inputs by a factor of 2^3, so
> seven layers of these OR gates would be needed.

6 not 7

> Wiring would be interesting as well...

That is why we have 10 layers of metal--oh wait DRAMs don't have that
much metal.....

(*) NANDs having 4 inputs while NORs only have 3 is a consequence of
P-channel transistors having lower transconductance and higher body
effects, and there are differences between planar transistors and
finFETs here, too.