EricP wrote:

> MitchAlsup wrote:
>> Anton Ertl wrote:
>>
>>
>> Rowhammer happens when you beat on the same cache line multiple times
>> {causing a charge sharing problem on the word lines. Every time you cause
>> the DRAM to precharge (deActivate) you lose the count on how many times
>> you have to bang on the same word line to disrupt the stored cells.
>>
>> So, the trick is to detect the RowHammering and insert refresh commands.

> It's not just the immediately physically adjacent rows -
> I think I read that the effect falls off for up to +-3 rows away.

My understanding is that RowHammer has to access the same row multiple times
to disrupt bits in an adjacent row. This sounds like a charge sharing problem.
A long time ago We found a problem with one manufactures SRAM when the same
row was hit >6,000 times, there was enough charge sharing that the adjacent
dynamic word decoder also fired so we had 2 or 3 word lines active at the
same time. We encountered this when a LD missed the cache and was sent down
through NorthBridge, SouthBridge, onto another bus, finally out to the device
and back, while the CPU was continuing to read the ICache every cycle.

My limited understanding of RowPress is that you should not keep the Row open
for more than a page of data transfer (about ¼ of 7.8µs DDR4 limit). My bet is
that this is a leakage issue on the bit line made sensitive by the word line.

> Also it may be data dependent - 0's bleed into adjacent 1's and 1's into 0's.

DRAMs are funny like this. Adjacent bit lines store data differently. Even
bits store 0 as 0 and 1 as 1 while odd cells store 0 as 1 and 1 as 0. They
do this so the sense amplified has a differential to sense, either the even
cell or the odd cell is asserted on the bit line pair and the sense amp then
has a differential to sense. One line goes up a little or down a little while
the other bit line stays where it is.

> And the threshold when it triggers has been changing as drams become more
> dense. In 2014 when this was first encountered it took 139K activations.
> By 2020 that was down to 4.8K.

> So figuring out how much a row has been damaged is complicated,
> and the window for detecting it is getting smaller.

MitchAlsup wrote:
> EricP wrote:
>
>> MitchAlsup wrote:
>>> Anton Ertl wrote:
>>>
>>>
>>> Rowhammer happens when you beat on the same cache line multiple times
>>> {causing a charge sharing problem on the word lines. Every time you
>>> cause
>>> the DRAM to precharge (deActivate) you lose the count on how many times
>>> you have to bang on the same word line to disrupt the stored cells.
>>>
>>> So, the trick is to detect the RowHammering and insert refresh commands.
>
>> It's not just the immediately physically adjacent rows -
>> I think I read that the effect falls off for up to +-3 rows away.
>
> My understanding is that RowHammer has to access the same row multiple
> times
> to disrupt bits in an adjacent row. This sounds like a charge sharing
> problem.

Yes, as I understand it charge migration.
I had a nice document on the root cause of Rowhammer but I can't seem to
find it again. This one is a little heavy on the semiconductor physics:

On dram rowhammer and the physics of insecurity, 2020
https://ieeexplore.ieee.org/iel7/16/9385809/09366976.pdf

"Experimental evidence points to two mechanisms for the RH disturb,
namely cell transistor subthreshold leakage and electron injection
into the p-well of the DRAM array from the hammered cell transistors
and their subsequent capture by the storage node (SN) junctions [13].

Regarding the subthreshold leakage, lower cell transistor threshold
voltages have been shown to correlate with higher susceptibility to RH.
This is consistent with crosstalk between the switching aggressor wordline
and the victim wordlines pulling up the latter sufficiently in the
potential to drain away some of the victim cell’s stored charge [14], [15].

Regarding the injected electrons from the hammered cell transistors,
the blame for these has been placed on two different origins.
The first describes a collapsing inversion layer associated with the
hammered cell transistor where a population of electrons is injected
into the p-well as the transistor’s gate turns off [16]. The second
describes electron injection from charge traps near the silicon/gate
dielectric interface of the cell select transistor [13], [17].
Several studies look into techniques for hampering the migration of
these injected electrons."

> A long time ago We found a problem with one manufactures SRAM when the same
> row was hit >6,000 times, there was enough charge sharing that the
> adjacent dynamic word decoder also fired so we had 2 or 3 word lines
> active at the same time. We encountered this when a LD missed the cache
> and was sent down
> through NorthBridge, SouthBridge, onto another bus, finally out to the
> device
> and back, while the CPU was continuing to read the ICache every cycle.

I think of this as aging: each activation ages the rows up to some distance
by amounts depending on the distance due to charge migration.

Originally it was found by activating rows immediately adjacent to the
victim but then they looked and found it further out to +-4 rows.
This effect appears to be called the Rowhammer "blast radius".

This paper is from 2023 but I'm sure I've seen mention of this effect
before but not called blast radius.

BLASTER: Characterizing the Blast Radius of Rowhammer, 2023
https://www.research-collection.ethz.ch/handle/20.500.11850/617284
https://dramsec.ethz.ch/papers/blaster.pdf

"In particular, we show for the first time that BLASTER significantly
reduces the number of necessary activations to the victim-adjacent
aggressors using other aggressor rows that are up to four rows away
from the victim."

> My limited understanding of RowPress is that you should not keep the Row
> open
> for more than a page of data transfer (about ¼ of 7.8µs DDR4 limit). My
> bet is
> that this is a leakage issue on the bit line made sensitive by the word
> line.

Yes, from what I read the factors affecting Rowhammer vulnerability are:

1) DRAM chip temperature, 2) aggressor row active time,
and 3) victim DRAM cell’s physical location.

>> Also it may be data dependent - 0's bleed into adjacent 1's and 1's
>> into 0's.
>
> DRAMs are funny like this. Adjacent bit lines store data differently. Even
> bits store 0 as 0 and 1 as 1 while odd cells store 0 as 1 and 1 as 0. They
> do this so the sense amplified has a differential to sense, either the even
> cell or the odd cell is asserted on the bit line pair and the sense amp
> then
> has a differential to sense. One line goes up a little or down a little
> while
> the other bit line stays where it is.
>
>> And the threshold when it triggers has been changing as drams become more
>> dense. In 2014 when this was first encountered it took 139K activations.
>> By 2020 that was down to 4.8K.
>
>> So figuring out how much a row has been damaged is complicated,
>> and the window for detecting it is getting smaller.

EricP wrote:
> MitchAlsup wrote:
>
>> A long time ago We found a problem with one manufactures SRAM when the
>> same
>> row was hit >6,000 times, there was enough charge sharing that the
>> adjacent dynamic word decoder also fired so we had 2 or 3 word lines
>> active at the same time. We encountered this when a LD missed the cache
>> and was sent down
>> through NorthBridge, SouthBridge, onto another bus, finally out to the
>> device
>> and back, while the CPU was continuing to read the ICache every cycle.
>
> I think of this as aging: each activation ages the rows up to some distance
> by amounts depending on the distance due to charge migration.
>
> Originally it was found by activating rows immediately adjacent to the
> victim but then they looked and found it further out to +-4 rows.
> This effect appears to be called the Rowhammer "blast radius".
>
> This paper is from 2023 but I'm sure I've seen mention of this effect
> before but not called blast radius.
>
> BLASTER: Characterizing the Blast Radius of Rowhammer, 2023
> https://www.research-collection.ethz.ch/handle/20.500.11850/617284
> https://dramsec.ethz.ch/papers/blaster.pdf
>
> "In particular, we show for the first time that BLASTER significantly
> reduces the number of necessary activations to the victim-adjacent
> aggressors using other aggressor rows that are up to four rows away
> from the victim."

To elaborate a bit, as I understand it this means that if a dram
has a blast radius of +-3 and we take 7 rows A B C D E F G,
and assuming the aging factor is linear, then any read or refresh
of row D resets its age to 0 but ages C&E by 3, B&F by 2, A&G by 1.
If any row age total hits 15,000 its data dies.

This is why I thought canary bits might work, because they integrate the
sum of all adjacent activates while taking blast distance into account.
As long as the canary _reliably_ dies at age 12,000 and the data at 15,000
then the dram could transparently refresh the aged-out rows.

Michael S <already5chosen@yahoo.com> writes:
>On Wed, 31 Jan 2024 17:17:21 GMT
>anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>> The first paper on Rowhammer already outlined how the memory
>> controller could count how often adjacent DRAM rows are accessed and
>> thus weaken the row under consideration. This approach needs a little
>> adjustment for Double Rowhammer and not immediately neighbouring rows,
>> but otherwise seems to me to be the way to go.
>
>IMHO, all thise solutions are pure fantasy, because memory controller
>does not even know which rows are physically adjacent. POC authors
>typically run lengthy tests in order to figure it out.

Given that the attackers can find out, it is just a lack of
communication between DRAM manufacturers and memory controller
manufacturers that result in that ignorance. Not a valid excuse.

There is a standardization committee (JEDEC) that documents how
various DRAM types are accessed, refreshed etc. They put information
about that (and about RAM overclocking (XMP, Expo)) in the SPD ROMs of
the DIMMs, so they can also put information about line adjacency
there.

>> With autorefresh in
>> the DRAM devices these days, the DRAM manufacturers could implement
>> this on their own, without needing to coordinate with memory
>> controller designers. But apparently they think that the customers
>> don't care, so they can save the expense.
....
>They cared enough to implement the simplest of proposed solutions - TRR.
>Yes, it was quickly found insufficient, but at least there was a
>demonstration of good intentions.

Yes. However, looking at Table III of
<https://comsec.ethz.ch/wp-content/files/blacksmith_sp22.pdf>, there
seems to be significant differences between manufacturers A and D on
one hand, and B and C on the other, with exploits taking much longer
for B and C, and failing in some cases.

One may wonder if the DRAM manufacturers could have put their
physicists to the task of identifying the conditions under which bit
flips can occur, and identify the refreshes that are at least
necessary to prevent these conditions from occuring. If they have not
done so, or if they have not implemented the resulting recommendations
(or passed them to the memory controller people), a certain amount of
blame rests on them.

Anyway, never mind the blame, looking into the future, I find it
worrying that I did not find any mention of Rowhammer protection in
the specs of DIMMs when I last looked.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

EricP <ThatWouldBeTelling@thevillage.com> writes:
>MitchAlsup wrote:
>> Michael S wrote:
>>
>>> Original RH required very high hammering rate that certainly can't be
>>> achieved by playing with associativity of L3 cache.
>>
>>> Newer multiside hammering probably can do it in theory, but it would be
>>> very difficult in practice.
>>
>> The problem here is the fact that DRAMs do not use linear decoders, so
>> address X and address X+1 do not necessarily shared paired word lines.
>> The word lines could be as far as ½ the block away from each other.
>>
>> The DRAM decoders are faster and smaller when there is a grey-like-code
>> imposed on the logical-address to physical-word-line. This also happens
>> in SRAM decoders. Going back and looking at the most used logical to
>> physical mapping shows that while X and X+1 can (occasionally) be side
>> by side, X, X+1 and X+2 should never be 3 words lines in a row.
>
>A 16 Gb dram with 8kb rows has 2^21 = 2 million rows.
>So having a counter for each row is impractical.

A (say) 16-bit counter for each 8Kb row would be a 0.2% overhead.
Admittedly, if you just update the counter for a specific row and the
refresh all rows in the blast radius when a limit is reached, you
may get many more refreshes than the minimum necessary, but given that
normal programs usually do not hammer specific row ranges, the
additional refreshes may still be relatively few in non-attack
situations (and when being attacked, you prefer lower DRAM performance
to a successful attack).

Alternatively, a kind of cache could be used. Keep counts of N most
recently accessed rows, remove the row on refresh; when accessing a
row that has not been in the cache, evict the entry for the row with
the lowest count C, and set the count of the loaded row to C+1. When
a count (or ensemble of counts) reaches the limit, refresh every row.

This would take much less memory, but require finding the entry with
the lowest count. By dividing the cache into sets, this becomes more
realistic; upon reaching a limit, only the rows in the blast radius of
the lines in a set need to be refreshed.

>I was wondering if each row could have "canary" bit,
>a specially weakened bit that always flips early.
>This would also intrinsically handle the cases of effects
>falling off over the +-3 adjacent rows.
>
>Then a giant 2 million input OR gate would tell us if any row's
>canary had flipped.

Yes, doing it in analog has its charms. However, I see the following
difficulties:

* How do you measure whether a bit has flipped without refreshing it
and thus resetting the canary?

* To flip a bit in one direction, AFAIK the hammering rows have to
have a specific content. I guess with a blast radius of 4 rows on
each side, you could have 4 columns. Each row has a canary in one
of these columns and the three adjacent bits in this column are
attacker bits that have the value that is useful for effecting a bit
flip in a canary. Probably a more refined variant of this idea
would be necessary is necessary to deal with diagonal influence and
the non-uniform encoding of 0 and 1 in the DRAMs discussed somewhere
in this thread.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Anton Ertl wrote:

> Michael S <already5chosen@yahoo.com> writes:
>>On Wed, 31 Jan 2024 17:17:21 GMT
>>anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>>> The first paper on Rowhammer already outlined how the memory
>>> controller could count how often adjacent DRAM rows are accessed and
>>> thus weaken the row under consideration. This approach needs a little
>>> adjustment for Double Rowhammer and not immediately neighbouring rows,
>>> but otherwise seems to me to be the way to go.
>>
>>IMHO, all thise solutions are pure fantasy, because memory controller
>>does not even know which rows are physically adjacent. POC authors
>>typically run lengthy tests in order to figure it out.

> Given that the attackers can find out, it is just a lack of
> communication between DRAM manufacturers and memory controller
> manufacturers that result in that ignorance. Not a valid excuse.

> There is a standardization committee (JEDEC) that documents how
> various DRAM types are accessed, refreshed etc. They put information
> about that (and about RAM overclocking (XMP, Expo)) in the SPD ROMs of
> the DIMMs, so they can also put information about line adjacency
> there.

>>> With autorefresh in
>>> the DRAM devices these days, the DRAM manufacturers could implement
>>> this on their own, without needing to coordinate with memory
>>> controller designers. But apparently they think that the customers
>>> don't care, so they can save the expense.
> ....
>>They cared enough to implement the simplest of proposed solutions - TRR.
>>Yes, it was quickly found insufficient, but at least there was a
>>demonstration of good intentions.

> Yes. However, looking at Table III of
> <https://comsec.ethz.ch/wp-content/files/blacksmith_sp22.pdf>, there
> seems to be significant differences between manufacturers A and D on
> one hand, and B and C on the other, with exploits taking much longer
> for B and C, and failing in some cases.

> One may wonder if the DRAM manufacturers could have put their
> physicists to the task of identifying the conditions under which bit
> flips can occur, and identify the refreshes that are at least
> necessary to prevent these conditions from occuring. If they have not
> done so, or if they have not implemented the resulting recommendations
> (or passed them to the memory controller people), a certain amount of
> blame rests on them.

> Anyway, never mind the blame, looking into the future, I find it
> worrying that I did not find any mention of Rowhammer protection in
> the specs of DIMMs when I last looked.

My information is that they (DRAM mfgs) looked and said they could not
fix a problem that emanated from the DRAM controller.

> - anton

MitchAlsup wrote:
> EricP wrote:
>
>> MitchAlsup wrote:
>>> Anton Ertl wrote:
>>>
>>>
>>> Rowhammer happens when you beat on the same cache line multiple times
>>> {causing a charge sharing problem on the word lines. Every time you
>>> cause
>>> the DRAM to precharge (deActivate) you lose the count on how many times
>>> you have to bang on the same word line to disrupt the stored cells.
>>>
>>> So, the trick is to detect the RowHammering and insert refresh commands.
>
>> It's not just the immediately physically adjacent rows -
>> I think I read that the effect falls off for up to +-3 rows away.
>
> My understanding is that RowHammer has to access the same row multiple
> times
> to disrupt bits in an adjacent row. This sounds like a charge sharing
> problem.
> A long time ago We found a problem with one manufactures SRAM when the same
> row was hit >6,000 times, there was enough charge sharing that the
> adjacent dynamic word decoder also fired so we had 2 or 3 word lines
> active at the same time. We encountered this when a LD missed the cache
> and was sent down
> through NorthBridge, SouthBridge, onto another bus, finally out to the
> device
> and back, while the CPU was continuing to read the ICache every cycle.
>
> My limited understanding of RowPress is that you should not keep the Row
> open
> for more than a page of data transfer (about ¼ of 7.8µs DDR4 limit). My
> bet is
> that this is a leakage issue on the bit line made sensitive by the word
> line.

Ah I see from the RowPress paper that it is different from RowHammer.
RowHammer is based on activation counts and RowPress on activation time.
Previously papers had just said that activation time correlated with
bit flips and I guess everyone just assumed it was the same mechanism.
But the RowPress paper shows it affects different bits from RowHammer.
Also RowPress and RowHammer tend to flip in different directions,
RowHammer flips 0 to 1 and RowPress 1 to 0 (taking the true and anti
cell logic states into account). Possibly one is doing electron injection
and the other hole injection.

Anton Ertl wrote:

>
>>Then a giant 2 million input OR gate would tell us if any row's
>>canary had flipped.

> Yes, doing it in analog has its charms. However, I see the following
> difficulties:

> * How do you measure whether a bit has flipped without refreshing it
> and thus resetting the canary?

You know what its value should be and you raise hell when it is not as
expected. This may require 2 canary bits.

> * To flip a bit in one direction, AFAIK the hammering rows have to
> have a specific content. I guess with a blast radius of 4 rows on
> each side, you could have 4 columns. Each row has a canary in one
> of these columns and the three adjacent bits in this column are
> attacker bits that have the value that is useful for effecting a bit
> flip in a canary. Probably a more refined variant of this idea
> would be necessary is necessary to deal with diagonal influence and
> the non-uniform encoding of 0 and 1 in the DRAMs discussed somewhere
> in this thread.

> - anton

mitchalsup@aol.com (MitchAlsup) writes:
>Anton Ertl wrote:
>
>>
>>>Then a giant 2 million input OR gate would tell us if any row's
>>>canary had flipped.
>
>> Yes, doing it in analog has its charms. However, I see the following
>> difficulties:
>
>> * How do you measure whether a bit has flipped without refreshing it
>> and thus resetting the canary?
>
>You know what its value should be and you raise hell when it is not as
>expected.

So that is about detecting Rowhammer after the fact. Yes, you could
do that when the row is refreshed. The only problem is that by then
the attacker could have extracted the secret(s) with the
Rowhammer-based attack. Better than nothing, but still not a very
attractive approach.

I prefer a solution that detects that a row might suffer a bit flip
after several more accesses, and refreshes the row befor that happens.
And I don't think that this can be implemented with an analog canary
that works like a DRAM cell; but I am not a solid-state physicist,
maybe there is a way.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Anton Ertl wrote:

> mitchalsup@aol.com (MitchAlsup) writes:
>>Anton Ertl wrote:
>>
>>>
>>>>Then a giant 2 million input OR gate would tell us if any row's
>>>>canary had flipped.
>>
>>> Yes, doing it in analog has its charms. However, I see the following
>>> difficulties:
>>
>>> * How do you measure whether a bit has flipped without refreshing it
>>> and thus resetting the canary?
>>
>>You know what its value should be and you raise hell when it is not as
>>expected.

> So that is about detecting Rowhammer after the fact. Yes, you could
> do that when the row is refreshed. The only problem is that by then
> the attacker could have extracted the secret(s) with the
> Rowhammer-based attack. Better than nothing, but still not a very
> attractive approach.

> I prefer a solution that detects that a row might suffer a bit flip
> after several more accesses, and refreshes the row before that happens.
> And I don't think that this can be implemented with an analog canary
> that works like a DRAM cell; but I am not a solid-state physicist,
> maybe there is a way.

Sooner or later, designers will have to come to the realization that
an external DRAM controller can never guarantee everything every DRAM
actually needs to retain data under all conditions, and the DRAMs
are going to have to change the interface such that requests flow
in and results flow out based on the DRAM internal controller--much
like that of a SATA disk drive.

Let us face it, the DDR-6 interface model is based on the 16K-bit
DRAM chips from about 1979: RAS and CAS, it got speed up, pipelined,
double data rated, and each step added address bits to RAS and CAS.

I suspect when this happens, the DRAMs will partition the inbound
address into 3 or 4 sections, and use each section independently
Bank-Row-Column or block-bank-row-column.

In addition each building block will be internally self timed, no
external need to refresh the bank-row, and the only non access
command in the arsenal is power-down and power-up.

You can only put so much lipstick on a pig.

> - anton

Anton Ertl wrote:
> EricP <ThatWouldBeTelling@thevillage.com> writes:
>> MitchAlsup wrote:
>>> Michael S wrote:
>>>
>>>> Original RH required very high hammering rate that certainly can't be
>>>> achieved by playing with associativity of L3 cache.
>>>> Newer multiside hammering probably can do it in theory, but it would be
>>>> very difficult in practice.
>>> The problem here is the fact that DRAMs do not use linear decoders, so
>>> address X and address X+1 do not necessarily shared paired word lines.
>>> The word lines could be as far as ½ the block away from each other.
>>>
>>> The DRAM decoders are faster and smaller when there is a grey-like-code
>>> imposed on the logical-address to physical-word-line. This also happens
>>> in SRAM decoders. Going back and looking at the most used logical to
>>> physical mapping shows that while X and X+1 can (occasionally) be side
>>> by side, X, X+1 and X+2 should never be 3 words lines in a row.
>> A 16 Gb dram with 8kb rows has 2^21 = 2 million rows.
>> So having a counter for each row is impractical.
>
> A (say) 16-bit counter for each 8Kb row would be a 0.2% overhead.
> Admittedly, if you just update the counter for a specific row and the
> refresh all rows in the blast radius when a limit is reached, you
> may get many more refreshes than the minimum necessary, but given that
> normal programs usually do not hammer specific row ranges, the
> additional refreshes may still be relatively few in non-attack
> situations (and when being attacked, you prefer lower DRAM performance
> to a successful attack).

They said that the current threshold for causing flips in an immediate
neighbor is 4800 activations, but with a blast radius of +-4 that
can be in any of the 8 neighbors, so your counter threshold will have
to trigger refresh at 1/8 of that level or every 600 activations.

And as the dram features get smaller that threshold number will go down
and probably the blast radius will go up. So this could have scaling
issues in the future.

> Alternatively, a kind of cache could be used. Keep counts of N most
> recently accessed rows, remove the row on refresh; when accessing a
> row that has not been in the cache, evict the entry for the row with
> the lowest count C, and set the count of the loaded row to C+1. When
> a count (or ensemble of counts) reaches the limit, refresh every row.

That would be a CAM or assoc sram and would have to hold a large
number of entries. This would have to be in the memory controller.

> This would take much less memory, but require finding the entry with
> the lowest count. By dividing the cache into sets, this becomes more
> realistic; upon reaching a limit, only the rows in the blast radius of
> the lines in a set need to be refreshed.
>
>> I was wondering if each row could have "canary" bit,
>> a specially weakened bit that always flips early.
>> This would also intrinsically handle the cases of effects
>> falling off over the +-3 adjacent rows.
>>
>> Then a giant 2 million input OR gate would tell us if any row's
>> canary had flipped.
>
> Yes, doing it in analog has its charms. However, I see the following
> difficulties:
>
> * How do you measure whether a bit has flipped without refreshing it
> and thus resetting the canary?

The canary would have to be a little more complicated than a standard
storage cell because it has to compare the cell to the expected value
and then drive an output transistor to pull down a dynamic bit line
for a wired-OR of all the canaries in a bank.
Hopefully that would isolate the canary from its read bit line changes.

Fitting this into a dram row could be a problem.
This would all have the same height as a normal row to fit horizontally
along a dram row so it didn't bugger up the row spacing.

> * To flip a bit in one direction, AFAIK the hammering rows have to
> have a specific content. I guess with a blast radius of 4 rows on
> each side, you could have 4 columns. Each row has a canary in one
> of these columns and the three adjacent bits in this column are
> attacker bits that have the value that is useful for effecting a bit
> flip in a canary. Probably a more refined variant of this idea
> would be necessary is necessary to deal with diagonal influence and
> the non-uniform encoding of 0 and 1 in the DRAMs discussed somewhere
> in this thread.
>
> - anton

Each canary might be 3 cells with alternating patterns,
even row numbers are inited to 010 and odd rows to 101,
positioned in vertical columns. Presumably this would put
the maximum and a predictable stress on the center bit.
Since the expected value for each row is hard wired it is
easy to test if it changes.

Anton Ertl wrote:

> EricP <ThatWouldBeTelling@thevillage.com> writes:
>>MitchAlsup wrote:
>>> Michael S wrote:
>>>
>>>> Original RH required very high hammering rate that certainly can't be
>>>> achieved by playing with associativity of L3 cache.
>>>
>>>> Newer multiside hammering probably can do it in theory, but it would be
>>>> very difficult in practice.
>>>
>>> The problem here is the fact that DRAMs do not use linear decoders, so
>>> address X and address X+1 do not necessarily shared paired word lines.
>>> The word lines could be as far as ½ the block away from each other.
>>>
>>> The DRAM decoders are faster and smaller when there is a grey-like-code
>>> imposed on the logical-address to physical-word-line. This also happens
>>> in SRAM decoders. Going back and looking at the most used logical to
>>> physical mapping shows that while X and X+1 can (occasionally) be side
>>> by side, X, X+1 and X+2 should never be 3 words lines in a row.
>>
>>A 16 Gb dram with 8kb rows has 2^21 = 2 million rows.
>>So having a counter for each row is impractical.

> A (say) 16-bit counter for each 8Kb row would be a 0.2% overhead.

You are comparing a 16-bit incrementor and its associated flip-flop
with a single transistor divided by the number of them in a word. My
guess is that you are off by 20× (should be close to 4%)

mitchalsup@aol.com (MitchAlsup1) writes:
>Anton Ertl wrote:
>
>> EricP <ThatWouldBeTelling@thevillage.com> writes:
>>>A 16 Gb dram with 8kb rows has 2^21 = 2 million rows.
>>>So having a counter for each row is impractical.
>
>> A (say) 16-bit counter for each 8Kb row would be a 0.2% overhead.
>
>You are comparing a 16-bit incrementor and its associated flip-flop
>with a single transistor divided by the number of them in a word.

I was thinking about counting each access only when the cache line is
accessed. Then there needs to be only one incrementor per bank, and
the counter can be stored in DRAM like the payload data.

But thinking about it again, I wonder how counters would be reset.
Maybe, when the counter reaches the limit, all lines in its blast
radius are refereshed, and the counter of the present line is reset to
0.

Another disadvantage would be that we have to make decisions about
possible rowhammering only based on one counter, and have to trigger
refreshes of all lines in the blast radius based on worst-case
scenarios (i.e., assuming that other rows in the blast radius have any
count up to the limit).

Both disadvantages lead to far more refreshes than necessary to
prevent Rowhammer, but that approach may still be good enough.

Alternatively, if you want to invest more, one could follow your idea
and have counter SRAM (maybe including counting circuitry) for each
row; each refresh of a line would increment the counters in the blast
radius by an appropriate amount, and when a counter reaches its limit,
it would trigger a refresh of that row.

>My guess is that you are off by 20× (should be close to 4%)

Even 4% is not "impractical".

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

EricP <ThatWouldBeTelling@thevillage.com> writes:
>Anton Ertl wrote:
>> A (say) 16-bit counter for each 8Kb row would be a 0.2% overhead.
>> Admittedly, if you just update the counter for a specific row and the
>> refresh all rows in the blast radius when a limit is reached, you
>> may get many more refreshes than the minimum necessary, but given that
>> normal programs usually do not hammer specific row ranges, the
>> additional refreshes may still be relatively few in non-attack
>> situations (and when being attacked, you prefer lower DRAM performance
>> to a successful attack).
>
>They said that the current threshold for causing flips in an immediate
>neighbor is 4800 activations, but with a blast radius of +-4 that
>can be in any of the 8 neighbors, so your counter threshold will have
>to trigger refresh at 1/8 of that level or every 600 activations.

So only 10 bits of counter are necessary, reducing the overhead to
0.125%:-).

>And as the dram features get smaller that threshold number will go down
>and probably the blast radius will go up. So this could have scaling
>issues in the future.

Yes.

>> Alternatively, a kind of cache could be used. Keep counts of N most
>> recently accessed rows, remove the row on refresh; when accessing a
>> row that has not been in the cache, evict the entry for the row with
>> the lowest count C, and set the count of the loaded row to C+1. When
>> a count (or ensemble of counts) reaches the limit, refresh every row.
>
>That would be a CAM or assoc sram and would have to hold a large
>number of entries. This would have to be in the memory controller.

Possibly. Recent DRAMs also support self-refresh (to allow powering
down the connection to the memory controller); this kind of stuff
could also be on the DRAM device, avoiding all the problems that
memory controllers have with knowing the characteristics of the DRAM
device.

>> * How do you measure whether a bit has flipped without refreshing it
>> and thus resetting the canary?
>
>The canary would have to be a little more complicated than a standard
>storage cell because it has to compare the cell to the expected value

Maybe capacitative coupling (as used for flash AFAIK) could be used to
measure the contents of the canary without discharging it. There
still would be tunneling, as in Rowhammer itself, but I guess one
could account for that.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

mitchalsup@aol.com (MitchAlsup) writes:
>Sooner or later, designers will have to come to the realization that
>an external DRAM controller can never guarantee everything every DRAM
>actually needs to retain data under all conditions, and the DRAMs
>are going to have to change the interface such that requests flow
>in and results flow out based on the DRAM internal controller--much
>like that of a SATA disk drive.
>
>Let us face it, the DDR-6 interface model is based on the 16K-bit
>DRAM chips from about 1979: RAS and CAS, it got speed up, pipelined,
>double data rated, and each step added address bits to RAS and CAS.

I don't know about DDR6, but the DDR5 command interface is
significantly more complex
<https://en.wikipedia.org/wiki/DDR5#Command_encoding> than early
asynchronous DRAM.

>I suspect when this happens, the DRAMs will partition the inbound
>address into 3 or 4 sections, and use each section independently
>Bank-Row-Column or block-bank-row-column.

Looking at the commands from the link above, Activate already
transfers the row in two pieces, and the read and write are also
transferred in two pieces.

>In addition each building block will be internally self timed, no
>external need to refresh the bank-row, and the only non access
>command in the arsenal is power-down and power-up.

Self-refresh is already there, but AFAIK only used when processing is
suspended.

However, there are many commands, many more than in the 16kx1 DRAMs of
old. What would make them go in the direction of simplifying the
interface? The hardest part these days seems to be getting the high
transfer rates to work, the rest of the interface is probably
comparatively easy.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Anton Ertl wrote:

> mitchalsup@aol.com (MitchAlsup) writes:
>>Sooner or later, designers will have to come to the realization that
>>an external DRAM controller can never guarantee everything every DRAM
>>actually needs to retain data under all conditions, and the DRAMs
>>are going to have to change the interface such that requests flow
>>in and results flow out based on the DRAM internal controller--much
>>like that of a SATA disk drive.
>>
>>Let us face it, the DDR-6 interface model is based on the 16K-bit
>>DRAM chips from about 1979: RAS and CAS, it got speed up, pipelined,
>>double data rated, and each step added address bits to RAS and CAS.

> I don't know about DDR6, but the DDR5 command interface is
> significantly more complex
> <https://en.wikipedia.org/wiki/DDR5#Command_encoding> than early
> asynchronous DRAM.

>>I suspect when this happens, the DRAMs will partition the inbound
>>address into 3 or 4 sections, and use each section independently
>>Bank-Row-Column or block-bank-row-column.

> Looking at the commands from the link above, Activate already
> transfers the row in two pieces, and the read and write are also
> transferred in two pieces.

>>In addition each building block will be internally self timed, no
>>external need to refresh the bank-row, and the only non access
>>command in the arsenal is power-down and power-up.

> Self-refresh is already there, but AFAIK only used when processing is
> suspended.

My DRAM controller (AMD Opteron rev G) used ACTivate commands instead of
refresh commands to refresh rows in DDR2 DRAM. The timings were better.
It just did not come back and ask for data from the RASed row.

> However, there are many commands, many more than in the 16kx1 DRAMs of
> old. What would make them go in the direction of simplifying the
> interface?

Pins that are less expensive.
> The hardest part these days seems to be getting the high
> transfer rates to work, the rest of the interface is probably
> comparatively easy.

This is from DDR4 and onward where one has to control drive strength
and clock edge offsets (with a DLL) to transfer data that fast.

> - anton

Anton Ertl wrote:
> mitchalsup@aol.com (MitchAlsup1) writes:
>> Anton Ertl wrote:
>>
>>> EricP <ThatWouldBeTelling@thevillage.com> writes:
>>>> A 16 Gb dram with 8kb rows has 2^21 = 2 million rows.
>>>> So having a counter for each row is impractical.
>>> A (say) 16-bit counter for each 8Kb row would be a 0.2% overhead.
>> You are comparing a 16-bit incrementor and its associated flip-flop
>> with a single transistor divided by the number of them in a word.
>
> I was thinking about counting each access only when the cache line is
> accessed. Then there needs to be only one incrementor per bank, and
> the counter can be stored in DRAM like the payload data.

Dram row reads are destructive so a single row activate command
internally has three cycles: read, sense and redrive, restore.

The counter could be stored in the dram cells and the
N-bit incrementer integrated into the bit line sense amp latches,
such that when the activate command does its restore cycle
it writes back the incremented counter.
The incremented counter would also be available in the row buffer.

Since the next precharge can't happen for 40-50 ns we have some
time to decide what to do next.

> But thinking about it again, I wonder how counters would be reset.
> Maybe, when the counter reaches the limit, all lines in its blast
> radius are refereshed, and the counter of the present line is reset to
> 0.

On a row read if the counter hits its threshold limit the restore
cycle writes back a count of 0, otherwise the incremented counter.

The problem is with the +-4 blast radius refreshes. Each of those refreshes
ages its neighbors which we need to track, so we can't reset those counters.
This could cause a write amplification where each refresh repeatedly
triggers 4 more refreshes.

It is possible to use the counter as a state machine.
Something like...
1) For normal, periodic refreshes set count to some initial value.
2) For reads increment count and if carry-out then reset to initial value
and schedule immediate blast refresh of +-4 neighbor rows.
3) For blast row refresh increment count but don't check for overflow.
If there is a count overflow it gets detected on its next row read.

> Another disadvantage would be that we have to make decisions about
> possible rowhammering only based on one counter, and have to trigger
> refreshes of all lines in the blast radius based on worst-case
> scenarios (i.e., assuming that other rows in the blast radius have any
> count up to the limit).

Yes, unless there is a way to infer the total counts for the neighbors.
Bloom filter?
But see below.

> Both disadvantages lead to far more refreshes than necessary to
> prevent Rowhammer, but that approach may still be good enough.

Lets see how bad this is.

The single line threshold of 4800 and blast radius of 8 = 600 trigger count.
That triggers an extra 8 row refreshes, so 8/600 = 1.3% overhead.
And the whole dram is refreshed every 64 ms reseting all the counters
so the counts are not cumulative.

That overhead is only going to grow as dram density increases.

EricP wrote:

> Anton Ertl wrote:
>> mitchalsup@aol.com (MitchAlsup1) writes:
>>> Anton Ertl wrote:

>> Both disadvantages lead to far more refreshes than necessary to
>> prevent Rowhammer, but that approach may still be good enough.

Would you rather have a few more refreshes or a few more ECC repairs ?!?
with the potential for a few ECC repair fails ?!!?

> Lets see how bad this is.

> The single line threshold of 4800 and blast radius of 8 = 600 trigger count.
> That triggers an extra 8 row refreshes, so 8/600 = 1.3% overhead.
> And the whole dram is refreshed every 64 ms reseting all the counters
> so the counts are not cumulative.

I think what RowPress tells us that waiting 60± ms and then refreshing every row
is worse for data retention than spreading the refreshes out over the 64ms max
interval rather evenly.

> That overhead is only going to grow as dram density increases.

So are all the attack vectors.

Anton Ertl wrote:

> EricP <ThatWouldBeTelling@thevillage.com> writes:
>>Anton Ertl wrote:
>>> A (say) 16-bit counter for each 8Kb row would be a 0.2% overhead.
>>> Admittedly, if you just update the counter for a specific row and the
>>> refresh all rows in the blast radius when a limit is reached, you
>>> may get many more refreshes than the minimum necessary, but given that
>>> normal programs usually do not hammer specific row ranges, the
>>> additional refreshes may still be relatively few in non-attack
>>> situations (and when being attacked, you prefer lower DRAM performance
>>> to a successful attack).
>>
>>They said that the current threshold for causing flips in an immediate
>>neighbor is 4800 activations, but with a blast radius of +-4 that
>>can be in any of the 8 neighbors, so your counter threshold will have
>>to trigger refresh at 1/8 of that level or every 600 activations.

> So only 10 bits of counter are necessary, reducing the overhead to
> 0.125%:-).

>>And as the dram features get smaller that threshold number will go down
>>and probably the blast radius will go up. So this could have scaling
>>issues in the future.

> Yes.

If the DRAM manufactures placed a Faraday shield over the DRAM arrays
{A gound plane} the blast radius goes from a linear charge sharing issue
to a quadratic charge sharing issue. Such a ground plane is a layer of
metal with a single <never changing> voltage on it. This might change the
blast radius from 8 to 2.

{{We did this kind of things for SRAM so we could run large signal count
busses over the SRAM arrays.}}

> - anton

mitchalsup@aol.com (MitchAlsup1) writes:
>EricP wrote:
>
>> Anton Ertl wrote:
>>> mitchalsup@aol.com (MitchAlsup1) writes:
>>>> Anton Ertl wrote:
>
>>> Both disadvantages lead to far more refreshes than necessary to
>>> prevent Rowhammer, but that approach may still be good enough.
>
>Would you rather have a few more refreshes or a few more ECC repairs ?!?
>with the potential for a few ECC repair fails ?!!?

That's not the issue at hand here. The issue at hand here is whether
the relatively cheap mechanism I described has an acceptable number of
additional refreshes during normal operation, or whether a more
expensive (in terms of area) mechanism is needed to fix Rowhammer.

Concerning ECC, many computers do not have ECC memory, and for those
that have it, ECC does not reliably fix Rowhammer; if it did, the fix
would be simple: Use ECC, which is a good idea anyway, even if it
costs 25% more chips in case of DDR5 DIMMs.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

MitchAlsup1 wrote:
> EricP wrote:
>
>> Anton Ertl wrote:
>>> mitchalsup@aol.com (MitchAlsup1) writes:
>>>> Anton Ertl wrote:
>
>>> Both disadvantages lead to far more refreshes than necessary to
>>> prevent Rowhammer, but that approach may still be good enough.
>
> Would you rather have a few more refreshes or a few more ECC repairs ?!?
> with the potential for a few ECC repair fails ?!!?

I believe Rowhammer and RowPress can flip many bits at once.
Too many for SECDED.

>> Lets see how bad this is.
>
>> The single line threshold of 4800 and blast radius of 8 = 600 trigger
>> count.
>> That triggers an extra 8 row refreshes, so 8/600 = 1.3% overhead.
>> And the whole dram is refreshed every 64 ms reseting all the counters
>> so the counts are not cumulative.
>
> I think what RowPress tells us that waiting 60± ms and then refreshing
> every row
> is worse for data retention than spreading the refreshes out over the
> 64ms max
> interval rather evenly.

Would any memory controller that would do that,
refresh the whole dram in one big burst instead of periodically by row?
I would expect doing so would introduce big stalls into memory access.

64 ms / 8192 rows per block = 7.8125 us row interval.
Lets say 50 ns row refresh time.
So thats either 50 ns every 7.8 us
verses 8192*50 ns = 409.6 us memory stall every 64 ms.

EricP wrote:

> MitchAlsup1 wrote:
>> EricP wrote:
>>
>>> Anton Ertl wrote:
>>>> mitchalsup@aol.com (MitchAlsup1) writes:
>>>>> Anton Ertl wrote:
>>
>>>> Both disadvantages lead to far more refreshes than necessary to
>>>> prevent Rowhammer, but that approach may still be good enough.
>>
>> Would you rather have a few more refreshes or a few more ECC repairs ?!?
>> with the potential for a few ECC repair fails ?!!?

> I believe Rowhammer and RowPress can flip many bits at once.
> Too many for SECDED.

>>> Lets see how bad this is.
>>
>>> The single line threshold of 4800 and blast radius of 8 = 600 trigger
>>> count.
>>> That triggers an extra 8 row refreshes, so 8/600 = 1.3% overhead.
>>> And the whole dram is refreshed every 64 ms reseting all the counters
>>> so the counts are not cumulative.
>>
>> I think what RowPress tells us that waiting 60± ms and then refreshing
>> every row
>> is worse for data retention than spreading the refreshes out over the
>> 64ms max
>> interval rather evenly.

> Would any memory controller that would do that,
> refresh the whole dram in one big burst instead of periodically by row?
> I would expect doing so would introduce big stalls into memory access.

> 64 ms / 8192 rows per block = 7.8125 us row interval.

My DRAM controller (Opteron RevF) had a timer set about 7µs and if the
back was active it would allow REF to slip. But on a second timer event
it would interrupt data transfer and induce 2 refreshes to catch up. In
general, this worked well as it almost never happened.

> Lets say 50 ns row refresh time.
> So thats either 50 ns every 7.8 us

A DDR5 at 6GBits/s transmits a 4096 byte page in 5µs.

When one changes page boundaries the HoB address bits are essentially
randomized by the TLB:: why not just close the row at that point ?

> verses 8192*50 ns = 409.6 us memory stall every 64 ms.

On Sun, 11 Feb 2024 19:57:34 +0000
mitchalsup@aol.com (MitchAlsup1) wrote:

> EricP wrote:
>
> > MitchAlsup1 wrote:
> >> EricP wrote:
> >>
> >>> Anton Ertl wrote:
> >>>> mitchalsup@aol.com (MitchAlsup1) writes:
> >>>>> Anton Ertl wrote:
> >>
> >>>> Both disadvantages lead to far more refreshes than necessary to
> >>>> prevent Rowhammer, but that approach may still be good enough.
> >>
> >> Would you rather have a few more refreshes or a few more ECC
> >> repairs ?!? with the potential for a few ECC repair fails ?!!?
>
> > I believe Rowhammer and RowPress can flip many bits at once.
> > Too many for SECDED.
>
> >>> Lets see how bad this is.
> >>
> >>> The single line threshold of 4800 and blast radius of 8 = 600
> >>> trigger count.
> >>> That triggers an extra 8 row refreshes, so 8/600 = 1.3% overhead.
> >>> And the whole dram is refreshed every 64 ms reseting all the
> >>> counters so the counts are not cumulative.
> >>
> >> I think what RowPress tells us that waiting 60± ms and then
> >> refreshing every row
> >> is worse for data retention than spreading the refreshes out over
> >> the 64ms max
> >> interval rather evenly.
>
> > Would any memory controller that would do that,
> > refresh the whole dram in one big burst instead of periodically by
> > row? I would expect doing so would introduce big stalls into memory
> > access.
>
> > 64 ms / 8192 rows per block = 7.8125 us row interval.
>
> My DRAM controller (Opteron RevF) had a timer set about 7µs and if the
> back was active it would allow REF to slip. But on a second timer
> event it would interrupt data transfer and induce 2 refreshes to
> catch up. In general, this worked well as it almost never happened.
>
> > Lets say 50 ns row refresh time.
> > So thats either 50 ns every 7.8 us
>
> A DDR5 at 6GBits/s transmits a 4096 byte page in 5µs.
>
> When one changes page boundaries the HoB address bits are essentially
> randomized by the TLB:: why not just close the row at that point ?
>
> > verses 8192*50 ns = 409.6 us memory stall every 64 ms.

On Sun, 11 Feb 2024 19:57:34 +0000
mitchalsup@aol.com (MitchAlsup1) wrote:

> EricP wrote:
>
> > MitchAlsup1 wrote:
> >> EricP wrote:
> >>
> >>> Anton Ertl wrote:
> >>>> mitchalsup@aol.com (MitchAlsup1) writes:
> >>>>> Anton Ertl wrote:
> >>
> >>>> Both disadvantages lead to far more refreshes than necessary to
> >>>> prevent Rowhammer, but that approach may still be good enough.
> >>
> >> Would you rather have a few more refreshes or a few more ECC
> >> repairs ?!? with the potential for a few ECC repair fails ?!!?
>
> > I believe Rowhammer and RowPress can flip many bits at once.
> > Too many for SECDED.
>
> >>> Lets see how bad this is.
> >>
> >>> The single line threshold of 4800 and blast radius of 8 = 600
> >>> trigger count.
> >>> That triggers an extra 8 row refreshes, so 8/600 = 1.3% overhead.
> >>> And the whole dram is refreshed every 64 ms reseting all the
> >>> counters so the counts are not cumulative.
> >>
> >> I think what RowPress tells us that waiting 60± ms and then
> >> refreshing every row
> >> is worse for data retention than spreading the refreshes out over
> >> the 64ms max
> >> interval rather evenly.
>
> > Would any memory controller that would do that,
> > refresh the whole dram in one big burst instead of periodically by
> > row? I would expect doing so would introduce big stalls into memory
> > access.
>
> > 64 ms / 8192 rows per block = 7.8125 us row interval.
>
> My DRAM controller (Opteron RevF) had a timer set about 7µs and if the
> back was active it would allow REF to slip. But on a second timer
> event it would interrupt data transfer and induce 2 refreshes to
> catch up. In general, this worked well as it almost never happened.
>
> > Lets say 50 ns row refresh time.
> > So thats either 50 ns every 7.8 us
>
> A DDR5 at 6GBits/s transmits a 4096 byte page in 5µs.
>

DDR5 channel is 32-bit.
4096B/(4B/T * 6e9 T/s) = 0.171 usec.
Or for more 0.204 usec for more realistic rate of 5e9 T/s

> When one changes page boundaries the HoB address bits are essentially
> randomized by the TLB:: why not just close the row at that point ?
>

Because memory controller is not aware of CPU page boundaries.
Besides, in aarch64 world 16KB pages are rather common. And in x86
world "transparent huge pages" are rather common.

> > verses 8192*50 ns = 409.6 us memory stall every 64 ms.

Michael S <already5chosen@yahoo.com> writes:
>On Sun, 11 Feb 2024 19:57:34 +0000
>mitchalsup@aol.com (MitchAlsup1) wrote:

>Because memory controller is not aware of CPU page boundaries.
>Besides, in aarch64 world 16KB pages are rather common. And in x86
>world "transparent huge pages" are rather common.

AArch64 supports translation granules of 4k, 16k and 64k. 4K
and 64K are the most common. While the architecture defines
16k, an implementation is free to not support it and I'm not aware of any
widespread usage.