“Capabilities” are an old idea for doing memory protection by storing the
access rights in unforgeable descriptors that are given to authorized
processes. This way, there is no need for the traditional unprivileged-
versus-privileged-processor-mode concept; process A can have privileged
access to memory region X but not Y, while process B can have privileged
access to memory region Y but not X, so neither is “more” privileged than
the other: each one is trusted with just a limited set of privileged
functions.

The idea fell out of use because of performance issues. But in these more
security-conscious times, the overhead seems more and more like a
reasonable price to pay for the greater control it offers. There is a
project called CHERI, whose concepts have been implemented in Arm’s
“Morello” chip.

<https://www.theregister.com/2022/07/26/cheri_computer_runs_kde/>

I have been in CPU design for a very long time. I did a HS level
design (calculator) in 1968 3 years before the Bomar Brain, did
a #60 design in college as a Jr, and started doing professional
designs (Mc 88100) in 1983.

With all this background and long term in this career, I can say
without a trace of doubt, that I am not <yet> smart enough to do
a capabilities ISA/system and get it out the door without errors.

On the other hand, My 66000 Architecture is immune to most attack
strategies now in vogue:: Return Oriented Programming, RowHammer,
Spectré, GOT overwrites, Buffer Overflows,... All without having
any semblance of capabilities; and all without any performance
degradations other than typical cache and TLB effects.

On 2024-03-08 5:38 p.m., Lawrence D'Oliveiro wrote:
> “Capabilities” are an old idea for doing memory protection by storing the
> access rights in unforgeable descriptors that are given to authorized
> processes. This way, there is no need for the traditional unprivileged-
> versus-privileged-processor-mode concept; process A can have privileged
> access to memory region X but not Y, while process B can have privileged
> access to memory region Y but not X, so neither is “more” privileged than
> the other: each one is trusted with just a limited set of privileged
> functions.
>
> The idea fell out of use because of performance issues. But in these more
> security-conscious times, the overhead seems more and more like a
> reasonable price to pay for the greater control it offers. There is a
> project called CHERI, whose concepts have been implemented in Arm’s
> “Morello” chip.
>
> <https://www.theregister.com/2022/07/26/cheri_computer_runs_kde/>

I may have a simple capabilities system going in with the system-on-chip
for Bigfoot. Rather than tag every memory word, there is a tag per 64kB
memory page. The tags can only be altered by the OS. The tag contains a
key and privilege level, so the page access is restricted based on that.
But I have the read-write-execute access rights in the page table
because there may be more than one user app accessing the page and they
may have different access rights. The combination of the tag in the
memory system and the access rights from the page table are used.

I gather that capabilities are generally fine-grained, and capability
pointers would be generated and handed out by the OS. What happens when
a pointer is incremented? A call to the OS to do so?
I think a tag for every word would be costly. It might be easier to tag
fetch units. Data being fetched in cache-line quantities.

On Fri, 8 Mar 2024 21:15:29 -0500, Robert Finch wrote:

> I gather that capabilities are generally fine-grained, and capability
> pointers would be generated and handed out by the OS. What happens when
> a pointer is incremented?

Each capability is a descriptor, describing a range of memory, not
(necessarily) just one address. So it is valid to use that to address any
area within the range.

On 2024-03-08 9:25 p.m., Lawrence D'Oliveiro wrote:
> On Fri, 8 Mar 2024 21:15:29 -0500, Robert Finch wrote:
>
>> I gather that capabilities are generally fine-grained, and capability
>> pointers would be generated and handed out by the OS. What happens when
>> a pointer is incremented?
>
> Each capability is a descriptor, describing a range of memory, not
> (necessarily) just one address. So it is valid to use that to address any
> area within the range.

Yeah, I got that after a little more reading. It looks a bit like a
segmentation system. It adds quite a bit to the instruction set,
doubling up on the load and store instructions to use capabilities and
to use ordinary pointers. It also needs wider registers. It is something
else to experiment with. Tags can be incorporated into the current
project for instructions relatively easily. But adding the extra tag bit
for data is trickier. It is desirable not to need to read two different
memory locations on every access. I suppose "block" capabilities could
be used instead of fine-grained. In a sense with the compressed format
for the bounds things are blocked.

On 3/8/2024 7:51 PM, MitchAlsup1 wrote:
> I have been in CPU design for a very long time. I did a HS level design
> (calculator) in 1968 3 years before the Bomar Brain, did
> a #60 design in college as a Jr, and started doing professional
> designs (Mc 88100) in 1983.
>
> With all this background and long term in this career, I can say
> without a trace of doubt, that I am not <yet> smart enough to do
> a capabilities ISA/system and get it out the door without errors.
>

Yeah.

As I see it, they seem to have a few drawbacks:
Overheads (in terms of performance and memory);
Not foolproof as a security model.

Most of this is assuming an approach similar to CHERI (rather than, say,
one based mostly on opaque handles or similar, which is its own thing).

A "practical" implementation of a C runtime on top of a capability
system is likely to have internal security holes, almost as a necessity
of implementing the C runtime (and giving typical expected C behaviors).

The big concern is the possibility of gaining access to "do anything"
capabilities:
It is unclear if a generally useful C runtime can be implemented without
them (absent imposing additional restrictions on the language, and thus
making portability harder);
The moment an application can get access to such a capability, a
security scheme based on capabilities is blown;
Within the existing model, any capability which is reachable by the C
runtime, is indirectly reachable from the program being run (provided it
can figure out a path to get access to such a capability).

If one disallows "unsafe" free-form pointer casting, this could
potentially eliminate this route, but a lot of existing C code would
break as well.

It seems like, if people can master the trickery needed to figure out
how to exploit buffer overflows (well proven at this point), they
already have the tools needed to figure out how to break through a
capability system in such an implementation.

Granted, it would be "harder" than it would be in a conventional flat
address space; but to claim it as a security feature for unsafe or
untrusted code, requires something stronger than, well, "you need to
exploit some arcane black-magic series of pointer casts and dereferences
or similar to get at what you really want". One needs a mechanism where
no such paths are possible (either because access is entirely prevented,
or the security-breaking capabilities do not exist).

Well, either this, or turn most non-trivial runtime calls (such as
"malloc()") into system calls.

But, OTOH, if they are running a mostly unmodified GLIBC or Newlib, such
an exploit path almost certainly exists.

Well, unless someone knows something I don't, and there is a way to pull
all this off in a way that doesn't have any implicit holes in the C
implementation (and also doesn't render the language "basically unusable").

Though, making capability-based security work might be easier if one
were using exclusively languages where "type safely" and "memory safety"
were the default option (say, it would probably work much better with a
language like C# or Java). But, in this case, the main "powers" of the
capability system would mostly be rendered moot.

If one passed them off as mostly:
Can be used to prevent buffer overflow exploits;
Can be useful to detect memory-access bugs as soon as they happen;
Avoids causing programs to be "slow as molasses" when using them;
...

This is at least more in line with what seems realistic.

But, in this latter case, a full capability system is almost a little
overkill.

Though, a watered down variant, such as "hardware-assisted
bounds-checked pointers" can be useful (and may also use a similar
encoding scheme to a capability system).

But, one can cut a lot of corners to make things cheaper (such as no
memory tagging, minimal new special-purpose instructions, ...).
But, as such, can't claim them as being capabilities.

I experimented with two schemes:
64 bit pointers:
(47: 0): Address
(63:48): Bounds Tag
(55:48): Size, as a E5.F3 microfloat
(59:56): Offset Bias (Same exponent as microfloat)
128 bit pointers:
( 47: 0): Address (Low)
( 63: 48): Bounds Tag (A)
(111: 64): Address (High)
(127:112): Bounds Tag (B)

There were specialized instructions for things like adjusting pointers,
and for checking an access for bounds violation (which would then raise
an exception), ...

The bounds checking was, at best, a coarse approximation (and trying to
take a pointer out-of-bounds would generally cause it to
"self-destruct", partly as the scheme couldn't represent out-of-bounds
addresses).

When using it for stack and global arrays, a partial option was to pad
the arrays (it would be padded up to a size and alignment that could be
represented with the tag; with the in-memory storage representing the
"worst case" bounds). Likewise, because the bounds were approximate, it
was still often possible to go "slightly out of bounds" before the fault
tripped, but the padding scheme allowed making it so that "nothing of
relevance" exists in this "slightly out-of-bounds" space.

So, say, with this feature enabled, nearly all such arrays were roughly
25% bigger in memory due to the padding constraints.

As I see it, this scheme similarly offers no additional protection
against malicious code, but can offer protection against buffer overflow
exploits (and was semi-useful as a debugging feature; well, after fixing
up the storm of subtle out-of-bounds accesses that may have been lurking
in the code).

Meanwhile, more traditional mechanisms for untrusted code, like running
them in a restrictive sandbox, are likely still to be workable (provided
there is no real way for the program being run to break free of the
sandbox).

> On the other hand, My 66000 Architecture is immune to most attack
> strategies now in vogue:: Return Oriented Programming, RowHammer,
> Spectré, GOT overwrites, Buffer Overflows,... All without having
> any semblance of capabilities; and all without any performance
> degradations other than typical cache and TLB effects.

I had decided to instead split the problem into sub-parts:
The possibility of page-level ACL checking;
The possibility of bounds-checked pointers;
More conventional "tried and true" methods.

And, also limit the scope of the claims:

Page-level ACL checks:
A task can only access pages for which access has been granted to it via
the ACL (to the task's PID, or via an additional key in the keyring).

This mostly assumes that the kernel arbitrates what has access to what.

Bound's checked pointers:
If the pointers are used "correctly", any out-of-bounds accesses will
trap (and works in a way that is mostly invisible to normal C code, but
does have a non-zero ABI and performance impact).

Mostly sufficient to help protect against things like buffer-overflow
exploits, but would be entirely ineffective against malicious code.

Granted, my existing "OS" is still painfully weak in these areas, as
memory-protection is still virtually non-existant (not entirely moved
programs to usemode, since this would still mean trying to go and finish
porting all the test programs over to TKGDI and eliminating the
remaining use of bare hardware interfaces).

Though, at present, this is mostly programs like ROTT and similar.

....

mitchalsup@aol.com (MitchAlsup1) writes:
>I have been in CPU design for a very long time. I did a HS level
>design (calculator) in 1968 3 years before the Bomar Brain, did
>a #60 design in college as a Jr, and started doing professional
>designs (Mc 88100) in 1983.
>
>With all this background and long term in this career, I can say
>without a trace of doubt, that I am not <yet> smart enough to do
>a capabilities ISA/system and get it out the door without errors.
>
>On the other hand, My 66000 Architecture is immune to most attack
>strategies now in vogue:: Return Oriented Programming, RowHammer,
>Spectré, GOT overwrites, Buffer Overflows,... All without having
>any semblance of capabilities; and all without any performance
>degradations other than typical cache and TLB effects.

On the gripping hand, the Burroughs Large systems capability based
design is still processing data almost sixty years after the original
B5500 was introduced.

There CHERI designs on silicon in existence

https://www.morello-project.org/

On 3/9/2024 9:09 AM, Scott Lurndal wrote:
> mitchalsup@aol.com (MitchAlsup1) writes:
>> I have been in CPU design for a very long time. I did a HS level
>> design (calculator) in 1968 3 years before the Bomar Brain, did
>> a #60 design in college as a Jr, and started doing professional
>> designs (Mc 88100) in 1983.
>>
>> With all this background and long term in this career, I can say
>> without a trace of doubt, that I am not <yet> smart enough to do
>> a capabilities ISA/system and get it out the door without errors.
>>
>> On the other hand, My 66000 Architecture is immune to most attack
>> strategies now in vogue:: Return Oriented Programming, RowHammer,
>> Spectré, GOT overwrites, Buffer Overflows,... All without having
>> any semblance of capabilities; and all without any performance
>> degradations other than typical cache and TLB effects.
>
> On the gripping hand, the Burroughs Large systems capability based
> design is still processing data almost sixty years after the original
> B5500 was introduced.
>
> There CHERI designs on silicon in existence
>
> https://www.morello-project.org/

It is doable, at least.

Main open question is if they can deliver enough on their claims in a
way that justifies the cost of the memory tagging (eg, where one needs
to tag whether or not each memory location holds a valid capability).

As I see it, "locking things down" would likely require turning things
like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
(and generally giving the kernel a much more active role in this process).

Or, change the description, as being mostly a tool to eliminate things
like buffer overflow exploits and memory corruption, and as a fairly
powerful debugging feature.

But, say, note that it would not be sufficient, say, for things like
sandboxing hostile code within a shared address space with another
program that needs to be kept protected.

Granted, the strength could likely be improved (in the face of trying to
prevent hostile code from being able to steal capabilities) through
creative use of ASLR. Along with ABI features, such as "scratch register
scrubbing" (say, loading zeroes into scratch registers on function
return, such as to prevent capabilities from being leaked through via
registers), marking function pointers as "Execute Only" etc.

As noted, a capability system would likely still be pretty strong
against things like buffer overflows (but if only being used to mitigate
buffer overflows, is a bit overkill; so the main "interesting" case is
if it can be used to make an "unbreakable sandbox" for potentially
hostile machine code).

*: If it is possible to perform a Load or (worse, Capability Load)
through a function pointer, this is likely to be a significant attack
vector. Need to make it so that function pointers can only be used to
call things. Protecting against normal data loads would be needed mostly
to try to prevent code from being able to gain access to a known pointer
and possibly side-step the ASLR (say, if it can figure out that the
address it wants to access is reachable from a capability that the code
has access to).

Though, on my side of things, it is possible I could revive a modified
form of the 128-bit ABI, while dropping the VAS back down to 48 bits,
and turn it into a more CHERI-like form (with explicit upper and lower
bounds and access-enable flags, rather than a shared-exponent size and
bias scheme).

Can note the XMOV.x and XLEA.x instructions were already defined as
having the needed semantics, unlike with 64-bit pointers, where the
bounds-checking semantics were opt-in. Would need to add instructions
for things like sub-setting the capabilities, etc.

In this case, register and memory tagging could be added, possibly with
a "Capability Enable" flag somewhere, which would modify some behaviors:
XMOV.x will trap if the address is not flagged as a valid capability;
The normal MOV.x instructions would be disallowed with this flag set.
With possibly a partial exception for PC rel.
Handling of global registers may need some rethinking.
The XMOV instruction would be required to MOV capabilities;
A pair of MOV instructions would no longer be valid.

Though, likely, this could be used in combination with the existing
security model, and with ASLR (well, because, if one has a good RNG,
this can be moderately effective; can't forge access to things if there
is no way to figure out where they are; and ASLR and page-level
protection can still offer a line of defense if the program can figure
out a hole to sidestep the capability system).

Though, still unclear how to prevent a program from "stealing" other
images global registers. Could ALSR the PBO indices (used for reloading
the Global Pointer), and if it is not possible to read through a
function pointer, it may make it difficult to guess the PBO index for
something like a shared library.

An approach used by something like ELF FDPIC would not be ideal, as the
function call mechanism would itself expose the capability of the
callee's GOT, which could likely be mined for capabilities to other
things, ...

It is likely that the capability memory tagging would need to be managed
by the L2 cache. Would need some mechanism for the tag-bits memory (say,
2MB for 256MB at 1b per 16B line). Would also need to somehow work this
flag bit into the ringbus messaging.

Though, in the immediate future, this wouldn't gain much in my case.

....

On 2024-03-09 1:56 p.m., BGB wrote:
> On 3/9/2024 9:09 AM, Scott Lurndal wrote:
>> mitchalsup@aol.com (MitchAlsup1) writes:
>>> I have been in CPU design for a very long time. I did a HS level
>>> design (calculator) in 1968 3 years before the Bomar Brain, did
>>> a #60 design in college as a Jr, and started doing professional
>>> designs (Mc 88100) in 1983.
>>>
>>> With all this background and long term in this career, I can say
>>> without a trace of doubt, that I am not <yet> smart enough to do
>>> a capabilities ISA/system and get it out the door without errors.
>>>
>>> On the other hand, My 66000 Architecture is immune to most attack
>>> strategies now in vogue:: Return Oriented Programming, RowHammer,
>>> Spectré, GOT overwrites, Buffer Overflows,... All without having
>>> any semblance of capabilities; and all without any performance
>>> degradations other than typical cache and TLB effects.
>>
>> On the gripping hand, the Burroughs Large systems capability based
>> design is still processing data almost sixty years after the original
>> B5500 was introduced.
>>
>> There CHERI designs on silicon in existence
>>
>> https://www.morello-project.org/
>
> It is doable, at least.
>
> Main open question is if they can deliver enough on their claims in a
> way that justifies the cost of the memory tagging (eg, where one needs
> to tag whether or not each memory location holds a valid capability).
>
Capabilities sounds like something previously implemented in mainframe
class computers. It reminds me of games where the units have capabilities.

The memory tagging could be done with BRAM accessed in parallel to the
DRAM, but there is not enough BRAM to provide tags for every
capabilities location in my test system. But, if capabilities were
located in capabilities pages it might be possible to implement some
sort of demo. They say in the text it was expected the capabilities
descriptor would come before an object being protected. For instance, 16
BRAMs could be dedicated to providing an extra tag bit for a 1k size
page of DRAM memory. Where there is 512MB of DRAM. The capabilities
would be protecting 1kB regions of memory. That might be okay for some
things like stack allocations by the OS, or context save areas. Mid to
large sized objects. Capabilities do not seem practical for small
objects. Taking at least 16B, probably would not want to use them for
less than 16B. 50% overhead.

For Femtiki OS, I have a single object describing an array of values.
For instance messages which are small objects, are described with a
single object for an array of messages. It is too costly to use an
object descriptor for each message.
>
> As I see it, "locking things down" would likely require turning things
> like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
> (and generally giving the kernel a much more active role in this process).
>
I think this may not be necessary, but I have to read some more. The
capabilities have transfer rules which might make it possible to use
existing code. They have ported things over to Riscv. It cannot be too
mountainous a task.

> Or, change the description, as being mostly a tool to eliminate things
> like buffer overflow exploits and memory corruption, and as a fairly
> powerful debugging feature.
>
> But, say, note that it would not be sufficient, say, for things like
> sandboxing hostile code within a shared address space with another
> program that needs to be kept protected.
>
>
> Granted, the strength could likely be improved (in the face of trying to
> prevent hostile code from being able to steal capabilities) through
> creative use of ASLR. Along with ABI features, such as "scratch register
> scrubbing" (say, loading zeroes into scratch registers on function
> return, such as to prevent capabilities from being leaked through via
> registers), marking function pointers as "Execute Only" etc.
>
> As noted, a capability system would likely still be pretty strong
> against things like buffer overflows (but if only being used to mitigate
> buffer overflows, is a bit overkill; so the main "interesting" case is
> if it can be used to make an "unbreakable sandbox" for potentially
> hostile machine code).
>
>
> *: If it is possible to perform a Load or (worse, Capability Load)
> through a function pointer, this is likely to be a significant attack
> vector. Need to make it so that function pointers can only be used to
> call things. Protecting against normal data loads would be needed mostly
> to try to prevent code from being able to gain access to a known pointer
> and possibly side-step the ASLR (say, if it can figure out that the
> address it wants to access is reachable from a capability that the code
> has access to).
>
>
>
> Though, on my side of things, it is possible I could revive a modified
> form of the 128-bit ABI, while dropping the VAS back down to 48 bits,
> and turn it into a more CHERI-like form (with explicit upper and lower
> bounds and access-enable flags, rather than a shared-exponent size and
> bias scheme).
>
Yeah, IMO explicit upper and lower bounds would be better even though it
uses more memory. The whole manipulation of the bounds is complex. I
sketched out using a 256b capability descriptor. Some of the bits can be
trimmed from the bounds if things are page aligned.

> Can note the XMOV.x and XLEA.x instructions were already defined as
> having the needed semantics, unlike with 64-bit pointers, where the
> bounds-checking semantics were opt-in. Would need to add instructions
> for things like sub-setting the capabilities, etc.
>
> In this case, register and memory tagging could be added, possibly with
> a "Capability Enable" flag somewhere, which would modify some behaviors:
>   XMOV.x will trap if the address is not flagged as a valid capability;
>   The normal MOV.x instructions would be disallowed with this flag set.
>     With possibly a partial exception for PC rel.
>     Handling of global registers may need some rethinking.
>   The XMOV instruction would be required to MOV capabilities;
>     A pair of MOV instructions would no longer be valid.
>
>
> Though, likely, this could be used in combination with the existing
> security model, and with ASLR (well, because, if one has a good RNG,
> this can be moderately effective; can't forge access to things if there
> is no way to figure out where they are; and ASLR and page-level
> protection can still offer a line of defense if the program can figure
> out a hole to sidestep the capability system).
>
> Though, still unclear how to prevent a program from "stealing" other
> images global registers. Could ALSR the PBO indices (used for reloading
> the Global Pointer), and if it is not possible to read through a
> function pointer, it may make it difficult to guess the PBO index for
> something like a shared library.
>
> An approach used by something like ELF FDPIC would not be ideal, as the
> function call mechanism would itself expose the capability of the
> callee's GOT, which could likely be mined for capabilities to other
> things, ...
>
>
>
> It is likely that the capability memory tagging would need to be managed
> by the L2 cache. Would need some mechanism for the tag-bits memory (say,
> 2MB for 256MB at 1b per 16B line). Would also need to somehow work this
> flag bit into the ringbus messaging.
>
> Though, in the immediate future, this wouldn't gain much in my case.
>
> ...
>
>

On Sat, 09 Mar 2024 15:09:46 GMT, Scott Lurndal wrote:

> ... the Burroughs Large systems capability based design ...

As I recall, that depended on not giving users access to a compiler that
could generate instructions that bypassed the protection system.

On Sat, 9 Mar 2024 14:58:24 -0500, Robert Finch wrote:

> Capabilities sounds like something previously implemented in mainframe
> class computers.

IBM’s System/38 and follow-on AS/400 (both long obsolete) may have had
something like them. Not sure if they count as “mainframe-class”.

On 3/9/2024 1:58 PM, Robert Finch wrote:
> On 2024-03-09 1:56 p.m., BGB wrote:
>> On 3/9/2024 9:09 AM, Scott Lurndal wrote:
>>> mitchalsup@aol.com (MitchAlsup1) writes:
>>>> I have been in CPU design for a very long time. I did a HS level
>>>> design (calculator) in 1968 3 years before the Bomar Brain, did
>>>> a #60 design in college as a Jr, and started doing professional
>>>> designs (Mc 88100) in 1983.
>>>>
>>>> With all this background and long term in this career, I can say
>>>> without a trace of doubt, that I am not <yet> smart enough to do
>>>> a capabilities ISA/system and get it out the door without errors.
>>>>
>>>> On the other hand, My 66000 Architecture is immune to most attack
>>>> strategies now in vogue:: Return Oriented Programming, RowHammer,
>>>> Spectré, GOT overwrites, Buffer Overflows,... All without having
>>>> any semblance of capabilities; and all without any performance
>>>> degradations other than typical cache and TLB effects.
>>>
>>> On the gripping hand, the Burroughs Large systems capability based
>>> design is still processing data almost sixty years after the original
>>> B5500 was introduced.
>>>
>>> There CHERI designs on silicon in existence
>>>
>>> https://www.morello-project.org/
>>
>> It is doable, at least.
>>
>> Main open question is if they can deliver enough on their claims in a
>> way that justifies the cost of the memory tagging (eg, where one needs
>> to tag whether or not each memory location holds a valid capability).
>>
> Capabilities sounds like something previously implemented in mainframe
> class computers. It reminds me of games where the units have capabilities.
>
> The memory tagging could be done with BRAM accessed in parallel to the
> DRAM, but there is not enough BRAM to provide tags for every
> capabilities location in my test system. But, if capabilities were
> located in capabilities pages it might be possible to implement some
> sort of demo. They say in the text it was expected the capabilities
> descriptor would come before an object being protected. For instance, 16
> BRAMs could be dedicated to providing an extra tag bit for a 1k size
> page of DRAM memory. Where there is 512MB of DRAM. The capabilities
> would be protecting 1kB regions of memory. That might be okay for some
> things like stack allocations by the OS, or context save areas. Mid to
> large sized objects. Capabilities do not seem practical for small
> objects. Taking at least 16B, probably would not want to use them for
> less than 16B. 50% overhead.
>
> For Femtiki OS, I have a single object describing an array of values.
> For instance messages which are small objects, are described with a
> single object for an array of messages. It is too costly to use an
> object descriptor for each message.

For a CHERI like approach, one would need a tag of 1 bit for every 16
bytes of RAM (to flag whether or not that RAM represents a valid
capability).

For the combination of RAM sizes and FPGAs I have access to, this is
non-viable, as I would need more BRAM for the tag memory than exists in
the FPGAs.

In effect, this will mean needing another smaller cache which is bolted
onto the L2 cache or similar, whose sole purpose is to provide tag-bits
(and probably bounce requests to some other area of RAM which contains
the tag-bits memory).

>>
>> As I see it, "locking things down" would likely require turning things
>> like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
>> (and generally giving the kernel a much more active role in this
>> process).
>>
> I think this may not be necessary, but I have to read some more. The
> capabilities have transfer rules which might make it possible to use
> existing code. They have ported things over to Riscv. It cannot be too
> mountainous a task.
>

You can make it work, yes, but the question is less "can you make it
work, technically", but more:
Can you make it work in a way that provides both a fairly normal C
experience, and *also* an unbreakable sandbox, at the same time.

My skepticism here is that, short of drastic measures like moving malloc
and libdl and similar into kernel space, it may not be possible to keep
the sandbox secure using solely capabilities.

ASLR could help, but using ASLR to maintain an image of integrity for
the capability system would be "kinda weak".

One could ask though:
How is my security model (with Keyrings) any different?

Well, the partial answer mostly is that a call that switches keyrings is
effectively accomplished via context switches (with the two keyrings
effectively running in separate threads).

So, like, even if the untrusted thread has a pointer to the protected
thread's memory, it can't access it...

Though, a similar model could potentially be accomplished with
conventional page-tables, by making pseudo-processes which only share
parts of their address space with another process (and the protected
memory is located in the non-shared spaces, with any calls between them
via an RPC mechanism).

Had considered mechanisms which could pull this off without a context
switch, but most would fall short of "acceptably secure" (if a path
exists where a task could modify its own KRR or similar, this mechanism
is blown).

My bounds-checking scheme also worked, but with a caveat:
It only works if code does not get "overly clever" with the use of pointers.

So, it worked well enough to where I was able to enable full
bounds-checking in Doom and similar, but was not entirely transparent to
some of the runtime code. If you cast between pointers and integers, and
manipulate the pointer bits, there are "gotchas".

A full capability system is going to have a similar restriction.
Either pointer<->integer casting would need to be disallowed, or (more
likely), turned into a runtime call which can "bless" the address before
returning it as a capability, which would exist as another potential
attack surface (unless, of course, this mechanism is itself turned into
a system call).

OTOH:
If one can't implement something like a conventional JavaScript VM, or
if it takes a significant performance hit, this would not be ideal.

>> Or, change the description, as being mostly a tool to eliminate things
>> like buffer overflow exploits and memory corruption, and as a fairly
>> powerful debugging feature.
>>
>> But, say, note that it would not be sufficient, say, for things like
>> sandboxing hostile code within a shared address space with another
>> program that needs to be kept protected.
>>
>>
>> Granted, the strength could likely be improved (in the face of trying
>> to prevent hostile code from being able to steal capabilities) through
>> creative use of ASLR. Along with ABI features, such as "scratch
>> register scrubbing" (say, loading zeroes into scratch registers on
>> function return, such as to prevent capabilities from being leaked
>> through via registers), marking function pointers as "Execute Only" etc.
>>
>> As noted, a capability system would likely still be pretty strong
>> against things like buffer overflows (but if only being used to
>> mitigate buffer overflows, is a bit overkill; so the main
>> "interesting" case is if it can be used to make an "unbreakable
>> sandbox" for potentially hostile machine code).
>>
>>
>> *: If it is possible to perform a Load or (worse, Capability Load)
>> through a function pointer, this is likely to be a significant attack
>> vector. Need to make it so that function pointers can only be used to
>> call things. Protecting against normal data loads would be needed
>> mostly to try to prevent code from being able to gain access to a
>> known pointer and possibly side-step the ASLR (say, if it can figure
>> out that the address it wants to access is reachable from a capability
>> that the code has access to).
>>
>>
>>
>> Though, on my side of things, it is possible I could revive a modified
>> form of the 128-bit ABI, while dropping the VAS back down to 48 bits,
>> and turn it into a more CHERI-like form (with explicit upper and lower
>> bounds and access-enable flags, rather than a shared-exponent size and
>> bias scheme).
>>
> Yeah, IMO explicit upper and lower bounds would be better even though it
> uses more memory. The whole manipulation of the bounds is complex. I
> sketched out using a 256b capability descriptor. Some of the bits can be
> trimmed from the bounds if things are page aligned.
>

BGB <cr88192@gmail.com> writes:
>On 3/9/2024 9:09 AM, Scott Lurndal wrote:

>>
>> There [are] CHERI designs on silicon in existence
>>
>> https://www.morello-project.org/
>
>It is doable, at least.
>
>Main open question is if they can deliver enough on their claims in a
>way that justifies the cost of the memory tagging (eg, where one needs
>to tag whether or not each memory location holds a valid capability).
>
>
>As I see it, "locking things down" would likely require turning things
>like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
>(and generally giving the kernel a much more active role in this process).

All of these have been addressed in CHERI.

BGB wrote:

> On 3/9/2024 1:58 PM, Robert Finch wrote:
>> On 2024-03-09 1:56 p.m., BGB wrote:
>>> On 3/9/2024 9:09 AM, Scott Lurndal wrote:
>>>> mitchalsup@aol.com (MitchAlsup1) writes:
<snip>
>>
>> For Femtiki OS, I have a single object describing an array of values.
>> For instance messages which are small objects, are described with a
>> single object for an array of messages. It is too costly to use an
>> object descriptor for each message.

> For a CHERI like approach, one would need a tag of 1 bit for every 16
> bytes of RAM (to flag whether or not that RAM represents a valid
> capability).

> For the combination of RAM sizes and FPGAs I have access to, this is
> non-viable, as I would need more BRAM for the tag memory than exists in
> the FPGAs.

Yes, indeed, not viable. Now imagine a page of those, and now you have
to write out 4096 bytes and 2048-tag-bits onto a disk with standard
sectors......

> In effect, this will mean needing another smaller cache which is bolted
> onto the L2 cache or similar, whose sole purpose is to provide tag-bits
> (and probably bounce requests to some other area of RAM which contains
> the tag-bits memory).

Denelcor HEP had tag-like-bits and all the crud they bring (but they were
used as locks instead of tags).

>>>
>>> As I see it, "locking things down" would likely require turning things
>>> like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
>>> (and generally giving the kernel a much more active role in this
>>> process).
>>>
>> I think this may not be necessary, but I have to read some more. The
>> capabilities have transfer rules which might make it possible to use
>> existing code. They have ported things over to Riscv. It cannot be too
>> mountainous a task.
>>

> You can make it work, yes, but the question is less "can you make it
> work, technically", but more:
> Can you make it work in a way that provides both a fairly normal C
> experience, and *also* an unbreakable sandbox, at the same time.

And here the answer is essentially <wait for it> no.

> My skepticism here is that, short of drastic measures like moving malloc
> and libdl and similar into kernel space, it may not be possible to keep
> the sandbox secure using solely capabilities.

> ASLR could help, but using ASLR to maintain an image of integrity for
> the capability system would be "kinda weak".

How do you ALSR code when a latent capability on disk still points at
its defined memory area ? Yes, you can ALSR at boot, but you can use
the file system to hold capabilities {which is something most capability
systems desire and promote.}

> One could ask though:
> How is my security model (with Keyrings) any different?

> Well, the partial answer mostly is that a call that switches keyrings is
> effectively accomplished via context switches (with the two keyrings
> effectively running in separate threads).

> So, like, even if the untrusted thread has a pointer to the protected
> thread's memory, it can't access it...

> Though, a similar model could potentially be accomplished with
> conventional page-tables, by making pseudo-processes which only share
> parts of their address space with another process (and the protected
> memory is located in the non-shared spaces, with any calls between them
> via an RPC mechanism).

Capability manipulation via messages.

> Had considered mechanisms which could pull this off without a context
> switch, but most would fall short of "acceptably secure" (if a path
> exists where a task could modify its own KRR or similar, this mechanism
> is blown).

> My bounds-checking scheme also worked, but with a caveat:
> It only works if code does not get "overly clever" with the use of pointers.

Which no-one can trust of C programs.

> So, it worked well enough to where I was able to enable full
> bounds-checking in Doom and similar, but was not entirely transparent to
> some of the runtime code. If you cast between pointers and integers, and
> manipulate the pointer bits, there are "gotchas".

Gee, if only we had trained programmers to avoid some of the things we
are now requiring new languages to prevent.....

> A full capability system is going to have a similar restriction.

Understatement of the year candidate !

> Either pointer<->integer casting would need to be disallowed, or (more
> likely), turned into a runtime call which can "bless" the address before
> returning it as a capability, which would exist as another potential
> attack surface (unless, of course, this mechanism is itself turned into
> a system call).

> OTOH:
> If one can't implement something like a conventional JavaScript VM, or
> if it takes a significant performance hit, this would not be ideal.

Going for 2 in one post !!

>>> Or, change the description, as being mostly a tool to eliminate things
>>> like buffer overflow exploits and memory corruption, and as a fairly
>>> powerful debugging feature.
>>>
>>> But, say, note that it would not be sufficient, say, for things like
>>> sandboxing hostile code within a shared address space with another
>>> program that needs to be kept protected.
>>>
>>>
>>> Granted, the strength could likely be improved (in the face of trying
>>> to prevent hostile code from being able to steal capabilities) through
>>> creative use of ASLR. Along with ABI features, such as "scratch
>>> register scrubbing" (say, loading zeroes into scratch registers on
>>> function return, such as to prevent capabilities from being leaked
>>> through via registers), marking function pointers as "Execute Only" etc.
>>>
>>> As noted, a capability system would likely still be pretty strong
>>> against things like buffer overflows (but if only being used to
>>> mitigate buffer overflows, is a bit overkill; so the main
>>> "interesting" case is if it can be used to make an "unbreakable
>>> sandbox" for potentially hostile machine code).
>>>
>>>
>>> *: If it is possible to perform a Load or (worse, Capability Load)
>>> through a function pointer, this is likely to be a significant attack
>>> vector. Need to make it so that function pointers can only be used to
>>> call things. Protecting against normal data loads would be needed
>>> mostly to try to prevent code from being able to gain access to a
>>> known pointer and possibly side-step the ASLR (say, if it can figure
>>> out that the address it wants to access is reachable from a capability
>>> that the code has access to).
>>>
>>>
>>>
>>> Though, on my side of things, it is possible I could revive a modified
>>> form of the 128-bit ABI, while dropping the VAS back down to 48 bits,
>>> and turn it into a more CHERI-like form (with explicit upper and lower
>>> bounds and access-enable flags, rather than a shared-exponent size and
>>> bias scheme).
>>>
>> Yeah, IMO explicit upper and lower bounds would be better even though it
>> uses more memory. The whole manipulation of the bounds is complex. I
>> sketched out using a 256b capability descriptor. Some of the bits can be
>> trimmed from the bounds if things are page aligned.
>>

> IIRC, they were using 128-bit descriptors with a bit-slicing scheme.

> So, say, if I were to do similar (within my existing pointer layout):
> ( 27: 0): Base Address
> ( 47: 28): Shared Address (47:28)
> ( 63: 48): Type Tag Bits
> ( 87: 64): Lower Bound (27:4)
> (111: 88): Upper Bound (27:4)
> ( 112): Base Adjust
> ( 113): Lower Bound Adjust
> ( 114): Upper Bound Adjust
> (127:115): Access Flags / Etc

Why is lower bound NOT 0 ?
How can you assume/work-with a base address smaller than 48-bits ??
How can the bounds not be at least 47-bits in size ??

> Though, this particular encoding would limit bounds-checking to a 256MB
> region, which is lame (or eat more tag bits, and have slightly bigger
> regions).

It is worse than lame.

Lawrence D'Oliveiro <ldo@nz.invalid> writes:
>On Sat, 09 Mar 2024 15:09:46 GMT, Scott Lurndal wrote:
>
>> ... the Burroughs Large systems capability based design ...
>
>As I recall, that depended on not giving users access to a compiler that
>could generate instructions that bypassed the protection system.

Yes. Which isn't a problem (once they fixed the bug in the early
1970's that allowed loading a compiler from a library tape). Fixed
by removing the compiler privilege when loading from tape - the
operator/system administrator would need issue a privileged
command to mark the executable as a compiler.

On 3/9/2024 4:04 PM, MitchAlsup1 wrote:
> BGB wrote:
[...]

> Gee, if only we had trained programmers to avoid some of the things we
> are now requiring new languages to prevent.....
[...]

We must put corks on the forks to prevent a programmer from hurting
itself and/or others...

Did you ever see a movie by the name of "Dirty Rotten Scoundrels"? Here
is a relevant scene. The, say, C programmer would be the Ruprecht
character played by Steve Martin in the following scene. The "Arch Gods
of Help" would be the Michael Caine Character? Humm.... ;^o

https://youtu.be/SKDX-qJaJ08

The programmers are the source of all problems! ;^) lol...

BGB <cr88192@gmail.com> wrote:
> > There CHERI designs on silicon in existence
> >
> > https://www.morello-project.org/

Member of the CHERI team here...

> Main open question is if they can deliver enough on their claims in a
> way that justifies the cost of the memory tagging (eg, where one needs
> to tag whether or not each memory location holds a valid capability).

Crudely speaking, it's 1/128 of memory so 0.78%. Through use of
a tag cache you don't need to have a speciifc 129th bit on your DRAM
(although that's an option, like ECC) but can just wall off a piece of
regular DRAM and use it for a tag table that's accessed via the tag cache.
You only pay the cost for DRAM you actually have (eg one tag table per DIMM,
sized for that DIMM).

The tag cache can also be compressed[1] so that if a memory page has no
capabilities you don't need to store tags for it. This reduces tag cache
bandwidth further.

> As I see it, "locking things down" would likely require turning things
> like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
> (and generally giving the kernel a much more active role in this process).

You don't need memory allocations to be system calls, because capabilities
can be manipulated in userspace (which is a key design goal). ie if
malloc() possesses a capability to 1MB of memory and a client requests 100
bytes, it can take its 1MB capability, offset the base to point to the start
of the allocation, set the offset to zero, and shrink the top to be
base+100, and then return that 100-byte capability.

In a Unix OS, userspace malloc() does occasionally need to make a syscall -
when it runs out of memory pages on hand it has to ask the kernel to
allocate it some more via mmap(). But this is orthogonal to use of
capabilities (which are virtually addressed in such a system).

For dynamic linking, you need something that holds capabilities that allow
you to turn read-write memory into read-execute memory. That can be part of
the dynamic linker or part of the OS. Since you are typically making
syscalls to read the shared library from disc anyway, as well as marking RW
pages as RX pages in the page table, the OS is already involved.

> But, say, note that it would not be sufficient, say, for things like
> sandboxing hostile code within a shared address space with another
> program that needs to be kept protected.
>
>
> Granted, the strength could likely be improved (in the face of trying to
> prevent hostile code from being able to steal capabilities) through
> creative use of ASLR. Along with ABI features, such as "scratch register
> scrubbing" (say, loading zeroes into scratch registers on function
> return, such as to prevent capabilities from being leaked through via
> registers), marking function pointers as "Execute Only" etc.
>
> As noted, a capability system would likely still be pretty strong
> against things like buffer overflows (but if only being used to mitigate
> buffer overflows, is a bit overkill; so the main "interesting" case is
> if it can be used to make an "unbreakable sandbox" for potentially
> hostile machine code).

Code capabilities prevent a lot of control flow attacks, because you can
only execute code you have capabilities to. For example you're in a
function - you possess a return address capability (which has bounds based
on your local environment or compartment you're in) so you can manipulate
the address of that return address, but you can't jump to arbitrary code and
you can't forge return addresses. So no stack smashing, no ROP/JOP attacks,
etc.

Through setting the bounds on a code capability you can sandbox small pieces
of code to a function granularity, which is more efficient than typical MMU
sandboxing.

> *: If it is possible to perform a Load or (worse, Capability Load)
> through a function pointer, this is likely to be a significant attack
> vector. Need to make it so that function pointers can only be used to
> call things. Protecting against normal data loads would be needed mostly
> to try to prevent code from being able to gain access to a known pointer
> and possibly side-step the ASLR (say, if it can figure out that the
> address it wants to access is reachable from a capability that the code
> has access to).

You can't arbitrarily change data to code capabilities - they are different
types.

> It is likely that the capability memory tagging would need to be managed
> by the L2 cache. Would need some mechanism for the tag-bits memory (say,
> 2MB for 256MB at 1b per 16B line). Would also need to somehow work this
> flag bit into the ringbus messaging.

AXI has user fields which can be used for sending capabilities across the
interconnect, through third party AXI IPs like muxes, arbiters, etc.

Theo

[1] https://www.cl.cam.ac.uk/research/security/ctsrd/pdfs/201711-iccd2017-efficient-tags.pdf

MitchAlsup1 <mitchalsup@aol.com> wrote:
> BGB wrote:
>
> > On 3/9/2024 1:58 PM, Robert Finch wrote:
> >> On 2024-03-09 1:56 p.m., BGB wrote:
> >>> On 3/9/2024 9:09 AM, Scott Lurndal wrote:
> >>>> mitchalsup@aol.com (MitchAlsup1) writes:
> <snip>
> >>
> >> For Femtiki OS, I have a single object describing an array of values.
> >> For instance messages which are small objects, are described with a
> >> single object for an array of messages. It is too costly to use an
> >> object descriptor for each message.
>
> > For a CHERI like approach, one would need a tag of 1 bit for every 16
> > bytes of RAM (to flag whether or not that RAM represents a valid
> > capability).
>
> > For the combination of RAM sizes and FPGAs I have access to, this is
> > non-viable, as I would need more BRAM for the tag memory than exists in
> > the FPGAs.

If you have ECC RAM on your FPGA board you could use the ECC bits for tags.
Otherwise a tag cache is another way. The L1s and L2s carry tags (ie 129
bit datapath), but you just put the tag cache on the front of your DRAM.

> Yes, indeed, not viable. Now imagine a page of those, and now you have
> to write out 4096 bytes and 2048-tag-bits onto a disk with standard
> sectors......

Our swapping implementation keeps the tag bits in RAM, while the page is
swapped out. Eventually you need to swap out a page of tag bits, but that's
much less common.

> > In effect, this will mean needing another smaller cache which is bolted
> > onto the L2 cache or similar, whose sole purpose is to provide tag-bits
> > (and probably bounce requests to some other area of RAM which contains
> > the tag-bits memory).
>
> Denelcor HEP had tag-like-bits and all the crud they bring (but they were
> used as locks instead of tags).
>
> >>>
> >>> As I see it, "locking things down" would likely require turning things
> >>> like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
> >>> (and generally giving the kernel a much more active role in this
> >>> process).
> >>>
> >> I think this may not be necessary, but I have to read some more. The
> >> capabilities have transfer rules which might make it possible to use
> >> existing code. They have ported things over to Riscv. It cannot be too
> >> mountainous a task.
> >>
>
> > You can make it work, yes, but the question is less "can you make it
> > work, technically", but more:
> > Can you make it work in a way that provides both a fairly normal C
> > experience, and *also* an unbreakable sandbox, at the same time.

The C experience is fairly normal, as long as you are actually playing by
the C rules. You can't arbitraily cast integers to pointers - if you plan
to do that you need to use intptr_t so the compiler knows to keep the data
in a capability so it can use it as a pointer later.

Tricks which store data in the upper or lower bits of pointers are awkward.
Other tricks like XOR linked lists of pointers don't work. This is all
stuff that's pushing into the 'undefined behaviour' parts of C (even if C
doesn't explicitly call it out).

Changes in a 6M LoC KDE desktop codebase were 0.026% of lines:
https://www.capabilitieslimited.co.uk/_files/ugd/f4d681_e0f23245dace466297f20a0dbd22d371.pdf

Depends what you mean by 'unbreakable sandbox': this is compiling code with
every pointer being a capability, so every memory access is bounds checked.

Sandboxing involves dividing code into compartments; that involves some
decision making as to where you draw the security boundaries. There aren't
good tools to do that (they are being worked on). CHERI offers you the
tools to implement whatever compartmentalisation stategy you wish, but it's
not quite as simple as just recompiling.

> And here the answer is essentially <wait for it> no.
>
> > My skepticism here is that, short of drastic measures like moving malloc
> > and libdl and similar into kernel space, it may not be possible to keep
> > the sandbox secure using solely capabilities.
>
> > ASLR could help, but using ASLR to maintain an image of integrity for
> > the capability system would be "kinda weak".
>
> How do you ALSR code when a latent capability on disk still points at
> its defined memory area ? Yes, you can ALSR at boot, but you can use
> the file system to hold capabilities {which is something most capability
> systems desire and promote.}

Why would you want to ASLR? ASLR is to prevent you guessing valid addresses
for things so you can't craft pointers to them. CHERI prevents you crafting
pointers to arbitrary things in the first place.

> > One could ask though:
> > How is my security model (with Keyrings) any different?
>
> > Well, the partial answer mostly is that a call that switches keyrings is
> > effectively accomplished via context switches (with the two keyrings
> > effectively running in separate threads).
>
> > So, like, even if the untrusted thread has a pointer to the protected
> > thread's memory, it can't access it...
>
> > Though, a similar model could potentially be accomplished with
> > conventional page-tables, by making pseudo-processes which only share
> > parts of their address space with another process (and the protected
> > memory is located in the non-shared spaces, with any calls between them
> > via an RPC mechanism).
>
> Capability manipulation via messages.

That's the microkernel setup: the software running in system mode holds
the privilege to alter access control (via page tables), so any time you
want to change that you have to ask the system (microkernel or whatever) to do
so. That's slow, in particular TLB manipulation (invalidation and
shootdowns). CHERI allows you to manipulate them in userspace without
having to call out to the kernel. Additionally it is finer grained than page
granularity.

Some experimental OSes have done things with manipulating page tables from
userspace processes which avoids syscall overhead but not TLB costs - and it
probably depends on the architecture whether you can do TLB invalidations
from userspace.

> > Had considered mechanisms which could pull this off without a context
> > switch, but most would fall short of "acceptably secure" (if a path
> > exists where a task could modify its own KRR or similar, this mechanism
> > is blown).
>
>
> > My bounds-checking scheme also worked, but with a caveat:
> > It only works if code does not get "overly clever" with the use of pointers.
>
> Which no-one can trust of C programs.

A lot of modern software is well behaved (see figure above). Particular
software like JIT compilers can be more awkward - ideally you would really want
the JIT compiler to emit capability-aware code. You can still run generated
aarch64/rv64 non-capability code, but without the benefit of capability
checks.

> > So, it worked well enough to where I was able to enable full
> > bounds-checking in Doom and similar, but was not entirely transparent to
> > some of the runtime code. If you cast between pointers and integers, and
> > manipulate the pointer bits, there are "gotchas".

That's the kind of thing that fall down: software being 'clever', where it
doesn't need to be. I get the sense Doom's primary purpose in life was
being 'clever' in order to be fast on a 386.

> Gee, if only we had trained programmers to avoid some of the things we
> are now requiring new languages to prevent.....

If only we could rewrite all the software out there in memory-safe
languages... then we'd have twice as much software (and more bugs).

> > Either pointer<->integer casting would need to be disallowed, or (more
> > likely), turned into a runtime call which can "bless" the address before
> > returning it as a capability, which would exist as another potential
> > attack surface (unless, of course, this mechanism is itself turned into
> > a system call).
>
>
> > OTOH:
> > If one can't implement something like a conventional JavaScript VM, or
> > if it takes a significant performance hit, this would not be ideal.
>
> Going for 2 in one post !!

We've had the DukTape Javascript interpreter working for CHERI for a while.
Work is under way to port Chromium and V8 - that's a much bigger project,
just because Chromium is a huge piece of software (and we're running on
FreeBSD, which is not a platform that Chrome supports building for). The
work in V8 is to get it to implement the JS object model using CHERI
instructions as part of its generated code.

Theo Markettos wrote:

> MitchAlsup1 <mitchalsup@aol.com> wrote:
>> BGB wrote:
>> <snip>
>> > You can make it work, yes, but the question is less "can you make it
>> > work, technically", but more:
>> > Can you make it work in a way that provides both a fairly normal C
>> > experience, and *also* an unbreakable sandbox, at the same time.

> The C experience is fairly normal, as long as you are actually playing by
> the C rules. You can't arbitraily cast integers to pointers - if you plan
> to do that you need to use intptr_t so the compiler knows to keep the data
> in a capability so it can use it as a pointer later.

As a 'for instance' how does one take a capability and align it to a cache
line boundary ?? Say in/after malloc() ?!?

> Tricks which store data in the upper or lower bits of pointers are awkward.

Especially so when you have a 64-bit VaS to play in.

> Other tricks like XOR linked lists of pointers don't work.

This should have died out with the PDP-11s. With modern machines it does not
save enough space to warrant the loss in performance.

> This is all
> stuff that's pushing into the 'undefined behaviour' parts of C (even if C
> doesn't explicitly call it out).

<snip>

> Why would you want to ASLR? ASLR is to prevent you guessing valid addresses
> for things so you can't craft pointers to them. CHERI prevents you crafting
> pointers to arbitrary things in the first place.

ALSR has become a catch-phrase used to give the listener a good feeling
about the security of the present system--all the while knowing that is
it little more than window dressing on a building already in flames.

On 3/10/2024 11:24 AM, Theo Markettos wrote:
> MitchAlsup1 <mitchalsup@aol.com> wrote:
>> BGB wrote:
>>
>>> On 3/9/2024 1:58 PM, Robert Finch wrote:
>>>> On 2024-03-09 1:56 p.m., BGB wrote:
>>>>> On 3/9/2024 9:09 AM, Scott Lurndal wrote:
>>>>>> mitchalsup@aol.com (MitchAlsup1) writes:
>> <snip>
>>>>
>>>> For Femtiki OS, I have a single object describing an array of values.
>>>> For instance messages which are small objects, are described with a
>>>> single object for an array of messages. It is too costly to use an
>>>> object descriptor for each message.
>>
>>> For a CHERI like approach, one would need a tag of 1 bit for every 16
>>> bytes of RAM (to flag whether or not that RAM represents a valid
>>> capability).
>>
>>> For the combination of RAM sizes and FPGAs I have access to, this is
>>> non-viable, as I would need more BRAM for the tag memory than exists in
>>> the FPGAs.
>
> If you have ECC RAM on your FPGA board you could use the ECC bits for tags.
> Otherwise a tag cache is another way. The L1s and L2s carry tags (ie 129
> bit datapath), but you just put the tag cache on the front of your DRAM.
>

The RAM on my FPGA boards only provides a 16-bit bus interface.

If one spent the "big money" on something like a Nexys Video, IIRC, this
is 32-bit RAM.

>> Yes, indeed, not viable. Now imagine a page of those, and now you have
>> to write out 4096 bytes and 2048-tag-bits onto a disk with standard
>> sectors......
>
> Our swapping implementation keeps the tag bits in RAM, while the page is
> swapped out. Eventually you need to swap out a page of tag bits, but that's
> much less common.
>

Yeah. In any case, tag bits need to be somewhere where the page-swapping
code has access to them...

>>> In effect, this will mean needing another smaller cache which is bolted
>>> onto the L2 cache or similar, whose sole purpose is to provide tag-bits
>>> (and probably bounce requests to some other area of RAM which contains
>>> the tag-bits memory).
>>
>> Denelcor HEP had tag-like-bits and all the crud they bring (but they were
>> used as locks instead of tags).
>>
>>>>>
>>>>> As I see it, "locking things down" would likely require turning things
>>>>> like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
>>>>> (and generally giving the kernel a much more active role in this
>>>>> process).
>>>>>
>>>> I think this may not be necessary, but I have to read some more. The
>>>> capabilities have transfer rules which might make it possible to use
>>>> existing code. They have ported things over to Riscv. It cannot be too
>>>> mountainous a task.
>>>>
>>
>>> You can make it work, yes, but the question is less "can you make it
>>> work, technically", but more:
>>> Can you make it work in a way that provides both a fairly normal C
>>> experience, and *also* an unbreakable sandbox, at the same time.
>
> The C experience is fairly normal, as long as you are actually playing by
> the C rules. You can't arbitraily cast integers to pointers - if you plan
> to do that you need to use intptr_t so the compiler knows to keep the data
> in a capability so it can use it as a pointer later.
>
> Tricks which store data in the upper or lower bits of pointers are awkward.
> Other tricks like XOR linked lists of pointers don't work. This is all
> stuff that's pushing into the 'undefined behaviour' parts of C (even if C
> doesn't explicitly call it out).
>

The model of C that I consider normal arguably has a fair bit of UB.

> Changes in a 6M LoC KDE desktop codebase were 0.026% of lines:
> https://www.capabilitieslimited.co.uk/_files/ugd/f4d681_e0f23245dace466297f20a0dbd22d371.pdf
>
> Depends what you mean by 'unbreakable sandbox': this is compiling code with
> every pointer being a capability, so every memory access is bounds checked.
>
> Sandboxing involves dividing code into compartments; that involves some
> decision making as to where you draw the security boundaries. There aren't
> good tools to do that (they are being worked on). CHERI offers you the
> tools to implement whatever compartmentalisation stategy you wish, but it's
> not quite as simple as just recompiling.
>

Seems so.

Looks like, at a minimum, things like "malloc()" need to be modified to
be aware of the capability system.

>> And here the answer is essentially <wait for it> no.
>>
>>> My skepticism here is that, short of drastic measures like moving malloc
>>> and libdl and similar into kernel space, it may not be possible to keep
>>> the sandbox secure using solely capabilities.
>>
>>> ASLR could help, but using ASLR to maintain an image of integrity for
>>> the capability system would be "kinda weak".
>>
>> How do you ALSR code when a latent capability on disk still points at
>> its defined memory area ? Yes, you can ALSR at boot, but you can use
>> the file system to hold capabilities {which is something most capability
>> systems desire and promote.}
>
> Why would you want to ASLR? ASLR is to prevent you guessing valid addresses
> for things so you can't craft pointers to them. CHERI prevents you crafting
> pointers to arbitrary things in the first place.
>

It stops you from crafting pointers directly...

But, that is not the question, it is what happens if the program manages
to steal a capability that gives it access to something that it
shouldn't have access to, and then can use this capability to access
something else.

If you ASLR things, this may make this latter scenario more difficult
(and, if it does happen, one is merely back to the situation on current
systems).

Say, for example, we have an ABI where, say:
Function pointer points to a 32-byte location, containing two capabilities:
The executable code to be run;
A capability representing the GOT to be used by the callee.

Here, I am assuming an FDPIC style ABI, don't actually know what ABI
design they are using (my quick look didn't find much, but the normal
ELF ABI isn't likely to work within a pure capability system, so...).

And, then someone realizes, say:
The malloc function has a GOT, and say, entry 31 has access to the array
of heap capabilities.

Then, say, the program does something like, say:
void **forbidden_array;
forbidden_array=((void ***)malloc)[1][31];

Or, say, for sake of "blessing pointers" (for int->pointer casts), if
there is a:
void *__blessify_qi2c(intptr_t a);
And, if "blessify" has access to a global capability (at index 5):
void *very_bad;
very_bad=((void ***)__blessify_qi2c)[1][5];

But, if such things are possible, capabilities can't be the sole line of
defense.

Though, ideally, the ABI would also include mechanisms to mitigate the
effectiveness of these sorts of attack vectors (likely also needing to
involve the use of random number generators).

Luckily, ASLR is "virtually free", and so there isn't a good reason to
not use it, if there is still a possibility of holes in the armor.

>>> One could ask though:
>>> How is my security model (with Keyrings) any different?
>>
>>> Well, the partial answer mostly is that a call that switches keyrings is
>>> effectively accomplished via context switches (with the two keyrings
>>> effectively running in separate threads).
>>
>>> So, like, even if the untrusted thread has a pointer to the protected
>>> thread's memory, it can't access it...
>>
>>> Though, a similar model could potentially be accomplished with
>>> conventional page-tables, by making pseudo-processes which only share
>>> parts of their address space with another process (and the protected
>>> memory is located in the non-shared spaces, with any calls between them
>>> via an RPC mechanism).
>>
>> Capability manipulation via messages.
>
> That's the microkernel setup: the software running in system mode holds
> the privilege to alter access control (via page tables), so any time you
> want to change that you have to ask the system (microkernel or whatever) to do
> so. That's slow, in particular TLB manipulation (invalidation and
> shootdowns). CHERI allows you to manipulate them in userspace without
> having to call out to the kernel. Additionally it is finer grained than page
> granularity.
>
> Some experimental OSes have done things with manipulating page tables from
> userspace processes which avoids syscall overhead but not TLB costs - and it
> probably depends on the architecture whether you can do TLB invalidations
> from userspace.
>

John Dallman <jgd@cix.co.uk> wrote:
> In article <Qry*XE3Ez@news.chiark.greenend.org.uk>,
> theom+news@chiark.greenend.org.uk (Theo Markettos) wrote:
>
> > The C experience is fairly normal, as long as you are actually
> > playing by the C rules. You can't arbitraily cast integers to
> > pointers - if you plan to do that you need to use intptr_t so
> > the compiler knows to keep the data in a capability so it can
> > use it as a pointer later.
>
> Makes sense, though it will require updating of older code for the rules
> being more thoroughly enforced. Not a bad thing.

Indeed.

> > Tricks which store data in the upper or lower bits of pointers are
> > awkward.
>
> Not compatible with Aarch64 Pointer Authentication, but CHERI should be a
> functional replacement anyway.

I think they can theoretically coexist - ie you can have an authenticated 64
bit pointer which looks like an integer as far as the capability checks are
concerned, but if you don't try to dereference it as a capability then
that's ok. (Morello doesn't implement PA as it's based on a prior
microarchitecture)

> > Changes in a 6M LoC KDE desktop codebase were 0.026% of lines:
> > https://www.capabilitieslimited.co.uk/_files/ugd/f4d681_e0f23245dace
> > 466297f20a0dbd22d371.pdf
>
> 15,000 or so changes. Quite a lot. Is the code backwards-compatible to a
> conventional C platform?

Should be - it's mostly making things play by the rules. Once they play by
the rules then it means they will work the same (or less buggily) on a
regular C platform.

The above link describes the changes - a number being replacing 'long' with
intptr_t, some undefined behaviour, bad use of realloc(). Some of it was
modernisation of old codebases (eg add C11 atomics), other changes were just
to make optional code that isn't currently available (eg no OpenGL available
in VNC).

> > Sandboxing involves dividing code into compartments; that involves
> > some decision making as to where you draw the security boundaries.
> > There aren't good tools to do that (they are being worked on).
> > CHERI offers you the tools to implement whatever compartmentalisation
> > stategy you wish, but it's not quite as simple as just recompiling.
>
> I have a slightly odd case: the software I work on ships as a great big
> shared library that's used in-process by its caller. It isn't any kind of
> server, and doesn't use any IPC; in concept it's a huge math library that
> asks the caller to allocate memory for it. So it needs to share a heap
> with the caller. Presumably that model is workable?

Do you want to compartmentalise that shared library, ie put in trust
boundaries between the library and its caller?

If you just want to run the shared library as-is, you can recompile it and
get bounds checking etc. If you want to have some kind of trust boundary
(eg the library doesn't trust the app, or the app doesn't trust the library)
then you would need to put in a compartment boundary between the two. In
that case it might make sense for the memory allocator to be its own
compartment; the capabilities it hands out should be usable by both app and
library.

> > ... we're running on FreeBSD
>
> That was a point against my experimenting with Morello when we were
> offered it last year; the requirement to port to FreeBSD first. Morello
> Linux seems insufficiently mature at present; do you have any idea of the
> timescale for it to be robustly usable for porting application code by
> someone who isn't experienced in Linux internals?

FreeBSD is more advanced, in part because it's had more development effort
on it over the years and partly since it's less of a moving target (Linux
has huge amounts of churn). That means more things have capability support
in the kernel and userspace (including a lot of packages available).

I believe Morello Linux is able to support console-mode apps - ie it has
support context switching and use of capabilities in userspace, with some
support in glibc. I believe there is now a dynamic linker, but not sure of
the status. There is only limited use of capabilities in the kernel, so
anything to do with kernel compartmentalisation would be more work. I think
someone was working on building Debian packages pure-capability - I haven't
heard the current status of that work.

https://www.morello-project.org/cheri-feature-matrix/
has a comparison table.

Theo

MitchAlsup1 <mitchalsup@aol.com> wrote:
> Theo Markettos wrote:
>
> > MitchAlsup1 <mitchalsup@aol.com> wrote:
> >> BGB wrote:
> >> <snip>
> >> > You can make it work, yes, but the question is less "can you make it
> >> > work, technically", but more:
> >> > Can you make it work in a way that provides both a fairly normal C
> >> > experience, and *also* an unbreakable sandbox, at the same time.
>
> > The C experience is fairly normal, as long as you are actually playing by
> > the C rules. You can't arbitraily cast integers to pointers - if you plan
> > to do that you need to use intptr_t so the compiler knows to keep the data
> > in a capability so it can use it as a pointer later.
>
> As a 'for instance' how does one take a capability and align it to a cache
> line boundary ?? Say in/after malloc() ?!?

I'm not sure what you mean:

Capabilities are 128-bit fields stored aligned in memory. It's not allowed
to store a capability that isn't 128-bit aligned. Those naturally align
with cache lines. Every 128 bits has a tag associated with it, stored
together or apart (various schemes discussed in my previous posts).

The memory it points to can be arbitraily aligned. It is just a 64-bit
pointer. You dereference it using 8/16/32/64/128 bit load and store
instructions in the usual datapath (either explicitly using 'C load'/'C
store' instructions or switching to a mode where every regular load/store
implicitly dereferences a capability rather than integer pointer)

The bounds have a certain representation limits, because they're packing
192+ bits of information into a 128 bit space. This boils down to an
alignment granularity: eg if you allocate a (1MiB+1) byte buffer the bounds
might be 1MiB+64 (or whatever, I can't remember what the rounding is at this
size). malloc() should ensure it doesn't hand out that memory to somebody
else; allocators typically do this anyway since they use slab allocators
which round up the allocation to a certain number of slabs.

There is a trickiness if somebody wants to generate a capability to a
subobject in the middle of a large object that isn't aligned: load in a
4.7GiB DVD wholesale into memory and try to generate a capability to a block
of frames in the middle of it, which is potentially large and yet the base
is unaligned, which would cause a loss of bounds precision (somebody could
access the frame before or after). It's possible to imagine things like
that, but we've not seen software actually do it.

I'm not sure how any of these relate to cache lines? Aside for ensuring the
caches store capabilities atomically and preserve tags, any time you
dereference them they work just like regular memory accesses.

If you mean you ask malloc for something you later want to align to a cache
line, you ask for something larger increment the pointer to be cache
aligned, in the normal way:

#include <cheriintrin.h>
....
// 64 byte cache lines
ptr = malloc(size+63); // leave extra space for rounding up
offset = ptr & 0x3F;
ptr += (0x40 - offset); // round up to cache line

and then increment the base bound to match the new position of 'ptr' and set
the top to be ptr+size:

ptr = cheri_bounds_set(ptr, size);

Theo

Theo Markettos wrote:

> MitchAlsup1 <mitchalsup@aol.com> wrote:
>> Theo Markettos wrote:
>>
>> > MitchAlsup1 <mitchalsup@aol.com> wrote:
>> >> BGB wrote:
>> >> <snip>
>> >> > You can make it work, yes, but the question is less "can you make it
>> >> > work, technically", but more:
>> >> > Can you make it work in a way that provides both a fairly normal C
>> >> > experience, and *also* an unbreakable sandbox, at the same time.
>>
>> > The C experience is fairly normal, as long as you are actually playing by
>> > the C rules. You can't arbitraily cast integers to pointers - if you plan
>> > to do that you need to use intptr_t so the compiler knows to keep the data
>> > in a capability so it can use it as a pointer later.
>>
>> As a 'for instance' how does one take a capability and align it to a cache
>> line boundary ?? Say in/after malloc() ?!?

> I'm not sure what you mean:

> Capabilities are 128-bit fields stored aligned in memory. It's not allowed
> to store a capability that isn't 128-bit aligned. Those naturally align
> with cache lines. Every 128 bits has a tag associated with it, stored
> together or apart (various schemes discussed in my previous posts).

> The memory it points to can be arbitraily aligned.

For performance reasons one would want it* cache line aligned.
(*) or some part of the whole thing aligned to a cache line boundary.

p = (p_type *)((intpt_t)p & ~63);
> It is just a 64-bit
> pointer. You dereference it using 8/16/32/64/128 bit load and store
> instructions in the usual datapath (either explicitly using 'C load'/'C
> store' instructions or switching to a mode where every regular load/store
> implicitly dereferences a capability rather than integer pointer)

> The bounds have a certain representation limits, because they're packing
> 192+ bits of information into a 128 bit space. This boils down to an
> alignment granularity: eg if you allocate a (1MiB+1) byte buffer the bounds
> might be 1MiB+64 (or whatever, I can't remember what the rounding is at this
> size). malloc() should ensure it doesn't hand out that memory to somebody
> else; allocators typically do this anyway since they use slab allocators
> which round up the allocation to a certain number of slabs.

So how to you "encode" a petaByte array ?? of megaByte structs in a capability ??

> There is a trickiness if somebody wants to generate a capability to a
> subobject in the middle of a large object that isn't aligned: load in a
> 4.7GiB DVD wholesale into memory and try to generate a capability to a block
> of frames in the middle of it, which is potentially large and yet the base
> is unaligned, which would cause a loss of bounds precision (somebody could
> access the frame before or after). It's possible to imagine things like
> that, but we've not seen software actually do it.

> I'm not sure how any of these relate to cache lines?

Smaller agglomerations of memory want to be cache-line aligned for performance
reasons. If a struct fits in 1 cache line you don't want it positioned so it
needs 2 cache lines in use.
> Aside for ensuring the
> caches store capabilities atomically and preserve tags, any time you
> dereference them they work just like regular memory accesses.

> If you mean you ask malloc for something you later want to align to a cache
> line, you ask for something larger increment the pointer to be cache
> aligned, in the normal way:

> #include <cheriintrin.h>
> ....
> // 64 byte cache lines
> ptr = malloc(size+63); // leave extra space for rounding up
> offset = ptr & 0x3F;
> ptr += (0x40 - offset); // round up to cache line

> and then increment the base bound to match the new position of 'ptr' and set
> the top to be ptr+size:

> ptr = cheri_bounds_set(ptr, size);

> Theo

On 3/10/2024 3:32 PM, Theo Markettos wrote:
> MitchAlsup1 <mitchalsup@aol.com> wrote:
>> Theo Markettos wrote:
>>
>>> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>>> BGB wrote:
>>>> <snip>
>>>>> You can make it work, yes, but the question is less "can you make it
>>>>> work, technically", but more:
>>>>> Can you make it work in a way that provides both a fairly normal C
>>>>> experience, and *also* an unbreakable sandbox, at the same time.
>>
>>> The C experience is fairly normal, as long as you are actually playing by
>>> the C rules. You can't arbitraily cast integers to pointers - if you plan
>>> to do that you need to use intptr_t so the compiler knows to keep the data
>>> in a capability so it can use it as a pointer later.
>>
>> As a 'for instance' how does one take a capability and align it to a cache
>> line boundary ?? Say in/after malloc() ?!?
>
> I'm not sure what you mean:
>
> Capabilities are 128-bit fields stored aligned in memory. It's not allowed
> to store a capability that isn't 128-bit aligned. Those naturally align
> with cache lines. Every 128 bits has a tag associated with it, stored
> together or apart (various schemes discussed in my previous posts).
>
> The memory it points to can be arbitraily aligned. It is just a 64-bit
> pointer. You dereference it using 8/16/32/64/128 bit load and store
> instructions in the usual datapath (either explicitly using 'C load'/'C
> store' instructions or switching to a mode where every regular load/store
> implicitly dereferences a capability rather than integer pointer)
>
> The bounds have a certain representation limits, because they're packing
> 192+ bits of information into a 128 bit space. This boils down to an
> alignment granularity: eg if you allocate a (1MiB+1) byte buffer the bounds
> might be 1MiB+64 (or whatever, I can't remember what the rounding is at this
> size). malloc() should ensure it doesn't hand out that memory to somebody
> else; allocators typically do this anyway since they use slab allocators
> which round up the allocation to a certain number of slabs.
>
> There is a trickiness if somebody wants to generate a capability to a
> subobject in the middle of a large object that isn't aligned: load in a
> 4.7GiB DVD wholesale into memory and try to generate a capability to a block
> of frames in the middle of it, which is potentially large and yet the base
> is unaligned, which would cause a loss of bounds precision (somebody could
> access the frame before or after). It's possible to imagine things like
> that, but we've not seen software actually do it.
>
> I'm not sure how any of these relate to cache lines? Aside for ensuring the
> caches store capabilities atomically and preserve tags, any time you
> dereference them they work just like regular memory accesses.
>
> If you mean you ask malloc for something you later want to align to a cache
> line, you ask for something larger increment the pointer to be cache
> aligned, in the normal way:
>
> #include <cheriintrin.h>
> ...
> // 64 byte cache lines
> ptr = malloc(size+63); // leave extra space for rounding up
> offset = ptr & 0x3F;
> ptr += (0x40 - offset); // round up to cache line
>
> and then increment the base bound to match the new position of 'ptr' and set
> the top to be ptr+size:
>
> ptr = cheri_bounds_set(ptr, size);

Something akin to my old alignment code:

https://groups.google.com/g/comp.lang.c/c/7oaJFWKVCTw/m/sSWYU9BUS_QJ

https://pastebin.com/raw/f37a23918

#define RALLOC_ALIGN_UP(mp_ptr, mp_align) \
((void*)( \
(((ralloc_uintptr_type)(mp_ptr)) + ((mp_align) - 1)) \
& ~(((mp_align) - 1)) \
))

Aligning and padding to cache lines is critical for certain algorithms.

On 3/10/2024 10:42 AM, Theo Markettos wrote:
> BGB <cr88192@gmail.com> wrote:
>>> There CHERI designs on silicon in existence
>>>
>>> https://www.morello-project.org/
>
> Member of the CHERI team here...
>
>> Main open question is if they can deliver enough on their claims in a
>> way that justifies the cost of the memory tagging (eg, where one needs
>> to tag whether or not each memory location holds a valid capability).
>
> Crudely speaking, it's 1/128 of memory so 0.78%. Through use of
> a tag cache you don't need to have a speciifc 129th bit on your DRAM
> (although that's an option, like ECC) but can just wall off a piece of
> regular DRAM and use it for a tag table that's accessed via the tag cache.
> You only pay the cost for DRAM you actually have (eg one tag table per DIMM,
> sized for that DIMM).
>
> The tag cache can also be compressed[1] so that if a memory page has no
> capabilities you don't need to store tags for it. This reduces tag cache
> bandwidth further.
>

Yeah, I was aware of it being 1 bit per 128 bits.

So, say, if you have a 256MB RAM module, this would be ~ 2MB of overhead.

For an FPGA implementation, there is a certain amount of LUT cost as well.

Also the bigger pointers, ...

>> As I see it, "locking things down" would likely require turning things
>> like "malloc()/free()", "dlopen()/dlsym()/...", etc, into system calls
>> (and generally giving the kernel a much more active role in this process).
>
> You don't need memory allocations to be system calls, because capabilities
> can be manipulated in userspace (which is a key design goal). ie if
> malloc() possesses a capability to 1MB of memory and a client requests 100
> bytes, it can take its 1MB capability, offset the base to point to the start
> of the allocation, set the offset to zero, and shrink the top to be
> base+100, and then return that 100-byte capability.
>
> In a Unix OS, userspace malloc() does occasionally need to make a syscall -
> when it runs out of memory pages on hand it has to ask the kernel to
> allocate it some more via mmap(). But this is orthogonal to use of
> capabilities (which are virtually addressed in such a system).
>
> For dynamic linking, you need something that holds capabilities that allow
> you to turn read-write memory into read-execute memory. That can be part of
> the dynamic linker or part of the OS. Since you are typically making
> syscalls to read the shared library from disc anyway, as well as marking RW
> pages as RX pages in the page table, the OS is already involved.
>

I am not claiming that one needs to have a syscall to make malloc work,
but rather, I am not certain if one can make malloc entirely secure
without a syscall...

>> But, say, note that it would not be sufficient, say, for things like
>> sandboxing hostile code within a shared address space with another
>> program that needs to be kept protected.
>>
>>
>> Granted, the strength could likely be improved (in the face of trying to
>> prevent hostile code from being able to steal capabilities) through
>> creative use of ASLR. Along with ABI features, such as "scratch register
>> scrubbing" (say, loading zeroes into scratch registers on function
>> return, such as to prevent capabilities from being leaked through via
>> registers), marking function pointers as "Execute Only" etc.
>>
>> As noted, a capability system would likely still be pretty strong
>> against things like buffer overflows (but if only being used to mitigate
>> buffer overflows, is a bit overkill; so the main "interesting" case is
>> if it can be used to make an "unbreakable sandbox" for potentially
>> hostile machine code).
>
> Code capabilities prevent a lot of control flow attacks, because you can
> only execute code you have capabilities to. For example you're in a
> function - you possess a return address capability (which has bounds based
> on your local environment or compartment you're in) so you can manipulate
> the address of that return address, but you can't jump to arbitrary code and
> you can't forge return addresses. So no stack smashing, no ROP/JOP attacks,
> etc.
>
> Through setting the bounds on a code capability you can sandbox small pieces
> of code to a function granularity, which is more efficient than typical MMU
> sandboxing.
>

Presumably, in addition to the code, one needs some way for the code to
be able to access its own ".data" and ".bss" sections when called.

Some options:
PC-relative:
Unclear if valid in this case.
GOT:
Table of pointers to things, loaded somehow.
One example here being the ELF FDPIC ABI.
Reloading a Global Pointer via a lookup table accessed via itself.
This is what my ABI uses...

I couldn't seem to find any technical descriptions of the CHERI/Morello
ABI. I had made a guess that it might work similar to FDPIC, as this
could be implemented without needing to use raw addresses (and seemed
like a "best fit").

But, an FDPIC style approach would have a weakness: One can get the GOT
from a function pointer...

Granted, with an ABI design (like mine) based on daisy-chaining via the
Global Pointer, one can also use the global pointer in unorthodox ways.

If capabilities give bounds information, then if there is a way for the
program to distinguish valid from invalid capabilities, then a hostile
program could perform searches similar to those in a conservative
garbage collector for finding capabilities that it may be able to use.

Though, if the program can't tell the difference (probably more secure)
then it may be possible to pepper "sensitive areas" with false positive
capabilities that look valid, but will crash the program if de-referenced.

>> *: If it is possible to perform a Load or (worse, Capability Load)
>> through a function pointer, this is likely to be a significant attack
>> vector. Need to make it so that function pointers can only be used to
>> call things. Protecting against normal data loads would be needed mostly
>> to try to prevent code from being able to gain access to a known pointer
>> and possibly side-step the ASLR (say, if it can figure out that the
>> address it wants to access is reachable from a capability that the code
>> has access to).
>
> You can't arbitrarily change data to code capabilities - they are different
> types.
>

Possible, but:
How is the C ABI implemented in this case?
....

>> It is likely that the capability memory tagging would need to be managed
>> by the L2 cache. Would need some mechanism for the tag-bits memory (say,
>> 2MB for 256MB at 1b per 16B line). Would also need to somehow work this
>> flag bit into the ringbus messaging.
>
> AXI has user fields which can be used for sending capabilities across the
> interconnect, through third party AXI IPs like muxes, arbiters, etc.
>

I was thinking if I did an implementation of a capability system in my
CPU core (as an extension of my existing bounds-checked pointer scheme).
Possibly with a special CPU mode that actively enforces this system (vs
just treating the bounds-checking system as opt-in at the machine-code
level).

In this case, I am not using the AXI bus, rather a custom bus built on
passing messages along a circular ring.

The existing bus has an interface where one has input/output connections:
Data, 128 bit;
Addr, 48 bit;
OPM (Request and Flags), 16 bits.
Low 8 bits holds the request/response/command code;
High 8 bits holds flags.
SEQ (Source/Destination and a Sequence Number), 16 bits.

At first, started looking at using different Request/Response IDs for
data and capabilities.

Did notice though, that the bit pattern for request/response flags was:
(7): Random Noise Bit
(6): Prefetch Hit
(5): Write Through
(4): No User
(3): No Cache
(2): No Execute
(1): No Write
(0): No Read

Had the idea that the Prefetch bit could be overloaded as a capability
flag for non prefetch messages.

Though, the other possibility is to use different request/response codes:
97/A7: Load/Store Data
AF: Store Capability
70: Loaded Data (Response)
60: Stored Data (Response)
78: Loaded Capability (Response)
...

Previously, the low 4 bits of the response mirrored bits (11:8), change
was to use exclusively (11:8) and leave the low order bits

Did modify my bus signaling slightly to allow for this, but somewhere on
the way my CPU core has partially broken (it has started crashing, and I
have yet to figure out what exactly has broken).

Appears that memory Loads and Stores are failing in some edge case, but
is being difficult to nail down.