On Monday, June 12, 2023 at 2:32:54 AM UTC-5, Anton Ertl wrote:
> MitchAlsup <Mitch...@aol.com> writes:
> >On Sunday, June 11, 2023 at 10:57:39=E2=80=AFAM UTC-5, Anton Ertl wrote:
> >> MitchAlsup <Mitch...@aol.com> writes:=20
> >> >In 64-bit architectures both are single registers.
> ><
> >> Yes, that's what makes the simple and efficient JVM implementation=20
> >> impossible. In the JVM int and float take one stack slot, while long=20
> >> and double take two (it was originally designed for being implemented=20
> >> with a 32-bit stack slot). There are JVM instructions working on the=20
> >> stack that do not include the types of the stack operand. So either=20
> >> you make the implementation complicated by doing compile-time type=20
> >> tracking, or you make it inefficient by keeping longs and doubles in=20
> >> two registers, and having instructions that work on them combine these=20
> >> two registers into one operand (once per operand), and split the=20
> >> result.=20
> ><
> >I don't get your point:: On the stack the int and float containers are
> >word size, after being loaded into a register they are still 32-bit
> >things but occupy a 64-bit container.
> Java int and float are not the problem, long and double are. You want
> them in one register on a 64-bit architecture, but a simple
> implementation will split them into two.
> >If you fetch+decode+issue+retire 4 inst per clock and they are all
> >integer you need 4 sets of integer ports, a few cycles later you FDIR
> >4 floating point inst in a cycle you need 4 sets of FP ports. Now, if
> >you FDIR 2 ints and 2 FPs you have 8 total ports and =C2=BD are going=20
> >unused.
>
> Maybe this made-up core has a front end that is too narrow for the
> execution engine. Let's look at the Cortex-X4:
>
> It has a dispatch width of 10 instructions, 8 ALUs (2 of which are
> also integer MACs), 3 branch units, 4 FP/SIMD units, and 4 memory
> units (2 LD, 1 ST, 1 LD/ST).
>
> I don't know how many read and write ports the register files on the
> Cortex-X4 have, but I am sure that if you combined the register files,
> and kept the ports the same as for the register file with the larger
> number of ports, you would see slowdowns from running out of ports.
>
> >> Have you also tried compiling code for the VAX? How does that compare=20
> >> to My66000 and RISC-V? The point: Why should the instruction count be=20
> >> particularly relevant?=20
> ><
> >I have around 12% more instructions than VAX. RISC-V has close to 150%
> >the instruction count of VAX.
> ><
> >So if VAX needs 100 instructions, My 66000 needs 112 instructions, and
> >RISC-V needs 150 instructions.
<
> So My 66000 has 112% the instruction count of VAX, while RISC-V has
> 50% more instructions than VAX:-).
<
At handwaving accuracy: Yes; the ratio between My 66000 and IRSC-V
is solid that of VAX much less so.
<
In 1983-ish, Mark Horowitz remarked that <Stanford> MIPS executed 50%
more instructions than VAX at 6× its frequency for a 4× performance
advantage.
<
I have lots of good data My 66000 versus RISC-V (using LLVM compilers)
and know this relationship with low noise. To bad this NG does not
supports *.jpg or I could show it.
<
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On 6/9/2023 8:19 PM, John Dallman wrote:
> In article <u5vrjb$1vorb$1@dont-email.me>, sfuld@alumni.cmu.edu.invalid
> (Stephen Fuld) wrote:
>
>> Have you looked to see if VVM would help you?
>
> AFAIK there's no prospect of hardware that actually supports VVM. Am I
> wrong about that?

In thinking about how to respond, I realized that I had completely
missed the point of your previous post, and for that I apologize.

The question is not which hardware architecture does better on your
code, but whether a compiler recognizes your structs as "eligible" for
whatever vectorization the target hardware provides. I gather that, so
far, none do.

That being said, I have been thinking about the benefits of VVM,
assuming the compiler would recognize it. I assume a common task is,
given two points, each described by a three element structure (one 64
bit floating point value element per dimension), compute the Euclidean
distance between the points. The question then is would VVM give better
performance than a GBOoO machine with equivalent number of FUs.

I think the answer is that VVM would provide some benefit, derived from
its smaller code footprint and streaming loads, but it would be modest
due to only looping three times.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Monday, June 12, 2023 at 1:34:08 PM UTC-5, Stephen Fuld wrote:
> On 6/9/2023 8:19 PM, John Dallman wrote:
> > In article <u5vrjb$1vorb$1...@dont-email.me>, sf...@alumni.cmu.edu.invalid
> > (Stephen Fuld) wrote:
> >
> >> Have you looked to see if VVM would help you?
> >
> > AFAIK there's no prospect of hardware that actually supports VVM. Am I
> > wrong about that?
> In thinking about how to respond, I realized that I had completely
> missed the point of your previous post, and for that I apologize.
>
> The question is not which hardware architecture does better on your
> code, but whether a compiler recognizes your structs as "eligible" for
> whatever vectorization the target hardware provides. I gather that, so
> far, none do.
>
> That being said, I have been thinking about the benefits of VVM,
> assuming the compiler would recognize it. I assume a common task is,
> given two points, each described by a three element structure (one 64
> bit floating point value element per dimension), compute the Euclidean
> distance between the points. The question then is would VVM give better
> performance than a GBOoO machine with equivalent number of FUs.
>
> I think the answer is that VVM would provide some benefit, derived from
> its smaller code footprint and streaming loads, but it would be modest
> due to only looping three times.
<
Think about it like this::
<
VVM allows a lowly 1-wide implementation to perform at vector rates
1 instruction per clock per function unit.
<
At the GBOoO side of things, VVM allows you to seed the loop in the
instruction queues once, and fire them all once per cycle without paying
the fetch-decode-issue power overheads. You can also recirculate the
register renaming from iteration to iteration by concatenating a modulo
loop index to the register renames.
<
At both small end and big end, VVM also allows multiple iterations of
the loop to begin each cycle (SIMD style).
<
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
> MitchAlsup <MitchAlsup@aol.com> writes:
>>On Sunday, June 11, 2023 at 10:57:39=E2=80=AFAM UTC-5, Anton Ertl wrote:
>>> MitchAlsup <Mitch...@aol.com> writes:=20
>>> >In 64-bit architectures both are single registers.
>><
>>> Yes, that's what makes the simple and efficient JVM implementation=20
>>> impossible. In the JVM int and float take one stack slot, while long=20
>>> and double take two (it was originally designed for being implemented=20
>>> with a 32-bit stack slot). There are JVM instructions working on the=20
>>> stack that do not include the types of the stack operand. So either=20
>>> you make the implementation complicated by doing compile-time type=20
>>> tracking, or you make it inefficient by keeping longs and doubles in=20
>>> two registers, and having instructions that work on them combine these=20
>>> two registers into one operand (once per operand), and split the=20
>>> result.=20
>><
>>I don't get your point:: On the stack the int and float containers are
>>word size, after being loaded into a register they are still 32-bit
>>things but occupy a 64-bit container.
>
> Java int and float are not the problem, long and double are. You want
> them in one register on a 64-bit architecture, but a simple
> implementation will split them into two.

Hm... so a solution could be to write a sophisticated implementation
instead?

Thomas Koenig <tkoenig@netcologne.de> writes:
>Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>> Java int and float are not the problem, long and double are. You want
>> them in one register on a 64-bit architecture, but a simple
>> implementation will split them into two.
>
>Hm... so a solution could be to write a sophisticated implementation
>instead?

Yes.

Now let's return to our original question of finding some scenario
where a combined integer/FP register file has an advantage: The
sophisticated JVM implementation works fine with separated integer and
FP registers.

So the combined register file provides the advantage of allowings
simple, efficient JVM implementations is there only for 32-bit
architectures (i.e., the 88000).

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Tuesday, June 13, 2023 at 12:32:19 PM UTC-5, Anton Ertl wrote:
> Thomas Koenig <tko...@netcologne.de> writes:
> >Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
> >> Java int and float are not the problem, long and double are. You want
> >> them in one register on a 64-bit architecture, but a simple
> >> implementation will split them into two.
> >
> >Hm... so a solution could be to write a sophisticated implementation
> >instead?
> Yes.
>
> Now let's return to our original question of finding some scenario
> where a combined integer/FP register file has an advantage: The
> sophisticated JVM implementation works fine with separated integer and
> FP registers.
<
Advantage: Area, utility, code size.
>
> So the combined register file provides the advantage of allowings
> simple, efficient JVM implementations is there only for 32-bit
> architectures (i.e., the 88000).
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
> Thomas Koenig <tkoenig@netcologne.de> writes:
>>Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>>> Java int and float are not the problem, long and double are. You want
>>> them in one register on a 64-bit architecture, but a simple
>>> implementation will split them into two.
>>
>>Hm... so a solution could be to write a sophisticated implementation
>>instead?
>
> Yes.
>
> Now let's return to our original question of finding some scenario
> where a combined integer/FP register file has an advantage:

[...]

Complex division with scaling.

(a,b)/(c,d) = ((a*c+b*d)/(c**2 + d**2), (b*c-a*d)/(c**2 + d**2))

This is easiest to do without loss of precision if you scale all
real and imagniary parts of the numbers exactly by a power of two,
so that no artificial overflow occurs.

In article <u67oes$36jnv$1@dont-email.me>, sfuld@alumni.cmu.edu.invalid
(Stephen Fuld) wrote:

> In thinking about how to respond, I realized that I had completely
> missed the point of your previous post, and for that I apologize.

Thank you.

> The question is not which hardware architecture does better on your
> code, but whether a compiler recognizes your structs as "eligible"
> for whatever vectorization the target hardware provides. I gather
> that, so far, none do.

Not last time I tried.

> That being said, I have been thinking about the benefits of VVM,
> assuming the compiler would recognize it. I assume a common task
> is, given two points, each described by a three element structure
> (one 64 bit floating point value element per dimension), compute
> the Euclidean distance between the points. The question then is
> would VVM give better performance than a GBOoO machine with
> equivalent number of FUs.
>
> I think the answer is that VVM would provide some benefit, derived
> from its smaller code footprint and streaming loads, but it would
> be modest due to only looping three times.

The small sizes of the structs are a problem for any kind of
vectorisation.

John

On Tuesday, June 13, 2023 at 3:52:37 PM UTC-5, John Dallman wrote:
> In article <u67oes$36jnv$1...@dont-email.me>, sf...@alumni.cmu.edu.invalid
> (Stephen Fuld) wrote:
>
> > In thinking about how to respond, I realized that I had completely
> > missed the point of your previous post, and for that I apologize.
> Thank you.
> > The question is not which hardware architecture does better on your
> > code, but whether a compiler recognizes your structs as "eligible"
> > for whatever vectorization the target hardware provides. I gather
> > that, so far, none do.
> Not last time I tried.
> > That being said, I have been thinking about the benefits of VVM,
> > assuming the compiler would recognize it. I assume a common task
> > is, given two points, each described by a three element structure
> > (one 64 bit floating point value element per dimension), compute
> > the Euclidean distance between the points. The question then is
> > would VVM give better performance than a GBOoO machine with
> > equivalent number of FUs.
> >
> > I think the answer is that VVM would provide some benefit, derived
> > from its smaller code footprint and streaming loads, but it would
> > be modest due to only looping three times.
<
> The small sizes of the structs are a problem for any kind of
> vectorisation.
<
Can you show a short code snippet that illustrates that claim ??
>
>
> John

In article <14159ddc-e516-414b-be70-d96ab898ca8dn@googlegroups.com>,
MitchAlsup@aol.com (MitchAlsup) wrote:
> On Tuesday, June 13, 2023 at 3:52:37_PM UTC-5, John Dallman wrote:
> > The small sizes of the structs are a problem for any kind of
> > vectorisation.
> Can you show a short code snippet that illustrates that claim ??

Sorry, spoke carelessly. The short vectors limit the gains from
vectorisation, and put a premium on minimal setup costs.

John

On Wednesday, June 14, 2023 at 3:12:04 PM UTC+1, John Dallman wrote:
> In article <14159ddc-e516-414b...@googlegroups.com>,
> Mitch...@aol.com (MitchAlsup) wrote:
> > On Tuesday, June 13, 2023 at 3:52:37_PM UTC-5, John Dallman wrote:
> > > The small sizes of the structs are a problem for any kind of
> > > vectorisation.
> > Can you show a short code snippet that illustrates that claim ??
> Sorry, spoke carelessly. The short vectors limit the gains from
> vectorisation, and put a premium on minimal setup costs.

in GPU-augmented front-end compilers (from the Khronos Group)
vec2/3/4 is a first-order construct, as is Swizzle in order to
cross-over RRRA or YXWW.

SVP64 has double-nested for-loops
....for i = 0 to VL-1
.......for j = 0 to SUBVL-1

as a hardware-level construct that is supported up to the ISA.

it can be done: it just needs someone to pay attention, and
unfortunately John you're using a General-Purpose ISA with
General-Purpose compilers, neither of which was ever intended
for or designed with 3D GPU workloads in mind.

the last ISA i heard of that was capable of such "Unified"
programming was the one from ICubeCorp. they had an
ex-SGI Compiler expert on board (Simon?) who knew VLIW
inside-out.

l.

luke.l...@gmail.com wrote:
> On Wednesday, June 14, 2023 at 3:12:04 PM UTC+1, John Dallman wrote:
>> In article <14159ddc-e516-414b...@googlegroups.com>,
>> Mitch...@aol.com (MitchAlsup) wrote:
>>> On Tuesday, June 13, 2023 at 3:52:37_PM UTC-5, John Dallman wrote:
>>>> The small sizes of the structs are a problem for any kind of
>>>> vectorisation.
>>> Can you show a short code snippet that illustrates that claim ??
>> Sorry, spoke carelessly. The short vectors limit the gains from
>> vectorisation, and put a premium on minimal setup costs.
>
> in GPU-augmented front-end compilers (from the Khronos Group)
> vec2/3/4 is a first-order construct, as is Swizzle in order to
> cross-over RRRA or YXWW.
>
> SVP64 has double-nested for-loops
> ...for i = 0 to VL-1
> ......for j = 0 to SUBVL-1
>
> as a hardware-level construct that is supported up to the ISA.
>
> it can be done: it just needs someone to pay attention, and
> unfortunately John you're using a General-Purpose ISA with
> General-Purpose compilers, neither of which was ever intended
> for or designed with 3D GPU workloads in mind.
>
> the last ISA i heard of that was capable of such "Unified"
> programming was the one from ICubeCorp. they had an
> ex-SGI Compiler expert on board (Simon?) who knew VLIW
> inside-out.

What about Larrabee? This has since morphed more or less into AVX-512,
but the original idea was to have a CPU which could also do GPU style
work, with cache lines as the unit of work.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On 6/9/23 1:44 PM, Scott Lurndal wrote:
[snip]
> Say, like Infiniband (which we used at 3leaf as the coherency transport between
> nodes). The most interesting characteristic of IB was the low switching
> latency (less than 100ns cut-through) when compared with 10Gigabit ethernet
> runing at similar speeds. This was due to the routing mechanisms used
> by infiniband (where the entire route was encapsulated in the packet,
> with each hop discarding the its destination in the packet
> header until the final destination was reached). RDMA was a bonus.

I wonder why the big datacenter organizations have not developed
such simpler routing for mostly-Ethernet-compatibility switches
and "cards". Perhaps something like DHCP could assign MAC
addresses to facilitate routing. (A DHCP-like mechanism would
obviously have overhead for reconnecting, but that seems likely to
be uncommon.) Such organizations seem to have the scale to support
semi-custom chips.

This would not just reduce latency but also (presumably) reduce
power as look-up tables would not be needed.

Since this seem obvious, presumably it is either already done or
the cost of implementation is higher than suspected by me (or the
benefits even lower than suspected). (Sometimes I wish I knew more
about more things — which is one reason why I appreciate others
casually sharing knowledge.)

In article <01c767e1-70c7-49b0-832f-38d45c957c11n@googlegroups.com>,
luke.leighton@gmail.com () wrote:

> it can be done: it just needs someone to pay attention, and
> unfortunately John you're using a General-Purpose ISA with
> General-Purpose compilers, neither of which was ever intended
> for or designed with 3D GPU workloads in mind.

Unfortunately, the 3D vectors work is only a part of what this modeller
does. It mixes that with pretty complex integer logic and data management.
Putting it all on a GPU-style architecture would be far worse than the
current situation.

> the last ISA i heard of that was capable of such "Unified"
> programming was the one from ICubeCorp. they had an
> ex-SGI Compiler expert on board (Simon?) who knew VLIW
> inside-out.

http://icubecorp.com/

The English-language website was last updated in 2014, although the
domain registration is up-to-date. As far as I can tell with Google
Translate, the Chinese site stopped updating at about the same time.

John

"Paul A. Clayton" <paaronclayton@gmail.com> writes:

> I wonder why the big datacenter organizations have not developed
> such simpler routing for mostly-Ethernet-compatibility switches
> and "cards".

It is called Segment Routing, and it is available over MPLS or
IPv6. Although you get pretty much the same benefit with plain MPLS.

It does not, however, turn out to be that much lower latency than
regular routing and switching, and at 100Gbps+ the FEC latency starts to
dominate.

It looks like routing/switching latency will be stuck in the low
hundreds of ns for a while.

/Benny

On Tuesday, June 20, 2023 at 1:13:11 PM UTC-4, Paul A. Clayton wrote:
> On 6/9/23 1:44 PM, Scott Lurndal wrote:
> [snip]
> > Say, like Infiniband (which we used at 3leaf as the coherency transport between
> > nodes). The most interesting characteristic of IB was the low switching
> > latency (less than 100ns cut-through) when compared with 10Gigabit ethernet
> > runing at similar speeds. This was due to the routing mechanisms used
> > by infiniband (where the entire route was encapsulated in the packet,
> > with each hop discarding the its destination in the packet
> > header until the final destination was reached). RDMA was a bonus.
> I wonder why the big datacenter organizations have not developed
> such simpler routing for mostly-Ethernet-compatibility switches
> and "cards". Perhaps something like DHCP could assign MAC
> addresses to facilitate routing. (A DHCP-like mechanism would
> obviously have overhead for reconnecting, but that seems likely to
> be uncommon.) Such organizations seem to have the scale to support
> semi-custom chips.
>
> This would not just reduce latency but also (presumably) reduce
> power as look-up tables would not be needed.
>
> Since this seem obvious, presumably it is either already done or
> the cost of implementation is higher than suspected by me (or the
> benefits even lower than suspected). (Sometimes I wish I knew more
> about more things — which is one reason why I appreciate others
> casually sharing knowledge.)

The next-hop routing Scott is describing is only used by the Subnet Manager when it is crawling the network topology to discover devices - i.e. go one hop out, if it's a switch, enumerate each port and go an additional hop, repeat. It goes through the Subnet Management Agent on a switch which is typically running on a very underpowered embedded processor and is dog slow. The rest of the time, IB uses LID-based routing (a 16-bit logical address, only up to the first 48k addresses are valid unicast addresses). There is also another addressing method that uses a 40 byte Global Routing Header, but I've only seen that used in geographically dispersed IB networks.

The main reason IB has lower latency is 1) a 16-bit LID is smaller than a 48-bit MAC and you can have a very small ram with all possible LIDs and their associated outbound ports as opposed to a bigger, slower TCAM, and 2) the IB market is VERY interested in that metric rather than layer-3,4,5,.. deep packet inspection or the 500 other things modern datacenter switches are expected to do.

-JohnG

On 6/20/23 7:12 PM, JohnG wrote:
> On Tuesday, June 20, 2023 at 1:13:11 PM UTC-4, Paul A. Clayton wrote:
>> On 6/9/23 1:44 PM, Scott Lurndal wrote:
>> [snip]
>>> Say, like Infiniband (which we used at 3leaf as the coherency transport between
>>> nodes). The most interesting characteristic of IB was the low switching
>>> latency (less than 100ns cut-through) when compared with 10Gigabit ethernet
>>> runing at similar speeds. This was due to the routing mechanisms used
>>> by infiniband (where the entire route was encapsulated in the packet,
>>> with each hop discarding the its destination in the packet
>>> header until the final destination was reached). RDMA was a bonus.
>> I wonder why the big datacenter organizations have not developed
>> such simpler routing for mostly-Ethernet-compatibility switches
>> and "cards". Perhaps something like DHCP could assign MAC
>> addresses to facilitate routing. (A DHCP-like mechanism would
>> obviously have overhead for reconnecting, but that seems likely to
>> be uncommon.) Such organizations seem to have the scale to support
>> semi-custom chips.
>>
>> This would not just reduce latency but also (presumably) reduce
>> power as look-up tables would not be needed.
>>
>> Since this seem obvious, presumably it is either already done or
>> the cost of implementation is higher than suspected by me (or the
>> benefits even lower than suspected). (Sometimes I wish I knew more
>> about more things — which is one reason why I appreciate others
>> casually sharing knowledge.)
>
> The next-hop routing Scott is describing is only used by the Subnet
> Manager when it is crawling the network topology to discover devices - > i.e. go one hop out, if it's a switch, enumerate each port and
go an
> additional hop, repeat. It goes through the Subnet Management Agent
> on a switch which is typically running on a very underpowered
> embedded processor and is dog slow. The rest of the time, IB uses
> LID-based routing (a 16-bit logical address, only up to the first
> 48k addresses are valid unicast addresses). There is also another
> addressing method that uses a 40 byte Global Routing Header, but
>I've only seen that used in geographically dispersed IB networks.
>
> The main reason IB has lower latency is 1) a 16-bit LID is
> smaller than a 48-bit MAC and you can have a very small ram with
> all possible LIDs and their associated outbound ports as opposed
> to a bigger, slower TCAM, and

My (small) point was that an extended Ethernet specialized for
large datacenter operators (who could afford the specialization)
could use something like DHCP to give each end-node a trivially
routable address. For such addresses no table/TCAM would be needed
(and an arbitrary number of bits could be ignored/always be zero;
I do not know whether using a different packet format would be
practical). I.e., "ethernet" could mimic this aspect of
Infiniband.

> 2) the IB market is VERY interested in that metric rather than
> layer-3,4,5,.. deep packet inspection or the 500 other things
> modern datacenter switches are expected to do.

This might well be the primary hindrance. Generality is often
more attractive than efficiency achieved with specialization. Even
so, there might be some scope for specialization (without eye-
watering pricing). (Economics seems to hinder such. It is more
profitable to sell specialized products to those who *need* the
features than to sell modestly more expensive fuller-featured base
products even when the system would be better off having broad
availability of such features.)

Anyway, it seems that it is not so much a _technical_ question as
an economics question: market demand et al. Since this is well
outside of my usual reading, I appreciate the information (even if
it is disappointing to the desire to approach the best technically
possible solution — the information also points to other features
[like deep packet inspection] having more value generally, which
reduces my ignorance).

On Sunday, June 25, 2023 at 8:41:09 AM UTC-7, Paul A. Clayton wrote:
> On 6/20/23 7:12 PM, JohnG wrote:
> > On Tuesday, June 20, 2023 at 1:13:11 PM UTC-4, Paul A. Clayton wrote:
> >> On 6/9/23 1:44 PM, Scott Lurndal wrote:
> >> [snip]
> >>> Say, like Infiniband (which we used at 3leaf as the coherency transport between
> >>> nodes). The most interesting characteristic of IB was the low switching
> >>> latency (less than 100ns cut-through) when compared with 10Gigabit ethernet
> >>> runing at similar speeds. This was due to the routing mechanisms used
> >>> by infiniband (where the entire route was encapsulated in the packet,
> >>> with each hop discarding the its destination in the packet
> >>> header until the final destination was reached). RDMA was a bonus.
> >> I wonder why the big datacenter organizations have not developed
> >> such simpler routing for mostly-Ethernet-compatibility switches
> >> and "cards". Perhaps something like DHCP could assign MAC
> >> addresses to facilitate routing. (A DHCP-like mechanism would
> >> obviously have overhead for reconnecting, but that seems likely to
> >> be uncommon.) Such organizations seem to have the scale to support
> >> semi-custom chips.
> >>
> >> This would not just reduce latency but also (presumably) reduce
> >> power as look-up tables would not be needed.
> >>
> >> Since this seem obvious, presumably it is either already done or
> >> the cost of implementation is higher than suspected by me (or the
> >> benefits even lower than suspected). (Sometimes I wish I knew more
> >> about more things — which is one reason why I appreciate others
> >> casually sharing knowledge.)
> >
> > The next-hop routing Scott is describing is only used by the Subnet
> > Manager when it is crawling the network topology to discover devices - > i.e. go one hop out, if it's a switch, enumerate each port and
> go an
> > additional hop, repeat. It goes through the Subnet Management Agent
> > on a switch which is typically running on a very underpowered
> > embedded processor and is dog slow. The rest of the time, IB uses
> > LID-based routing (a 16-bit logical address, only up to the first
> > 48k addresses are valid unicast addresses). There is also another
> > addressing method that uses a 40 byte Global Routing Header, but
> >I've only seen that used in geographically dispersed IB networks.
> >
> > The main reason IB has lower latency is 1) a 16-bit LID is
> > smaller than a 48-bit MAC and you can have a very small ram with
> > all possible LIDs and their associated outbound ports as opposed
> > to a bigger, slower TCAM, and
> My (small) point was that an extended Ethernet specialized for
> large datacenter operators (who could afford the specialization)
> could use something like DHCP to give each end-node a trivially
> routable address. For such addresses no table/TCAM would be needed
> (and an arbitrary number of bits could be ignored/always be zero;
> I do not know whether using a different packet format would be
> practical). I.e., "ethernet" could mimic this aspect of
> Infiniband.
> > 2) the IB market is VERY interested in that metric rather than
> > layer-3,4,5,.. deep packet inspection or the 500 other things
> > modern datacenter switches are expected to do.
> This might well be the primary hindrance. Generality is often
> more attractive than efficiency achieved with specialization. Even
> so, there might be some scope for specialization (without eye-
> watering pricing). (Economics seems to hinder such. It is more
> profitable to sell specialized products to those who *need* the
> features than to sell modestly more expensive fuller-featured base
> products even when the system would be better off having broad
> availability of such features.)
>
> Anyway, it seems that it is not so much a _technical_ question as
> an economics question: market demand et al. Since this is well
> outside of my usual reading, I appreciate the information (even if
> it is disappointing to the desire to approach the best technically
> possible solution — the information also points to other features
> [like deep packet inspection] having more value generally, which
> reduces my ignorance).

You're right in spirit - optimizing for datacenter (and now, AI - which looks a lot like classic HPC) has merit and a bunch of orgs have been doing that in-house for a while. It feels a lot like the late 90's where there were a bunch of boutique HPC interconnects and eventually it drove a few standardization efforts which ultimately led to InfiniBand. Now we appear to have the Ultra Ethernet Consortium. Here are a couple refs:

https://www.nextplatform.com/2023/07/20/ethernet-consortium-shoots-for-1-million-node-clusters-that-beat-infiniband/
(the links below are also all in-line in the above article)
https://ultraethernet.org/wp-content/uploads/sites/20/2023/07/23.07.12-UEC-1.0-Overview-FINAL-WITH-LOGO.pdf
https://www.nextplatform.com/2023/04/26/broadcom-takes-on-infiniband-with-jericho3-ai-switch-chips/
https://www.nextplatform.com/2023/06/22/cisco-guns-for-infiniband-with-silicon-one-g200/
https://www.nextplatform.com/2023/02/16/luminaries-argue-for-the-interconnect-we-could-have-already-had/
https://www.nextplatform.com/2022/04/12/with-aquila-google-abandons-ethernet-to-outdo-infiniband/

-JohnG

On Thursday, June 15, 2023 at 9:00:43 AM UTC+1, Terje Mathisen wrote:

> > the last ISA i heard of that was capable of such "Unified"
> > programming was the one from ICubeCorp. they had an
> > ex-SGI Compiler expert on board (Simon?) who knew VLIW
> > inside-out.
> What about Larrabee? This has since morphed more or less into AVX-512,
> but the original idea was to have a CPU which could also do GPU style
> work, with cache lines as the unit of work.

sorry, am not keeping up. larrabee failed commercially because
they made some poor decisions on a costly (slow) iterative feedback
loop. 1) they couldn't get good enough GPU performance so went
for "raw compute" instead only to find that 2) scientific computing
users were interested in FP64 where they'd of course put FP32 into
silicon.

jeff bush's nyuzi work (he has a whole blog) explains why larrabee's
approach as a "soft-core-only GPU with insufficient specialised
actual 3D GPU capabilities" was a scant 25% of the power-performance
ratio of other commercial hard-accelerated 3D GPU offerings...
exactly as **any** general-purpose "Vector" ISA will ever be.

tom forsyth's video "design lifecyle of a instruction set" is really
informative and also very funny.

l.

On Friday, July 21, 2023 at 4:26:42 AM UTC+3, JohnG wrote:
> On Sunday, June 25, 2023 at 8:41:09 AM UTC-7, Paul A. Clayton wrote:
> > On 6/20/23 7:12 PM, JohnG wrote:
> > > On Tuesday, June 20, 2023 at 1:13:11 PM UTC-4, Paul A. Clayton wrote:
> > >> On 6/9/23 1:44 PM, Scott Lurndal wrote:
> > >> [snip]
> > >>> Say, like Infiniband (which we used at 3leaf as the coherency transport between
> > >>> nodes). The most interesting characteristic of IB was the low switching
> > >>> latency (less than 100ns cut-through) when compared with 10Gigabit ethernet
> > >>> runing at similar speeds. This was due to the routing mechanisms used
> > >>> by infiniband (where the entire route was encapsulated in the packet,
> > >>> with each hop discarding the its destination in the packet
> > >>> header until the final destination was reached). RDMA was a bonus.
> > >> I wonder why the big datacenter organizations have not developed
> > >> such simpler routing for mostly-Ethernet-compatibility switches
> > >> and "cards". Perhaps something like DHCP could assign MAC
> > >> addresses to facilitate routing. (A DHCP-like mechanism would
> > >> obviously have overhead for reconnecting, but that seems likely to
> > >> be uncommon.) Such organizations seem to have the scale to support
> > >> semi-custom chips.
> > >>
> > >> This would not just reduce latency but also (presumably) reduce
> > >> power as look-up tables would not be needed.
> > >>
> > >> Since this seem obvious, presumably it is either already done or
> > >> the cost of implementation is higher than suspected by me (or the
> > >> benefits even lower than suspected). (Sometimes I wish I knew more
> > >> about more things — which is one reason why I appreciate others
> > >> casually sharing knowledge.)
> > >
> > > The next-hop routing Scott is describing is only used by the Subnet
> > > Manager when it is crawling the network topology to discover devices - > i.e. go one hop out, if it's a switch, enumerate each port and
> > go an
> > > additional hop, repeat. It goes through the Subnet Management Agent
> > > on a switch which is typically running on a very underpowered
> > > embedded processor and is dog slow. The rest of the time, IB uses
> > > LID-based routing (a 16-bit logical address, only up to the first
> > > 48k addresses are valid unicast addresses). There is also another
> > > addressing method that uses a 40 byte Global Routing Header, but
> > >I've only seen that used in geographically dispersed IB networks.
> > >
> > > The main reason IB has lower latency is 1) a 16-bit LID is
> > > smaller than a 48-bit MAC and you can have a very small ram with
> > > all possible LIDs and their associated outbound ports as opposed
> > > to a bigger, slower TCAM, and
> > My (small) point was that an extended Ethernet specialized for
> > large datacenter operators (who could afford the specialization)
> > could use something like DHCP to give each end-node a trivially
> > routable address. For such addresses no table/TCAM would be needed
> > (and an arbitrary number of bits could be ignored/always be zero;
> > I do not know whether using a different packet format would be
> > practical). I.e., "ethernet" could mimic this aspect of
> > Infiniband.
> > > 2) the IB market is VERY interested in that metric rather than
> > > layer-3,4,5,.. deep packet inspection or the 500 other things
> > > modern datacenter switches are expected to do.
> > This might well be the primary hindrance. Generality is often
> > more attractive than efficiency achieved with specialization. Even
> > so, there might be some scope for specialization (without eye-
> > watering pricing). (Economics seems to hinder such. It is more
> > profitable to sell specialized products to those who *need* the
> > features than to sell modestly more expensive fuller-featured base
> > products even when the system would be better off having broad
> > availability of such features.)
> >
> > Anyway, it seems that it is not so much a _technical_ question as
> > an economics question: market demand et al. Since this is well
> > outside of my usual reading, I appreciate the information (even if
> > it is disappointing to the desire to approach the best technically
> > possible solution — the information also points to other features
> > [like deep packet inspection] having more value generally, which
> > reduces my ignorance).
> You're right in spirit - optimizing for datacenter (and now, AI - which looks a lot like classic HPC) has merit and a bunch of orgs have been doing that in-house for a while. It feels a lot like the late 90's where there were a bunch of boutique HPC interconnects and eventually it drove a few standardization efforts which ultimately led to InfiniBand. Now we appear to have the Ultra Ethernet Consortium. Here are a couple refs:
>
> https://www.nextplatform.com/2023/07/20/ethernet-consortium-shoots-for-1-million-node-clusters-that-beat-infiniband/
> (the links below are also all in-line in the above article)
> https://ultraethernet.org/wp-content/uploads/sites/20/2023/07/23.07.12-UEC-1.0-Overview-FINAL-WITH-LOGO.pdf
> https://www.nextplatform.com/2023/04/26/broadcom-takes-on-infiniband-with-jericho3-ai-switch-chips/
> https://www.nextplatform.com/2023/06/22/cisco-guns-for-infiniband-with-silicon-one-g200/
> https://www.nextplatform.com/2023/02/16/luminaries-argue-for-the-interconnect-we-could-have-already-had/
> https://www.nextplatform.com/2022/04/12/with-aquila-google-abandons-ethernet-to-outdo-infiniband/
>
> -JohnG

Yesterday I tried to figure out what Microsoft's Azure Boost actually does.
But that's too far away from my area of competence, so I understood nothing..
Is it in the same batch or completely different?
https://azure.microsoft.com/en-us/updates/preview-azure-boost/