According to Quadibloc <jsavard@ecn.ab.ca>:
>I have an idea, from what I've read, about what lengths are desirable
>for floating-point numbers.
>
>Integers... well, the primary integer type needs to be big enough to
>serve as an index to an array. 32 bits used to do that, and now we need
>64 bits. Although the physical memory addresses are really only 48
>bits... but then, if bigger arrays can live in virtual memory, then indexes
>into them will also be wanted.

The 801 decided that it need registers big enough to hold addresses,
which in that era were 24 bits. I don't see anything's changed there
except that addresses are bigger.

>On the System/360, of course, packed decimal integers were like
>*strings*, and could be any length. But those operations were
>memory to memory, and thus they would be very slow on today's
>computers. So the ability to do packed decimal operations in
>registers is important.

They considered and rejected decimal registers because in that era,
decimal calculations tended to be simple and I/O limited so there
weren't enough intermediate values to make registers worth it.

But on z/Series they do indeed have packed decimal vector instructions
using the 128 bit vector registers as 31 digits and a sign. There is
also decimal floating point, but vector floating point is only in
binary.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

On Friday, September 22, 2023 at 10:26:38 AM UTC-4, MitchAlsup wrote:
> On Thursday, September 21, 2023 at 9:05:14 PM UTC-5, JimBrakefield wrote:
> > On Wednesday, September 20, 2023 at 3:32:03 PM UTC-5, Thomas Koenig wrote:
> > > MitchAlsup <Mitch...@aol.com> schrieb:
> > > > On Sunday, September 17, 2023 at 3:30:19 PM UTC-5, John Levine wrote:
> > > >> According to Thomas Koenig <tko...@netcologne.de>:
> > > >> >> That's not a power-of-two length, so how do I keep using these numbers both
> > > >> >> efficient and simple?
> > > >> >
> > > >> >Make the architecture byte-addressable, with another width for the
> > > >> >bytes; possible choices are 6 and 9.
> > > >> I'm pretty sure the world has spoken and we are going to use 8-bit
> > > >> bytes forever. I liked the PDP-8 and PDP-10 but they are, you know, dead.
> > > ><
> > > > In addition, the world has spoken and little endian also won.
> > > ><
> > > >> >Then make your architecture capable of misaligned loads and stores
> > > >> >and an extra floating point format, maybe 45 bits, with 9 bits
> > > >> >exponent and 36 bits of significand.
> > > ><
> > > >> If you're worried about performance, use your 45 bit format and store
> > > >> it in a 64 bit word.
> > > ><
> > > > In 1985 one could get a descent 32-bit pipelined RISC architecture in 1cm^2
> > > > Today this design in < 0.1mm^2 or you can make a GBOoO version < 2mm^2.
> > > ><
> > > > And you really need 5mm^2 to get enough pins on the part to feed what you
> > > > can put inside; 7mm^2 makes even more sense on pins versus perf.
> > > ><
> > > > So, why are you catering to ANY bit counts less than 64 ??
> > > > Intel has version with 512-bit data paths, GPUs generally use 1024-bits in
> > > > and 1024 bits out per cycle continuously per shader core.
> > > ><
> > > > It is no longer 1990, adjust your thinking to the modern realities or our time !
> > >
> > > There could be a justification for an intermediate floating point
> > > design - memory bandwidth (and ALU width).
> > >
> > > If you look at linear algebra solvers, these are usually limited
> > > by memory bandwidth. A 512-bit cache line size accomodates
> > > 8 64-bit numbers, 10 48-bit numbers, 12 40-bit numbers, 14
> > > 36-bit numbers or 16 32-bit numbers.
> > >
> > > For problems where 32 bits are not enough, but a few more bits
> > > might suffice, having additional intermediate floating point sizes
> > > could offer significant speedup.
> > Ugh The business case for non-power-of-two floats:
> > The core count (or lane count) increases for shorter floats
> > 25% increase for 48-bit floats, 60% for 40-bit floats and 75% for 36-bit floats versus 64-bit floats.
> > Ignoring super-linear transistor counts and logic delay, this directly translates into performance advantage.
> <
> One builds FP calculation resources as big as longest container needed at full throughput.
> In a 64-bit machine, this is one with a 11-bit exponent and a 52-bit fraction.
> On such a machine, the latency is set by the calculations on this sized number.
> AND
> Smaller width numbers do not save any cycles.
> <
> So, the only advantage one has with 48-bit, ... numbers is memory footprint.
> There is NO (nada, zero, zilch) advantage in calculation latency.
> <
Does that include complicated calculations too? What about trig functions, square root, or other iterative functions?
As I have implemented reciprocal square root in micro-code it takes longer for greater precision. Makes me think
there is some benefit to supporting varying precisions.

> > As L1, L2 and L3 data caches are on chip, they can be specialized for the float size.
> > Data transfers between DRAM and the processor chip become more complicated, but as DRAM is much slower,
> > the effect is less noticeable.
> > The instructions for these different floating point units can remain 8-bit byte sized, e.g. employ a Harvard architecture.
> > (a given chip would normally support a single float size or a half or fourth thereof)

On 9/19/2023 4:32 AM, Terje Mathisen wrote:
> robf...@gmail.com wrote:
>> What kind of precision is needed for space-time co-ordinates? I tried
>> looking this
>> up and get the impression that 64-bit floats may not be enough, but
>> 128-bit floats
>> are overkill. The reference was a space game based in a galaxy.
>>
> I would use the same trick as LiDAR, with coordinates in each chunk/file
> being 32-bit integers, and the header contains the origo offset and
> scale factors needed to convert into global coords.
>
> In a game you probably only need exact coords for items which are
> reasonably close to you.
>

In my 3D engines, one common issue came up here:
OpenGL only really works with 32-bit floats;
If your game world is "bigger than Texas", this isn't going to work...

So, typically, coordinates were often stored relative to the current
region. One would add these to the region's address to get a location in
the world.

For rendering and local interactions, the whole coordinate space would
move, typically with the region containing the camera being used as the
local origin for the coordinate space. As the camera moves to a
different region, the whole space moves such that the current region is
now the origin.

So, there may end up being several ways to represent locations:
64-bit, 3x Float21 (Binary16+5b): Region local coordinates.
Binary16 isn't quite sufficient even for this.
96/128-bit, 3x Binary32: Camera local coordinates
128-bit, 3x Binary42 (Binary32+10b): World-global coordinates.
This was used for currently active entities.

This was sufficient for a world size of roughly 1024km x 1024km x 256m
(in my BGBTech2 engine, 1).
In my 3D engines, the world was a torus, rather than a plane;
The way that the coordinate-space worked allowed a seamless wrap.

Some logic (like the terrain system) also used fixed point 3x 20.12
coordinates.

1: In BGBTech3, this was dropped to 64km x 64km x 128m, but was a
similar idea (region size dropping from 256x256x256 to 128x128x128;
which only needs around 1/8 as much RAM per loaded region).

Rendering would work across the seam, which was mostly invisible as far
as the player was concerned (where region coordinates would wrap across
the edge).

For things like entity collisions, there was a little bit of hacking
across the seam (seam-crossing entities would be handled as-if they
existed on both sides of the seam at the same time).

Can also note terrain-generation noise functions:
The BGBTech1 engine had used Perlin noise;
The BGBTech2 had use spatial hashing:
Hash the coordinates via arcane use of shift and xor.
shift-xor gave better results in this case
multiply-by-prime-and-add has obvious repeating patterns.
Use hash result as a random number between 0 and 1.
Interpolate between points on a 2D or 3D grid.
Scale as necessary to get desired results.
Add multiple "frequencies" at different "weights" to get results;
One can mix magic numbers into the hash for multiple.
IIRC, BGBTech3 had used an intermediate strategy.

Actual Perlin noise required an intermediate lookup table and a bunch of
extra math which was seemingly unnecessary if the noise is driven
directly by a 2D or 3D hash function.

Similarly, each sub-layer effectively needs its own lookup table for the
noise function, where the costs add up quickly.

Had to get creative with XOR, as while intuitively, one might think they
could do, say:
h=((((((((x*65521)+y)*65521)+z)*65521)+w)*65521)>>16)&255;
This actually sucks, generating obvious patterns that tend to repeat
along the diagonal axes.

Contrast with, for example:
h=w;
h=(h<<7)^(~(h>>17))^x;
h=(h<<7)^(~(h>>17))^y;
h=(h<<7)^(~(h>>17))^z;
h=(h<<7)^(~(h>>17))^w;
h=(h<<7)^(~(h>>17)); //add more mixing steps here as needed
h=h^(h>>16);
h=(h^(h>>8))&255;
Where, say, x/y/z are (integer) coordinates and w is a magic number
(say, a 'w' value is generated for each sub-function as a random number
generated from the world seed or similar).

And, if one wants a floating point value from 0 .. 1, they do
"v=(h/255.0);" or similar.

Not entirely sure why seemingly this sort of strategy isn't more common.

....

John Levine wrote:
> According to Quadibloc <jsavard@ecn.ab.ca>:
>> I have an idea, from what I've read, about what lengths are desirable
>> for floating-point numbers.
>>
>> Integers... well, the primary integer type needs to be big enough to
>> serve as an index to an array. 32 bits used to do that, and now we need
>> 64 bits. Although the physical memory addresses are really only 48
>> bits... but then, if bigger arrays can live in virtual memory, then indexes
>> into them will also be wanted.
>
> The 801 decided that it need registers big enough to hold addresses,
> which in that era were 24 bits. I don't see anything's changed there
> except that addresses are bigger.
>
>> On the System/360, of course, packed decimal integers were like
>> *strings*, and could be any length. But those operations were
>> memory to memory, and thus they would be very slow on today's
>> computers. So the ability to do packed decimal operations in
>> registers is important.
>
> They considered and rejected decimal registers because in that era,
> decimal calculations tended to be simple and I/O limited so there
> weren't enough intermediate values to make registers worth it.
>
> But on z/Series they do indeed have packed decimal vector instructions
> using the 128 bit vector registers as 31 digits and a sign. There is

So really nybble math?

I find it interesting that Intel included a set Ascii/Nybble
instructions on the 8086/88, this might have been the least used part
of the entire x86 instruction set?

There are some exceptions, typically related to size optimization
contests where these single and double-byte opcodes were abused because
they were shorter than the normal way to do stuff, like splitting a byte
into hex nybbles by using the initially undocumented feature of using 16
to do div/mod 16 instead of mask and shift.

Even on the original 8086 you could pack 4 nybbles into a register and
operate on them in parallel, and as soon as we got to the 386, packing 8
nybbles into a 32-bit reg was so much faster than any nybble-based loop
was just stupid.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

robf...@gmail.com wrote:
> On Friday, September 22, 2023 at 10:26:38 AM UTC-4, MitchAlsup wrote:
>> One builds FP calculation resources as big as longest container needed at full throughput.
>> In a 64-bit machine, this is one with a 11-bit exponent and a 52-bit fraction.
>> On such a machine, the latency is set by the calculations on this sized number.
>> AND
>> Smaller width numbers do not save any cycles.
>> <
>> So, the only advantage one has with 48-bit, ... numbers is memory footprint.
>> There is NO (nada, zero, zilch) advantage in calculation latency.
>> <
> Does that include complicated calculations too? What about trig
> functions, square root, or other iterative functions? As I have
> implemented reciprocal square root in micro-code it takes longer for
> greater precision. Makes me think there is some benefit to supporting
> varying precisions.
This is easy to verify: Lookup the latency for both 32 and 64-bit
versions of the function you are interested in!

If they differ by less then 25%, then anything intermediate really
doesn't make sense for a HW op.

In software you can more easily play tricks like the infamous InvSqrt()
function of Quake III fame, where just 10 bits was sufficient to make
lighting calculations look OK. Today you can do the same, using the
approximate reciprocal square root vector op, and simply skip all the
normal NR stages to follow.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

John Levine <johnl@taugh.com> writes:
>But on z/Series they do indeed have packed decimal vector instructions
>using the 128 bit vector registers as 31 digits and a sign. There is
>also decimal floating point

Decimal floating-point hardware seems to be a marketing feature to me.
Is there any real-world application that uses that? How much CPU-time
does it use? Intel has neglected its DFP implementation (in
software), which indicates that any such applications, if they exist,
are insignificant.

As for packed BCD stuff, I expect that the Cobol stuff benefits from
that. But given that the byte-by-byte instructions were good enough
in earlier times with slower CPUs, do we need faster instructions for
that now. Have the amounts of data processed by these programs
increased so much?

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Friday, September 22, 2023 at 8:50:53 PM UTC-5, robf...@gmail.com wrote:
> On Friday, September 22, 2023 at 10:26:38 AM UTC-4, MitchAlsup wrote:
> > On Thursday, September 21, 2023 at 9:05:14 PM UTC-5, JimBrakefield wrote:
> > > On Wednesday, September 20, 2023 at 3:32:03 PM UTC-5, Thomas Koenig wrote:
> > > > MitchAlsup <Mitch...@aol.com> schrieb:
> > > > > On Sunday, September 17, 2023 at 3:30:19 PM UTC-5, John Levine wrote:
> > > > >> According to Thomas Koenig <tko...@netcologne.de>:
> > > > >> >> That's not a power-of-two length, so how do I keep using these numbers both
> > > > >> >> efficient and simple?
> > > > >> >
> > > > >> >Make the architecture byte-addressable, with another width for the
> > > > >> >bytes; possible choices are 6 and 9.
> > > > >> I'm pretty sure the world has spoken and we are going to use 8-bit
> > > > >> bytes forever. I liked the PDP-8 and PDP-10 but they are, you know, dead.
> > > > ><
> > > > > In addition, the world has spoken and little endian also won.
> > > > ><
> > > > >> >Then make your architecture capable of misaligned loads and stores
> > > > >> >and an extra floating point format, maybe 45 bits, with 9 bits
> > > > >> >exponent and 36 bits of significand.
> > > > ><
> > > > >> If you're worried about performance, use your 45 bit format and store
> > > > >> it in a 64 bit word.
> > > > ><
> > > > > In 1985 one could get a descent 32-bit pipelined RISC architecture in 1cm^2
> > > > > Today this design in < 0.1mm^2 or you can make a GBOoO version < 2mm^2.
> > > > ><
> > > > > And you really need 5mm^2 to get enough pins on the part to feed what you
> > > > > can put inside; 7mm^2 makes even more sense on pins versus perf.
> > > > ><
> > > > > So, why are you catering to ANY bit counts less than 64 ??
> > > > > Intel has version with 512-bit data paths, GPUs generally use 1024-bits in
> > > > > and 1024 bits out per cycle continuously per shader core.
> > > > ><
> > > > > It is no longer 1990, adjust your thinking to the modern realities or our time !
> > > >
> > > > There could be a justification for an intermediate floating point
> > > > design - memory bandwidth (and ALU width).
> > > >
> > > > If you look at linear algebra solvers, these are usually limited
> > > > by memory bandwidth. A 512-bit cache line size accomodates
> > > > 8 64-bit numbers, 10 48-bit numbers, 12 40-bit numbers, 14
> > > > 36-bit numbers or 16 32-bit numbers.
> > > >
> > > > For problems where 32 bits are not enough, but a few more bits
> > > > might suffice, having additional intermediate floating point sizes
> > > > could offer significant speedup.
> > > Ugh The business case for non-power-of-two floats:
> > > The core count (or lane count) increases for shorter floats
> > > 25% increase for 48-bit floats, 60% for 40-bit floats and 75% for 36-bit floats versus 64-bit floats.
> > > Ignoring super-linear transistor counts and logic delay, this directly translates into performance advantage.
> > <
> > One builds FP calculation resources as big as longest container needed at full throughput.
> > In a 64-bit machine, this is one with a 11-bit exponent and a 52-bit fraction.
> > On such a machine, the latency is set by the calculations on this sized number.
> > AND
> > Smaller width numbers do not save any cycles.
> > <
> > So, the only advantage one has with 48-bit, ... numbers is memory footprint.
> > There is NO (nada, zero, zilch) advantage in calculation latency.
> > <
> Does that include complicated calculations too? What about trig functions, square root, or other iterative functions?
<
FDIV 17 cycles Goldschmidt with 1 Newton-Raphson iteration
SQRT 22 cycles Goldschmidt with 1 Newton-Raphson iteration
Ln2 16 cycles
ln/ln10 19 cycles
Exp2 17 cycles
Exp/exp10 20 cycles
Sin/Cos 21 cycles including argument reduction
Tan 21 or 38 including argument reduction
Atan 21 or 38 cycles
Pow 36 cycles
All double precision all faithfully rounded all Chebyshev polynomials except as noted.
<
> As I have implemented reciprocal square root in micro-code it takes longer for greater precision. Makes me think
> there is some benefit to supporting varying precisions.
<
SQRT and RSQRT can be done such that precision doubles each iteration.
as such, if you can do 32-bits in K cycles you can do 64-bits in K+3 cycles,
there is not much room for making 48-bits faster. Oh, BTW, K = 19 and loop
iteration is 3 cycles.
<
> > > As L1, L2 and L3 data caches are on chip, they can be specialized for the float size.
> > > Data transfers between DRAM and the processor chip become more complicated, but as DRAM is much slower,
> > > the effect is less noticeable.
> > > The instructions for these different floating point units can remain 8-bit byte sized, e.g. employ a Harvard architecture.
> > > (a given chip would normally support a single float size or a half or fourth thereof)

Anton Ertl wrote:
> John Levine <johnl@taugh.com> writes:
>> But on z/Series they do indeed have packed decimal vector instructions
>> using the 128 bit vector registers as 31 digits and a sign. There is
>> also decimal floating point
>
> Decimal floating-point hardware seems to be a marketing feature to me.
> Is there any real-world application that uses that? How much CPU-time
> does it use? Intel has neglected its DFP implementation (in
> software), which indicates that any such applications, if they exist,
> are insignificant.
>
> As for packed BCD stuff, I expect that the Cobol stuff benefits from
> that. But given that the byte-by-byte instructions were good enough
> in earlier times with slower CPUs, do we need faster instructions for
> that now. Have the amounts of data processed by these programs
> increased so much?

Both Ascii and Nybble math can be done quite easily with SIMD vector
ops, giving an order of magntude better throughput than using the
single-digit instructions.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On Saturday, September 23, 2023 at 12:29:49 PM UTC-4, MitchAlsup wrote:
> On Friday, September 22, 2023 at 8:50:53 PM UTC-5, robf...@gmail.com wrote:
> > On Friday, September 22, 2023 at 10:26:38 AM UTC-4, MitchAlsup wrote:
> > > On Thursday, September 21, 2023 at 9:05:14 PM UTC-5, JimBrakefield wrote:
> > > > On Wednesday, September 20, 2023 at 3:32:03 PM UTC-5, Thomas Koenig wrote:
> > > > > MitchAlsup <Mitch...@aol.com> schrieb:
> > > > > > On Sunday, September 17, 2023 at 3:30:19 PM UTC-5, John Levine wrote:
> > > > > >> According to Thomas Koenig <tko...@netcologne.de>:
> > > > > >> >> That's not a power-of-two length, so how do I keep using these numbers both
> > > > > >> >> efficient and simple?
> > > > > >> >
> > > > > >> >Make the architecture byte-addressable, with another width for the
> > > > > >> >bytes; possible choices are 6 and 9.
> > > > > >> I'm pretty sure the world has spoken and we are going to use 8-bit
> > > > > >> bytes forever. I liked the PDP-8 and PDP-10 but they are, you know, dead.
> > > > > ><
> > > > > > In addition, the world has spoken and little endian also won.
> > > > > ><
> > > > > >> >Then make your architecture capable of misaligned loads and stores
> > > > > >> >and an extra floating point format, maybe 45 bits, with 9 bits
> > > > > >> >exponent and 36 bits of significand.
> > > > > ><
> > > > > >> If you're worried about performance, use your 45 bit format and store
> > > > > >> it in a 64 bit word.
> > > > > ><
> > > > > > In 1985 one could get a descent 32-bit pipelined RISC architecture in 1cm^2
> > > > > > Today this design in < 0.1mm^2 or you can make a GBOoO version < 2mm^2.
> > > > > ><
> > > > > > And you really need 5mm^2 to get enough pins on the part to feed what you
> > > > > > can put inside; 7mm^2 makes even more sense on pins versus perf..
> > > > > ><
> > > > > > So, why are you catering to ANY bit counts less than 64 ??
> > > > > > Intel has version with 512-bit data paths, GPUs generally use 1024-bits in
> > > > > > and 1024 bits out per cycle continuously per shader core.
> > > > > ><
> > > > > > It is no longer 1990, adjust your thinking to the modern realities or our time !
> > > > >
> > > > > There could be a justification for an intermediate floating point
> > > > > design - memory bandwidth (and ALU width).
> > > > >
> > > > > If you look at linear algebra solvers, these are usually limited
> > > > > by memory bandwidth. A 512-bit cache line size accomodates
> > > > > 8 64-bit numbers, 10 48-bit numbers, 12 40-bit numbers, 14
> > > > > 36-bit numbers or 16 32-bit numbers.
> > > > >
> > > > > For problems where 32 bits are not enough, but a few more bits
> > > > > might suffice, having additional intermediate floating point sizes
> > > > > could offer significant speedup.
> > > > Ugh The business case for non-power-of-two floats:
> > > > The core count (or lane count) increases for shorter floats
> > > > 25% increase for 48-bit floats, 60% for 40-bit floats and 75% for 36-bit floats versus 64-bit floats.
> > > > Ignoring super-linear transistor counts and logic delay, this directly translates into performance advantage.
> > > <
> > > One builds FP calculation resources as big as longest container needed at full throughput.
> > > In a 64-bit machine, this is one with a 11-bit exponent and a 52-bit fraction.
> > > On such a machine, the latency is set by the calculations on this sized number.
> > > AND
> > > Smaller width numbers do not save any cycles.
> > > <
> > > So, the only advantage one has with 48-bit, ... numbers is memory footprint.
> > > There is NO (nada, zero, zilch) advantage in calculation latency.
> > > <
> > Does that include complicated calculations too? What about trig functions, square root, or other iterative functions?
> <
> FDIV 17 cycles Goldschmidt with 1 Newton-Raphson iteration
> SQRT 22 cycles Goldschmidt with 1 Newton-Raphson iteration
> Ln2 16 cycles
> ln/ln10 19 cycles
> Exp2 17 cycles
> Exp/exp10 20 cycles
> Sin/Cos 21 cycles including argument reduction
> Tan 21 or 38 including argument reduction
> Atan 21 or 38 cycles
> Pow 36 cycles
> All double precision all faithfully rounded all Chebyshev polynomials except as noted.
> <
> > As I have implemented reciprocal square root in micro-code it takes longer for greater precision. Makes me think
> > there is some benefit to supporting varying precisions.
> <
> SQRT and RSQRT can be done such that precision doubles each iteration.
> as such, if you can do 32-bits in K cycles you can do 64-bits in K+3 cycles,
> there is not much room for making 48-bits faster. Oh, BTW, K = 19 and loop
> iteration is 3 cycles.
> <

I have many more than 3 cycles for an iteration. An FMA takes 8 cycles and there are multiple per iteration.
However, I should have looked at my micro-code more closely. There is indeed no difference in between
calculating out to 64 bit or 48 bits because of the number of bits reached in each iteration.

To get 48 bits an iteration faster would require a much more accurate initial approximation which probably
is not practical.
// RSQRT initial approximation 0
// y = y*(1.5f – xhalf *y*y); // first NR iteration 9.16 bits accurate
// y = y*(1.5f – xhalf *y*y); // second NR iteration 17.69 bits accurate
// y = y*(1.5f – xhalf *y*y); // third NR iteration 35 bits accurate
// y = y*(1.5f – xhalf *y*y); // fourth NR iteration 70 bits accurate

In my case it still looks like there may be value in supporting separate 8/16/32/64/128 bit ops. I guess it
depends on how fast an iteration is. Limited parallel hardware here.

Reciprocal estimate bits/clocks:
16 – 22 clocks
32 – 38 clocks
64 – 54 clocks
16 clocks per iteration plus some overhead.

Scratching my head over how to make the ISA less dependent on the implementation.

> > > > As L1, L2 and L3 data caches are on chip, they can be specialized for the float size.
> > > > Data transfers between DRAM and the processor chip become more complicated, but as DRAM is much slower,
> > > > the effect is less noticeable.
> > > > The instructions for these different floating point units can remain 8-bit byte sized, e.g. employ a Harvard architecture.
> > > > (a given chip would normally support a single float size or a half or fourth thereof)

On Saturday, September 23, 2023 at 2:12:30 PM UTC-5, robf...@gmail.com wrote:
> On Saturday, September 23, 2023 at 12:29:49 PM UTC-4, MitchAlsup wrote:
> > On Friday, September 22, 2023 at 8:50:53 PM UTC-5, robf...@gmail.com wrote:
> > > On Friday, September 22, 2023 at 10:26:38 AM UTC-4, MitchAlsup wrote:
> > > > On Thursday, September 21, 2023 at 9:05:14 PM UTC-5, JimBrakefield wrote:
> > > > > On Wednesday, September 20, 2023 at 3:32:03 PM UTC-5, Thomas Koenig wrote:
> > > > > > MitchAlsup <Mitch...@aol.com> schrieb:
> > > > > > > On Sunday, September 17, 2023 at 3:30:19 PM UTC-5, John Levine wrote:
> > > > > > >> According to Thomas Koenig <tko...@netcologne.de>:
> > > > > > >> >> That's not a power-of-two length, so how do I keep using these numbers both
> > > > > > >> >> efficient and simple?
> > > > > > >> >
> > > > > > >> >Make the architecture byte-addressable, with another width for the
> > > > > > >> >bytes; possible choices are 6 and 9.
> > > > > > >> I'm pretty sure the world has spoken and we are going to use 8-bit
> > > > > > >> bytes forever. I liked the PDP-8 and PDP-10 but they are, you know, dead.
> > > > > > ><
> > > > > > > In addition, the world has spoken and little endian also won.
> > > > > > ><
> > > > > > >> >Then make your architecture capable of misaligned loads and stores
> > > > > > >> >and an extra floating point format, maybe 45 bits, with 9 bits
> > > > > > >> >exponent and 36 bits of significand.
> > > > > > ><
> > > > > > >> If you're worried about performance, use your 45 bit format and store
> > > > > > >> it in a 64 bit word.
> > > > > > ><
> > > > > > > In 1985 one could get a descent 32-bit pipelined RISC architecture in 1cm^2
> > > > > > > Today this design in < 0.1mm^2 or you can make a GBOoO version < 2mm^2.
> > > > > > ><
> > > > > > > And you really need 5mm^2 to get enough pins on the part to feed what you
> > > > > > > can put inside; 7mm^2 makes even more sense on pins versus perf.
> > > > > > ><
> > > > > > > So, why are you catering to ANY bit counts less than 64 ??
> > > > > > > Intel has version with 512-bit data paths, GPUs generally use 1024-bits in
> > > > > > > and 1024 bits out per cycle continuously per shader core.
> > > > > > ><
> > > > > > > It is no longer 1990, adjust your thinking to the modern realities or our time !
> > > > > >
> > > > > > There could be a justification for an intermediate floating point
> > > > > > design - memory bandwidth (and ALU width).
> > > > > >
> > > > > > If you look at linear algebra solvers, these are usually limited
> > > > > > by memory bandwidth. A 512-bit cache line size accomodates
> > > > > > 8 64-bit numbers, 10 48-bit numbers, 12 40-bit numbers, 14
> > > > > > 36-bit numbers or 16 32-bit numbers.
> > > > > >
> > > > > > For problems where 32 bits are not enough, but a few more bits
> > > > > > might suffice, having additional intermediate floating point sizes
> > > > > > could offer significant speedup.
> > > > > Ugh The business case for non-power-of-two floats:
> > > > > The core count (or lane count) increases for shorter floats
> > > > > 25% increase for 48-bit floats, 60% for 40-bit floats and 75% for 36-bit floats versus 64-bit floats.
> > > > > Ignoring super-linear transistor counts and logic delay, this directly translates into performance advantage.
> > > > <
> > > > One builds FP calculation resources as big as longest container needed at full throughput.
> > > > In a 64-bit machine, this is one with a 11-bit exponent and a 52-bit fraction.
> > > > On such a machine, the latency is set by the calculations on this sized number.
> > > > AND
> > > > Smaller width numbers do not save any cycles.
> > > > <
> > > > So, the only advantage one has with 48-bit, ... numbers is memory footprint.
> > > > There is NO (nada, zero, zilch) advantage in calculation latency.
> > > > <
> > > Does that include complicated calculations too? What about trig functions, square root, or other iterative functions?
> > <
> > FDIV 17 cycles Goldschmidt with 1 Newton-Raphson iteration
> > SQRT 22 cycles Goldschmidt with 1 Newton-Raphson iteration
> > Ln2 16 cycles
> > ln/ln10 19 cycles
> > Exp2 17 cycles
> > Exp/exp10 20 cycles
> > Sin/Cos 21 cycles including argument reduction
> > Tan 21 or 38 including argument reduction
> > Atan 21 or 38 cycles
> > Pow 36 cycles
> > All double precision all faithfully rounded all Chebyshev polynomials except as noted.
> > <
> > > As I have implemented reciprocal square root in micro-code it takes longer for greater precision. Makes me think
> > > there is some benefit to supporting varying precisions.
> > <
> > SQRT and RSQRT can be done such that precision doubles each iteration.
> > as such, if you can do 32-bits in K cycles you can do 64-bits in K+3 cycles,
> > there is not much room for making 48-bits faster. Oh, BTW, K = 19 and loop
> > iteration is 3 cycles.
> > <
> I have many more than 3 cycles for an iteration. An FMA takes 8 cycles and there are multiple per iteration.
<
You should be able to start a new FMAC every cycle; once you can do this, the
iterations are just dropping stuff into the pipeline.
<
> However, I should have looked at my micro-code more closely. There is indeed no difference in between
> calculating out to 64 bit or 48 bits because of the number of bits reached in each iteration.
>
> To get 48 bits an iteration faster would require a much more accurate initial approximation which probably
> is not practical.
> // RSQRT initial approximation 0
> // y = y*(1.5f – xhalf *y*y); // first NR iteration 9.16 bits accurate
> // y = y*(1.5f – xhalf *y*y); // second NR iteration 17.69 bits accurate
> // y = y*(1.5f – xhalf *y*y); // third NR iteration 35 bits accurate
> // y = y*(1.5f – xhalf *y*y); // fourth NR iteration 70 bits accurate
>
3 dependent multiplies per iteration--Goldschmidt changes this to 2 dependent multiplies per iteration.
{{Then inside the FU:: the multiplies are treated as fixed point so the iteration latency is the height
of the multiplier tree not the latency of the FU itself (from 360/91 FDIV)}}
<
<
> In my case it still looks like there may be value in supporting separate 8/16/32/64/128 bit ops. I guess it
> depends on how fast an iteration is. Limited parallel hardware here.
>
> Reciprocal estimate bits/clocks:
> 16 – 22 clocks
> 32 – 38 clocks
> 64 – 54 clocks
> 16 clocks per iteration plus some overhead.
>
> Scratching my head over how to make the ISA less dependent on the implementation.
> > > > > As L1, L2 and L3 data caches are on chip, they can be specialized for the float size.
> > > > > Data transfers between DRAM and the processor chip become more complicated, but as DRAM is much slower,
> > > > > the effect is less noticeable.
> > > > > The instructions for these different floating point units can remain 8-bit byte sized, e.g. employ a Harvard architecture.
> > > > > (a given chip would normally support a single float size or a half or fourth thereof)

According to Terje Mathisen <terje.mathisen@tmsw.no>:
>> But on z/Series they do indeed have packed decimal vector instructions
>> using the 128 bit vector registers as 31 digits and a sign. There is
>
>So really nybble math?

No, packed decimal. They're fixed length 31 digit signeddecimal
numbers (the sign is in the 32nd nibble.) Most instructions can
specify the maximum number of significant digits allowed in the
result, and can force positive or negative signs on the result.
There's a multiply and then shift result right, shift dividend left
and divide, and shift and round, again with significant digit limits.
If the result has too many digits, it's either reported with a
condition code or an overflow interrupt. See chapter 25 of the zSeries
POO.

You could do all this with 128 bit integers but the significance
checks and decimal rounding would take a lot of extra instructions.

These are for financial stuff like bond pricing that have to match
formulae invented in the era of mechanical desk calculators. Forty
years ago, I managed to do bond prices and yields using 8087
arithmetic but it would have been a lot easier in decimal since then I
could have directly implemented the spec. I wouldn't have wanted to
try to code up calculations that chained them together.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

According to Anton Ertl <anton@mips.complang.tuwien.ac.at>:
>John Levine <johnl@taugh.com> writes:
>>But on z/Series they do indeed have packed decimal vector instructions
>>using the 128 bit vector registers as 31 digits and a sign. There is
>>also decimal floating point
>
>Decimal floating-point hardware seems to be a marketing feature to me.
>Is there any real-world application that uses that?

My impression is that there aren't a lot of people who want it, but
for the ones who do, they want it very much to do their financial
calculations.

Like I said a few minutes ago, there are decimal financial formulae
developed long ago for bond pricing and related time and interest
calculations. Since there are literaly trillions of dollars of
financial instruments priced and sold this way, I can see that for
some customers it would be worth a lot to have decimal arithmetic that
could implement this directly with well defined decimal rounding at
each step to make it easier to write correct code.
--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

On Saturday, September 23, 2023 at 11:59:35 PM UTC+3, John Levine wrote:
> According to Anton Ertl <an...@mips.complang.tuwien.ac.at>:
> >John Levine <jo...@taugh.com> writes:
> >>But on z/Series they do indeed have packed decimal vector instructions
> >>using the 128 bit vector registers as 31 digits and a sign. There is
> >>also decimal floating point
> >
> >Decimal floating-point hardware seems to be a marketing feature to me.
> >Is there any real-world application that uses that?
> My impression is that there aren't a lot of people who want it, but
> for the ones who do, they want it very much to do their financial
> calculations.
>
> Like I said a few minutes ago, there are decimal financial formulae
> developed long ago for bond pricing and related time and interest
> calculations. Since there are literaly trillions of dollars of
> financial instruments priced and sold this way, I can see that for
> some customers it would be worth a lot to have decimal arithmetic that
> could implement this directly with well defined decimal rounding at
> each step to make it easier to write correct code.
> --
> Regards,
> John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
> Please consider the environment before reading this e-mail. https://jl.ly

All easily done in sw with even more defined rounding semantics.
And considering how fast modern z CPUs are, they sure wouldn't break
a sweet doing so.

According to Michael S <already5chosen@yahoo.com>:
>> Like I said a few minutes ago, there are decimal financial formulae
>> developed long ago for bond pricing and related time and interest
>> calculations. Since there are literaly trillions of dollars of
>> financial instruments priced and sold this way, I can see that for
>> some customers it would be worth a lot to have decimal arithmetic that
>> could implement this directly with well defined decimal rounding at
>> each step to make it easier to write correct code.

>All easily done in sw with even more defined rounding semantics.
>And considering how fast modern z CPUs are, they sure wouldn't break
>a sweat doing so.

In the sense that all modern computers are Turing equivalent, sure.
You can do this sort of stuff in python.

But given the cost of getting stuff wrong, it's worth something to
them to know that the decimal stuff is locked down at the hardware
level.

Z implements a lot of the complex instructions in microcode which uses
the hardwired subset of the instruction set, which they call millicode
but apparently DFP is hardware.

This paper describes the hardware and software support for DFP:

https://speleotrove.com/mfc/files/schwarz2009-decimalFP-on-z10.pdf

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

robf...@gmail.com wrote:
> I have many more than 3 cycles for an iteration. An FMA takes 8 cycles and there are multiple per iteration.
> However, I should have looked at my micro-code more closely. There is indeed no difference in between
> calculating out to 64 bit or 48 bits because of the number of bits reached in each iteration.
>
> To get 48 bits an iteration faster would require a much more accurate initial approximation which probably
> is not practical.
> // RSQRT initial approximation 0
> // y = y*(1.5f – xhalf *y*y); // first NR iteration 9.16 bits accurate

What if I told you that you can get up to 1.7 more bits after the first
NR iteration? You use a slightly different magic number in the bit hack,
then you also modify the two constants in that first NR step: I.e. not
exactly 1.5 and 0.5 but modified to give a cheby style error
distribution over the (0.5 to 2.0) input range.

The result is about 10.8 bits!

> // y = y*(1.5f – xhalf *y*y); // second NR iteration 17.69 bits accurate
~19 bits
> // y = y*(1.5f – xhalf *y*y); // third NR iteration 35 bits accurate
~38 bits
> // y = y*(1.5f – xhalf *y*y); // fourth NR iteration 70 bits accurate
~75 bits

BTW, I independently came up with the idea to modify multiple constants
and got more than a a bit extra, then somebody tipped me off about a guy
from Poland who had done the full optimization of all three at the same
time and gotten half a bit more than me. :-)

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

John Levine wrote:
> According to Terje Mathisen <terje.mathisen@tmsw.no>:
>>> But on z/Series they do indeed have packed decimal vector instructions
>>> using the 128 bit vector registers as 31 digits and a sign. There is
>>
>> So really nybble math?
>
> No, packed decimal. They're fixed length 31 digit signeddecimal
> numbers (the sign is in the 32nd nibble.) Most instructions can
> specify the maximum number of significant digits allowed in the
> result, and can force positive or negative signs on the result.
> There's a multiply and then shift result right, shift dividend left
> and divide, and shift and round, again with significant digit limits.
> If the result has too many digits, it's either reported with a
> condition code or an overflow interrupt. See chapter 25 of the zSeries
> POO.
>
> You could do all this with 128 bit integers but the significance
> checks and decimal rounding would take a lot of extra instructions.
>
> These are for financial stuff like bond pricing that have to match
> formulae invented in the era of mechanical desk calculators. Forty
> years ago, I managed to do bond prices and yields using 8087
> arithmetic but it would have been a lot easier in decimal since then I
> could have directly implemented the spec. I wouldn't have wanted to
> try to code up calculations that chained them together.

I did the exact same thing around 1983, using 80-bit FP to implement
Norwegian rounding rules for my father-in-law's gift card business.

I had to extract the fees first from each individual transaction, then
add them all together to get the sum: Doing the same first summing and
then calculating the fee from the total would have given very nearly the
same results, but not guaranteed for all situations.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

John Levine <johnl@taugh.com> schrieb:

> Z implements a lot of the complex instructions in microcode which uses
> the hardwired subset of the instruction set, which they call millicode
> but apparently DFP is hardware.
>
> This paper describes the hardware and software support for DFP:
>
> https://speleotrove.com/mfc/files/schwarz2009-decimalFP-on-z10.pdf

Very interesting link, thanks!

A few interesting snippets: They give the cycle time of the z10
as 15 FO4, which they say is much faster than prior generations.
Not sure how that compares to current designs, but it seems
fast to me.

They also write

"[...] the execution pipeline [for the IBM z] for one instruction
includes both a memory access and an execution stage, whereas
RISC computers require multiple instructions to accomplish the
same task. Nevertheless, resolving memory interlock dependencies
is a concern. Since the operands are in memory, using the result
of a prior operation creates an interlock in memory. If the
operations are not spaced apart in time, the load/store unit (LSU)
or IDU must compare the full addresses to determine the interlock
and somehow bypass the operands. The new decimal floating-point
architecture makes dependencies easier and faster to handle because
the interlocks are simply in registers."

Fixed-point BCD operations are also (to me) surprisingly slow:

"For addition and subtraction, the execution latency is seven
cycles for operands of 8 bytes or less and nine cycles for
operands with greater length. This includes all special cases,
including overflow."

Seven cycles (105 FO4 gate delays) seems like a lot for adding,
but I guess that just speaks to the complexity of BCD arithmetic.

On Sunday, September 24, 2023 at 10:56:08 AM UTC+3, Thomas Koenig wrote:
> John Levine <jo...@taugh.com> schrieb:
> > Z implements a lot of the complex instructions in microcode which uses
> > the hardwired subset of the instruction set, which they call millicode
> > but apparently DFP is hardware.
> >
> > This paper describes the hardware and software support for DFP:
> >
> > https://speleotrove.com/mfc/files/schwarz2009-decimalFP-on-z10.pdf
> Very interesting link, thanks!
>
> A few interesting snippets: They give the cycle time of the z10
> as 15 FO4, which they say is much faster than prior generations.
> Not sure how that compares to current designs, but it seems
> fast to me.
>
> They also write
>
> "[...] the execution pipeline [for the IBM z] for one instruction
> includes both a memory access and an execution stage, whereas
> RISC computers require multiple instructions to accomplish the
> same task. Nevertheless, resolving memory interlock dependencies
> is a concern. Since the operands are in memory, using the result
> of a prior operation creates an interlock in memory. If the
> operations are not spaced apart in time, the load/store unit (LSU)
> or IDU must compare the full addresses to determine the interlock
> and somehow bypass the operands. The new decimal floating-point
> architecture makes dependencies easier and faster to handle because
> the interlocks are simply in registers."
>
> Fixed-point BCD operations are also (to me) surprisingly slow:
>
> "For addition and subtraction, the execution latency is seven
> cycles for operands of 8 bytes or less and nine cycles for
> operands with greater length. This includes all special cases,
> including overflow."
>
> Seven cycles (105 FO4 gate delays) seems like a lot for adding,
> but I guess that just speaks to the complexity of BCD arithmetic.

IIRC, z10 was IBM's last "native CISC" design in zArch series.
Starting from z196 they crack load-op into 2 or more uOus, just
like majority of x86 cores does.
It's hard to be sure, because terminology use by IBM is so unique.

On Sunday, September 24, 2023 at 2:52:35 AM UTC-4, Terje Mathisen wrote:
> robf...@gmail.com wrote:
> > I have many more than 3 cycles for an iteration. An FMA takes 8 cycles and there are multiple per iteration.
> > However, I should have looked at my micro-code more closely. There is indeed no difference in between
> > calculating out to 64 bit or 48 bits because of the number of bits reached in each iteration.
> >
> > To get 48 bits an iteration faster would require a much more accurate initial approximation which probably
> > is not practical.
> > // RSQRT initial approximation 0
> > // y = y*(1.5f – xhalf *y*y); // first NR iteration 9.16 bits accurate
> What if I told you that you can get up to 1.7 more bits after the first
> NR iteration? You use a slightly different magic number in the bit hack,
> then you also modify the two constants in that first NR step: I.e. not
> exactly 1.5 and 0.5 but modified to give a cheby style error
> distribution over the (0.5 to 2.0) input range.
>
> The result is about 10.8 bits!
> > // y = y*(1.5f – xhalf *y*y); // second NR iteration 17.69 bits accurate
> ~19 bits
> > // y = y*(1.5f – xhalf *y*y); // third NR iteration 35 bits accurate
> ~38 bits
> > // y = y*(1.5f – xhalf *y*y); // fourth NR iteration 70 bits accurate
> ~75 bits
>
> BTW, I independently came up with the idea to modify multiple constants
> and got more than a a bit extra, then somebody tipped me off about a guy
> from Poland who had done the full optimization of all three at the same
> time and gotten half a bit more than me. :-)
> Terje
>
I believe it. It is a good hack. I found the hack researching reciprocal square
roots, but decided to go with the simpler algorithm. It cost clock cycles to
load the different constants and the extra accuracy was not needed.

> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>John Levine <johnl@taugh.com> writes:
>>But on z/Series they do indeed have packed decimal vector instructions
>>using the 128 bit vector registers as 31 digits and a sign. There is
>>also decimal floating point
>
>Decimal floating-point hardware seems to be a marketing feature to me.

Burroughs had decimal floating point (100 digit mantissa, 2 digit exponent)
in the B3500 (circa 1965). It was memory-to-memory, no registers involved.

Turned out that the COBOL customers weren't interested, so the next generation
(B4700) removed it and replaced it with a 24 digit decimal accumulator (20 digit
mantissa, 2 digit exponent, two one-digit signs).

Look for 1025475_B2500_B3500_RefMan_Oct69.pdf on bitsavers.org.

On Sunday, September 17, 2023 at 11:58:18 AM UTC-6, John Dallman wrote:
> No
> architecture can do everything efficiently.

This is true enough.

I am aiming, though, to err in the direction of doing lots and lots of things _fast_,
with the penalty that the chip has a lot of extra transistors, and thus extra die
size, as the price of doing so.

The reason I'm making this choice is that I envisage the user's situation to be:

The user is sitting in front of one computer, with one particular type of chip in it.
Running out and buying a different computer with a different chip to make one
particular program run faster is not an option.

So I'm going to try to include in my chip stuff like...

hardware support for packed decimal
hardware support for IBM System/360 hexadecimal floating point

because people do run Hercules on their computers and so on.

On the page

http://www.quadibloc.com/arch/per14.htm

I have now added, at the bottom of the page, a scheme, involving having dual-channel
memory where each channel is 192 bits wide, that permits the operating system
to allocate blocks of 384-bit wide memory, 288-bit wide memory, 240--bit wide
memory, and 256-bit wide memory. Only the 256-bit wide memory requires a division
by three during the translation from internal logical addresses to physical addresses.

So a chip making use of this scheme could allow a very wide selection of architectures
from history to be efficiently emulated.

John Savard

According to Thomas Koenig <tkoenig@netcologne.de>:
>John Levine <johnl@taugh.com> schrieb:
>
>> Z implements a lot of the complex instructions in microcode which uses
>> the hardwired subset of the instruction set, which they call millicode
>> but apparently DFP is hardware.
>>
>> This paper describes the hardware and software support for DFP:
>>
>> https://speleotrove.com/mfc/files/schwarz2009-decimalFP-on-z10.pdf
>
>Very interesting link, thanks!
>
>A few interesting snippets: They give the cycle time of the z10
>as 15 FO4, which they say is much faster than prior generations.
>Not sure how that compares to current designs, but it seems
>fast to me.

The z15 redbook says the CPUs run at 5.2GHz, same as the z14, but it's
25% faster because it has bigger faster caches.

Read all about the z15 here. CPU description start here:

https://www.redbooks.ibm.com/redbooks/pdfs/sg248851.pdf

The machine is packaged in drawers each of which contain CPUs, four
levels of cache, memory, and I/O controllers. Every CPU can see all
the memory but using memory in other drawers is much slower so they
have stuff with odd acronyms to try and keep related work physically
in one place.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

Quadibloc <jsavard@ecn.ab.ca> schrieb:

> So I'm going to try to include in my chip stuff like...

[...]

> hardware support for IBM System/360 hexadecimal floating point

What on Earth for?

Binary is better, and people have not been running serious
scientific software on IBM mainfraimes for decades - there is not
even a Fortran 90, let alone a Fortran 2008 compiler for zOS -
VS Fortran is stuck at Fortran 77.

Michael S <already5chosen@yahoo.com> writes:
> IIRC, z10 was IBM's last "native CISC" design in zArch series.
> Starting from z196 they crack load-op into 2 or more uOus, just
> like majority of x86 cores does.
> It's hard to be sure, because terminology use by IBM is so unique.

they other thing part of z10->z196 was claim that at least half of the
per processor thruoughput increase was introduction of out-of-order
execution, branch prediction, etc ... i also assumed it implied
moving to micro-ops ...

--
virtualization experience starting Jan1968, online at home since Mar1970

According to Scott Lurndal <slp53@pacbell.net>:
>>Decimal floating-point hardware seems to be a marketing feature to me.
>
>Burroughs had decimal floating point (100 digit mantissa, 2 digit exponent)
>in the B3500 (circa 1965). It was memory-to-memory, no registers involved.
>
>Turned out that the COBOL customers weren't interested, ...

I'm not surprised. I would guess the modern market for DFP is more
likely to be high speed financial stuff (what's known as picking up
pennies in front of a steam roller), not accounting.
--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly