I have been polishing my shared memory application (iSPICE) a bit more.
The benchmark I previously showed compared running a circuit simulation
with a variable number of communicating CPUs. Only a minimum amount of data
is shared (a page with published parameters and achieved results, plus the
ready! flags). With this setup I got about a factor of 3 improvement for
8 CPUs. I hoped to improve this factor a bit with better hardware and maybe
some software tweaking.

What I didn't try until today was checking how fast the circuit simulation
ran on a single CPU, *not* using the shared memory framework. And indeed,
that is a problem, in that without shared memory the runtime is *3 times
less* than with shared memory. In other words, there is no net gain in
having 8 mem-shared cpu's. As a additional check I started the circuit run
in 3 separate windows. They all achieved the same speed as the single run
non-shared version, proving that the hardware (cpu/memory/disk) is amply
sufficient to provide an 8 times speed-up.

I will now start working on Anton's suggesting of a shared file. Or maybe
I should try this on Linux first, maybe shared memory works better there.

-marcel

Perhaps this is the reason why:

Windows shared memory is not the same as Linux only some things are similar.

The Unix mmap() API is practically equivalent to the CreateFileMapping/
MapViewOfFile Windows API. Both can map files and/or can create shared
(anonymous) maps that are backed by the swap device (if any). As a matter of
fact, glibc uses anonymous mmap() to implement malloc() when the requested
memory size is sufficiently large.

The biggest difference is the memory allocation granularity size. Linux is 4K
and Windows is 64K. If it's important to have say arbitrary 8K pages mapped
into specific 8K destinations well you are stuck on Windows and it just can't
be done.

Another difference is you can mmap a new page over the top of an existing page
effectively replacing the first page mapping. In Windows you can't do this
but instead must destroy the entire view and rebuild the entire view in what
ever new layout that is required. So if the "view" contains 1024 pages and
1 page changes then in Linux you can just change that one page. In Windows
you must drop all 1024 pages and re-view the same 1023 pages + the one new page.

IOW with only minimal data to share, Linux should be faster. A normal file
will probably do the job already, since most probably it is buffered in memory
anyway.

This is certainly interesting. Previously I wrote:

> Only a minimum amount of data is shared (a page with published
> parameters and achieved results, plus the ready! flags).

However, I see now that I asked for 'arbitrary size' in the system
call. Combined with a locked address, this could cause Windows to
swap a huge amount of memory on accesses, explaining the slow
execution.

I will have to spend more time reading the documentation after all.

Thanks a lot everybody, for the helpful comments!

-marcel

In article <2ec6da768657cb7e0838af11eb2d209e@www.novabbs.com>,
mhx <mhx@iae.nl> wrote:
>I have been polishing my shared memory application (iSPICE) a bit more.
>The benchmark I previously showed compared running a circuit simulation
>with a variable number of communicating CPUs. Only a minimum amount of data
>is shared (a page with published parameters and achieved results, plus the
>ready! flags). With this setup I got about a factor of 3 improvement for
>8 CPUs. I hoped to improve this factor a bit with better hardware and maybe
>some software tweaking.
>
>What I didn't try until today was checking how fast the circuit simulation
>ran on a single CPU, *not* using the shared memory framework. And indeed,
>that is a problem, in that without shared memory the runtime is *3 times
>less* than with shared memory. In other words, there is no net gain in
>having 8 mem-shared cpu's. As a additional check I started the circuit run
>in 3 separate windows. They all achieved the same speed as the single run
>non-shared version, proving that the hardware (cpu/memory/disk) is amply
>sufficient to provide an 8 times speed-up.
>
>I will now start working on Anton's suggesting of a shared file. Or maybe
>I should try this on Linux first, maybe shared memory works better there.

I simply use the clone system call on linux ( NR number is 56 for 64 bits)

( THREAD-PET KILL-PET PAUSE-PET ) CF: ?LI \ B5dec2
"CTA" WANTED "-syscalls-" WANTED HEX
\ Exit a thread. Indeed this is exit().
: EXIT-PET 0 _ _ __NR_exit XOS ;
\ Do a preemptive pause. ( abuse MS )
: PAUSE-PET 1 MS ;
\ Create a thread with dictionary SPACE. Execute XT in thread.
: THREAD-PET ALLOT CTA CREATE RSP@ SWAP RSP! R0 @ S0 @
ROT RSP! 2 CELLS - ( DSP) , ( TASK) , ( pid) 0 ,
DOES> DUP @ >R SWAP OVER CELL+ @ R@ 2! ( clone S: tp,xt)
100 R> _ __NR_clone XOS DUP IF
( Mother) DUP ?ERRUR SWAP 2 CELLS + ! ELSE
( Child) DROP RSP! CATCH DUP IF ERROR THEN EXIT-PET THEN ;
\ Kill a THREAD-PET , preemptively. Throw errors.
: KILL-PET >BODY 2 CELLS + @ 9 _ __NR_kill XOS ?ERRUR ;
DECIMAL

The idea is
1000 ( dictionary space ) CREATE extra

Now you run an xt as follows :
xt extra
The xt runs until it does an EXIT-PET, or is killed by a KILL-PET.

In r10par.frt it run 41 sec on one 27 on two processors for 10^12.
This was more a demonstration of parallel processing, the
communication and work load balancing kills the advantages for
more processors.

(This was prime counting)

Maybe try something simple before jumping into sockets and mapped
files.

The CTA words carves out a small dictionary space for the new
processes to be used, plus stacks and user space.
This is utterly system dependant, but in the ciforth model it
is just one screen, and portable over 32/64 arm/86 linux/windows.
It helps if you have a simple Forth to begin with ;-)
(CTA is used in cooperative multi tasking as well.)
>
>-marcel

Groetjes Albert
--
Don't praise the day before the evening. One swallow doesn't make spring.
You must not say "hey" before you have crossed the bridge. Don't sell the
hide of the bear until you shot it. Better one bird in the hand than ten in
the air. First gain is a cat purring. - the Wise from Antrim -

> Maybe try something simple before jumping into sockets and mapped
> files.

I have tried that way for the past 20 years already, and indeed it
works fine. However, my simple example shown above needs 24 threads
/processes/cores (whatever) each having about 2 to 4 GB of memory.

-marcel

In article <c53be72b665c3d10796bfe67a7f02dcf@www.novabbs.com>,
mhx <mhx@iae.nl> wrote:
>> Maybe try something simple before jumping into sockets and mapped
>> files.
>
>I have tried that way for the past 20 years already, and indeed it
>works fine. However, my simple example shown above needs 24 threads
>/processes/cores (whatever) each having about 2 to 4 GB of memory.

I have lost context, can you tell more about the simple example?
(My provider purges old messages swiftly)

And what with
lina -g 96000 lina96G

lina96G -e
...
WANT UNUSED
S[ ] OK UNUSED S>D DEC.

0,000,000,000,000,000,000,000,000,000,100,730,247,992
S[ ] OK
I'm sure most Forth's can do something similar.
(Overcommitting but not with my hp workstation with 256 Gbyte RAM).

>
>-marcel

Groetjes Albert
--
Don't praise the day before the evening. One swallow doesn't make spring.
You must not say "hey" before you have crossed the bridge. Don't sell the
hide of the bear until you shot it. Better one bird in the hand than ten in
the air. First gain is a cat purring. - the Wise from Antrim -

> I have lost context, can you tell more about the simple example?
> (My provider purges old messages swiftly)

I was in the exploring/debugging phase and have only very recently
completed the experiments.

The final results are that with shared memory, on Windows
11, it is possible to get an almost linear speedup with the
number of cores in use. The way shared memory is implemented
on Windows is with a memory-mapped file that uses the OS
pagefile as backup. The file is guaranteed to not be swapped
out under reasonable conditions, and Windows keeps its
management invisible for users.

I tried to make the file as small as possible. For this
iForth benchmark it was 11 int64's (11 * 8 bytes) and 24
extended floats (24 * 16 bytes), about 1/2 Kbyte. The file
is touched very infrequently, just 24 result writes and
then a loop over the 11 words to see if all cpu's finished
(check at 10ms intervals). At the moment I have no idea
what happens with very frequent read/writes (it is not
the intended type of use).

[During debugging I was lucky. When setting the number of
working cpu's interactively, completely wrong results
were obtained. This happened because #|cpus was defined
as a VALUE in a configuration file. When changing #|cpus
from the console, the value in sconfig.frt stayed the
same (of course) while all the dynamically started cores
used the on-disk value, not the value I typed in on
CPU #0. Easy to understand in hindsight, but this type
of 'black-hole' mistake can take hours to find in a 7000+
line program. For some reason I just knew that it had to
be #|cpus that was causing the problem.]

The benchmark is a circuit file that defines a voltage
source and a 2-resistor divider, all parameterized.
These values were swept for a total of 24 different
circuits. To calculate the result for one of the
combinations takes 2.277s on a single core with iSPICE,
or 24 x that value, 54.648s, for all 24 combinations.
In the benchmark the 24 simulations are spread out over
11 processes on an 8-core CPU :

iSPICE> .ticker-info
AMD Ryzen 7 5800X 8-Core Processor
TICKS-GET uses os time & PROCESSOR-CLOCK 4192MHz
Do: < n TO PROCESSOR-CLOCK RECALIBRATE >

The aim is to get an 8 times speedup, or more if
hyperthreads bring something, and do all combinations
in less than 6.831 seconds. The best I managed is
7.694s or about 7.67 "cores", which I consider not
that bad. Here are the details (run 4 times):

% cpus time [s] perf. ratio
1 49.874 1.46
2 25.314 2.39
3 17.391 3.23
4 13.335 4.11
5 10.565 5.17
6 9.468 5.71
7 8.712 6.22
8 7.694 7.67
9 7.260 7.37
10 7.874 6.72
11 7.856 6.73 ok

For your information: Running the same 24 variations
with LTspice 17.1.15, one of the fastest SPICE
implementations currently available, takes 382.265
seconds, almost exactly 7 times slower than the iSPICE
single-core run. Using 8 cores (LTspice pretends to
use 16 threads), that ratio becomes 62 times.

In the above table the performance ration for a single
cpu is 1.46 (1.46 times faster than doing the 24
simulations on a single core *without* shared memory),
which might seem strange. I think the phenomenon is
caused by the fact that a single combination takes
only 2.277s and this may be too slow for the processor
(or Windows) to ramp up the clock frequency. If the
performance factor is normalized by the timing for
1 cpu, the maximum speedup decreases to 5.25.
We'll see what happens on an HPZ840.
-marcel

In article <c2fb7eb58b7ae773f632a15c1abac917@www.novabbs.com>,
mhx <mhx@iae.nl> wrote:
>> I have lost context, can you tell more about the simple example?
>> (My provider purges old messages swiftly)
>
>I was in the exploring/debugging phase and have only very recently
>completed the experiments.

>
>The final results are that with shared memory, on Windows
>11, it is possible to get an almost linear speedup with the
>number of cores in use. The way shared memory is implemented
>on Windows is with a memory-mapped file that uses the OS
>pagefile as backup. The file is guaranteed to not be swapped
>out under reasonable conditions, and Windows keeps its
>management invisible for users.

Linear speedup? That must depend on the program.
Can I surmise that the context is that you're comparing your
version/clone iSpice with LTSpice.
>
>I tried to make the file as small as possible. For this
>iForth benchmark it was 11 int64's (11 * 8 bytes) and 24
>extended floats (24 * 16 bytes), about 1/2 Kbyte. The file
>is touched very infrequently, just 24 result writes and
>then a loop over the 11 words to see if all cpu's finished
>(check at 10ms intervals). At the moment I have no idea
>what happens with very frequent read/writes (it is not
>the intended type of use).

>
>[During debugging I was lucky. When setting the number of
> working cpu's interactively, completely wrong results
> were obtained. This happened because #|cpus was defined
> as a VALUE in a configuration file. When changing #|cpus
> from the console, the value in sconfig.frt stayed the
> same (of course) while all the dynamically started cores
> used the on-disk value, not the value I typed in on
> CPU #0. Easy to understand in hindsight, but this type
> of 'black-hole' mistake can take hours to find in a 7000+
> line program. For some reason I just knew that it had to
> be #|cpus that was causing the problem.]

>
>The benchmark is a circuit file that defines a voltage
>source and a 2-resistor divider, all parameterized.
>These values were swept for a total of 24 different
>circuits. To calculate the result for one of the
>combinations takes 2.277s on a single core with iSPICE,
>or 24 x that value, 54.648s, for all 24 combinations.
>In the benchmark the 24 simulations are spread out over
>11 processes on an 8-core CPU :
>
>iSPICE> .ticker-info
>AMD Ryzen 7 5800X 8-Core Processor
> TICKS-GET uses os time & PROCESSOR-CLOCK 4192MHz
> Do: < n TO PROCESSOR-CLOCK RECALIBRATE >
>
>The aim is to get an 8 times speedup, or more if
>hyperthreads bring something, and do all combinations
>in less than 6.831 seconds. The best I managed is
>7.694s or about 7.67 "cores", which I consider not
>that bad. Here are the details (run 4 times):
>
>% cpus time [s] perf. ratio
> 1 49.874 1.46
> 2 25.314 2.39
> 3 17.391 3.23
> 4 13.335 4.11
> 5 10.565 5.17
> 6 9.468 5.71
> 7 8.712 6.22
> 8 7.694 7.67
> 9 7.260 7.37
> 10 7.874 6.72
> 11 7.856 6.73 ok
>
>For your information: Running the same 24 variations
>with LTspice 17.1.15, one of the fastest SPICE
>implementations currently available, takes 382.265
>seconds, almost exactly 7 times slower than the iSPICE
>single-core run. Using 8 cores (LTspice pretends to
>use 16 threads), that ratio becomes 62 times.

So LT spice becomes slower by using 8 cores
going from 7 times slower to 62 time slower than iSPICE.
There must be a mistake here.
>
>In the above table the performance ration for a single
>cpu is 1.46 (1.46 times faster than doing the 24
>simulations on a single core *without* shared memory),
>which might seem strange. I think the phenomenon is
>caused by the fact that a single combination takes
>only 2.277s and this may be too slow for the processor
>(or Windows) to ramp up the clock frequency. If the
>performance factor is normalized by the timing for
>1 cpu, the maximum speedup decreases to 5.25.
>We'll see what happens on an HPZ840.

You are going to run Windows 11 on the HP work station?
I'm going to install a Linux version, for I want to
experiment with CUDA.

>
>-marcel
--
Don't praise the day before the evening. One swallow doesn't make spring.
You must not say "hey" before you have crossed the bridge. Don't sell the
hide of the bear until you shot it. Better one bird in the hand than ten in
the air. First gain is a cat purring. - the Wise from Antrim -

albert@spenarnc.xs4all.nl wrote:

> In article <c2fb7eb58b7ae773f632a15c1abac917@www.novabbs.com>,
> mhx <mhx@iae.nl> wrote:
>>> I have lost context, can you tell more about the simple example?
[..]
>>The final results are that with shared memory, on Windows
>>11, it is possible to get an almost linear speedup with the
>>number of cores in use.
[..]
> Linear speedup? That must depend on the program.
> Can I surmise that the context is that you're comparing your
> version/clone iSpice with LTSpice.

The example is *not* about trying to speed up programs
by adding threads to work on parts that can be parallelized.
A circuit simulator is used as the example here. Circuits
contain on average about 30% of operations that can be done
in parallel, so a fine-grained threaded approach with an
infinite amount of threads can at most give 30% of a speedup.

Most circuit simulation problems can not be solved with a
single simulation. In almost every case one wants to re-run
a job with small variations on the original specification.
The variations can be on the circuit components themselves,
on variations in environmental conditions like temperature,
humidity, noise, variations on input sources or output loads,
or even parameters of their (digital) control algorithms.
Between 10 and many thousands of simulations could be
necessary. At the top level, this problem is trivial to solve
by editing the input netlist with the necessary changes,
re-run the simulation, and store the results in a database.
When all runs are done, the data is evaluated by querying.

In practice, it is difficult to keep the administration
straight if the above is done by hand. What I am looking for
is a simple way to specify variations, create a list
of all the simulations needed, then distribute the tasks
to as many cpu cores as are available (locally, on the network,
or in the Cloud), combine the results, and generate reports.

To do this in Forth, I found it useful to use either shared
memory, or a shared file. The post is about experiments with
shared memory (useful when the number of cores is less than
256 and the main memory requirement is less than 1 TByte.)

The concrete example is to run N variations of a circuit on
an 8 core system with 32GB of memory, with the features I
describe above. The question was: is it possible to get
a speedup of 8 when the benchmark runs on an 8 core CPU.

>>iSPICE> .ticker-info
>>AMD Ryzen 7 5800X 8-Core Processor
>> TICKS-GET uses os time & PROCESSOR-CLOCK 4192MHz
>> Do: < n TO PROCESSOR-CLOCK RECALIBRATE >
>>
>>The aim is to get an 8 times speedup, or more if
>>hyperthreads bring something, and do all combinations
>>in less than 6.831 seconds. The best I managed is
>>7.694s or about 7.67 "cores", which I consider not
>>that bad. Here are the details (run 4 times):
>>
>>% cpus time [s] perf. ratio
>> 1 49.874 1.46
>> 2 25.314 2.39
>> 3 17.391 3.23
>> 4 13.335 4.11
>> 5 10.565 5.17
>> 6 9.468 5.71
>> 7 8.712 6.22
>> 8 7.694 7.67
>> 9 7.260 7.37
>> 10 7.874 6.72
>> 11 7.856 6.73 ok
>>

>>For your information: Running the same 24 variations
>>with LTspice 17.1.15, one of the fastest SPICE
>>implementations currently available, takes 382.265
>>seconds, almost exactly 7 times slower than the iSPICE
>>single-core run. Using 8 cores (LTspice pretends to
>>use 16 threads), that ratio becomes 62 times.

I realize now that this comparison of iSPICE with LTspice
can confuse the reader. It does not matter at all for this
benchmark which SPICE simulator is used.

> So LT spice becomes slower by using 8 cores
> going from 7 times slower to 62 time slower than iSPICE.
> There must be a mistake here.

There is no mistake. LTspice is 7 slower than iSPICE for
the specific type of task used here. Although LTspice has
a mechanism to run multiple variations, and claims to use
8 cores / 16 threads, it does not appear to use them as
efficiently as iSPICE does using shared memory.

[..]
>>We'll see what happens on an HPZ840.

> You are going to run Windows 11 on the HP work station?
> I'm going to install a Linux version, for I want to
> experiment with CUDA.

I certainly want to see what happens if I run iSPICE on
my 44-core HPZ840 :--) The fastest way to implement that
should be to install Windows 10 or 11 on the HP. However,
if that proves problematic I have no problem using Linux.
I did not try iSPICE on Linux/WSL2 yet and I probably will
do that first.

I also want to experiment with CUDA (BTW, why not OpenCL,
did you already find arguments against that route?),
however, that would be to investigate a new way of circuit
simulation that not uses the standard SPICE algorithms.

-marcel