Hi,

I don't have the code to show up as it's in a corporate environment, but
I'll explain where is my difficulty.

I've already built a script to parse text output from a command.
processLine does what I need, but I'd like to have it assyncrhonouly:

proc processLine {line inside_block} {
if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
# block name or header
set inside_block 1
set block_name .....
dict set parsed $block_name (simplified)
return $inside_block
}
if {$inside_block && blank line..} {
# it's a block separator
set inside_block 0
}
if {$inside_block} {
# lines after the block header and before the next blank line
# do some parsing and fill in parsed dict
}
}

proc parseLine {chan inside_block} {
set status [catch {chan gets $chan line} nchars]
if {$status == 0 && $nchars >= 0} {
set inside_block [processLine $line $inside_block]
return
}
if error or EOF
return
return $inside_block
}

And the main:
set inside_block 0
set chan [open |[list {*}shell_command] r]

chan event $chan readable [list parseLine $chan $inside_block]

vwait...
show dict parsed

So, in the first version I built, the synchronous one, the 'inside_block'
value is passed back and forth so that 'processLine' know where does the
context of the line.

But how can I do this when using channels for an asynchronous version?
It would be something like:
when 'chan event $chan readable ...' detects a new line arriving,
it should read the result of 'parseLine $chan $inside_block', so that for
the next line the new value of inside_block could be passed as an argument
to 'parseLine'.

I'd appreciate some help on this.
Thank you,

Luís Mendes

Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
> I don't have the code to show up as it's in a corporate environment, but
> I'll explain where is my difficulty.
>
> ... [see prior post for code]
>
> So, in the first version I built, the synchronous one, the
> 'inside_block' value is passed back and forth so that 'processLine'
> know where does the context of the line.
>
> But how can I do this when using channels for an asynchronous version?
> It would be something like:
> when 'chan event $chan readable ...' detects a new line arriving,
> it should read the result of 'parseLine $chan $inside_block', so that for
> the next line the new value of inside_block could be passed as an argument
> to 'parseLine'.
>
> I'd appreciate some help on this.
> Thank you,

You have several options.

1) make 'inside_block' a global:

proc processLine {line} {
if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
# block name or header
set ::inside_block 1
set block_name .....
dict set parsed $block_name (simplified)
}
if {$::inside_block && blank line..} {
# it's a block separator
set ::inside_block 0
}
if {$::inside_block} {
# lines after the block header and before the next blank line
# do some parsing and fill in parsed dict
}
}

But, then, you can only have one instance of this parse going on at one
time.

2) move "processLine" into a namespace, and use a namespace variable:

namespace eval SomeName {
variable inside_block 0

proc processLine {line} {
if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
# block name or header
set inside_block 1
set block_name .....
dict set parsed $block_name (simplified)
}
if {$inside_block && blank line..} {
# it's a block separator
set inside_block 0
}
if {$inside_block} {
# lines after the block header and before the next blank line
# do some parsing and fill in parsed dict
}
}
}

And then setup the callback to call SomeName::processLine

With this method you can have more than one parse at a time running,
provided you create differently named namespaces

3) Make processLine an object and use an object variable (code omitted)

With this method, you can create "processLine" objects as needed to
attach to the callbacks, and they each get their own unique variable
namespace automatically. If you do plan on running plural parses at
the same time then this is your better option, as it frees you from
needing to work out how to handle making custom named namespaces to
have plural parses going on.

If you only ever plan to parse one thing at a time, either a global or
a namespace work. Which is 'better' is somewhat dependent upon what
kind of 'code review' you might have to endure.

See https://www.magicsplat.com/articles/oo.html for a good description
of objects.

4) You can redefine the callback as you go, so you 'could' pass the
value in via the callback by redefining the callback to contain the
value that should be present for the next call to processLine.

And there are probably a few more options that have not come to mind
yet.

* Rich <rich@example.invalid>
| Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
| > It would be something like:
| > when 'chan event $chan readable ...' detects a new line arriving,
| > it should read the result of 'parseLine $chan $inside_block', so that for
| > the next line the new value of inside_block could be passed as an argument
| > to 'parseLine'.
| >
| > I'd appreciate some help on this.
| > Thank you,
>
| You have several options.
--<snip-snip>--
| And there are probably a few more options that have not come to mind
| yet.

Could coroutines come into play here? I haven't played with them yet,
but stored them as "keep context and continue from there after suspend",
so it sounds quite what the OP needs. Might be too complex for the task
at hand, though...

R'

Hi Rich,

Thank you for your answer.
Please, see below.

On Mon, 15 Apr 2024 22:11:35 -0000 (UTC), Rich wrote:
> You have several options.
>
> 1) make 'inside_block' a global:
> But, then, you can only have one instance of this parse going on at one
> time.
I want to have several instances running at the same time from several
channels, so no global variable.

> 2) move "processLine" into a namespace, and use a namespace variable:
>
> namespace eval SomeName {
> variable inside_block 0
>
> proc processLine {line} {
> if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
> # block name or header set inside_block 1 set block_name
> .....
> dict set parsed $block_name (simplified)
> }
> if {$inside_block && blank line..} {
> # it's a block separator set inside_block 0
> }
> if {$inside_block} {
> # lines after the block header and before the next blank
> line # do some parsing and fill in parsed dict
> }
> }
> }
>
> And then setup the callback to call SomeName::processLine
>
> With this method you can have more than one parse at a time running,
> provided you create differently named namespaces
So, would it be a namespace created dynamically?
Could you please share how a small example?

> 3) Make processLine an object and use an object variable (code omitted)
>
> With this method, you can create "processLine" objects as needed to
> attach to the callbacks, and they each get their own unique variable
> namespace automatically. If you do plan on running plural parses at the
> same time then this is your better option, as it frees you from needing
> to work out how to handle making custom named namespaces to have plural
> parses going on.
>
> If you only ever plan to parse one thing at a time, either a global or a
> namespace work. Which is 'better' is somewhat dependent upon what kind
> of 'code review' you might have to endure.
>
> See https://www.magicsplat.com/articles/oo.html for a good description
> of objects.
Never used OO in Tcl, not a fan of OO, but I'm trying to build something
around it.
The two main variable inside_block and chan could be set for the instance
to be visible all over the class and then I'd create instances of the
object like so?

oo::class create Parse {
variable inside_block chan
method parseLine {} {...}
method processLine {} {...}
}

For the main part of the script:
foreach id $exec_ids {
set inst${id} [Parse new]
}

>
> 4) You can redefine the callback as you go, so you 'could' pass the
> value in via the callback by redefining the callback to contain the
> value that should be present for the next call to processLine.
What should I change in this line?
chan event $chan readable [list parseLine $chan $inside_block]

Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
> Hi Rich,
>
> Thank you for your answer.
> Please, see below.
>
> On Mon, 15 Apr 2024 22:11:35 -0000 (UTC), Rich wrote:
>> You have several options.
>>
>> 1) make 'inside_block' a global:
>> But, then, you can only have one instance of this parse going on at one
>> time.
> I want to have several instances running at the same time from several
> channels, so no global variable.

Then, obviously, this is not a reasonable solution.

>> 2) move "processLine" into a namespace, and use a namespace variable:
>>
>> namespace eval SomeName {
>> variable inside_block 0
>>
>> proc processLine {line} {
>> if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
>> # block name or header set inside_block 1 set block_name
>> .....
>> dict set parsed $block_name (simplified)
>> }
>> if {$inside_block && blank line..} {
>> # it's a block separator set inside_block 0
>> }
>> if {$inside_block} {
>> # lines after the block header and before the next blank
>> line # do some parsing and fill in parsed dict
>> }
>> }
>> }
>>
>> And then setup the callback to call SomeName::processLine
>>
>> With this method you can have more than one parse at a time running,
>> provided you create differently named namespaces
> So, would it be a namespace created dynamically?
> Could you please share how a small example?

Yes, you'd have to do dynamic namespace creation. Which 'could' be
something like:

set uid 0

namespace eval pl-[incr uid] {
...
proc ... {} {...}
proc ... {} {...}
}

But, if you use objects, you get all of the above handled for your
automatically as part of the 'new' or 'create' call to creating class.
Which dramatically simplifies the effort you have to put in (you can
focus on your actual project, not on building a quasi-object layer out
of namespaces).

>> 3) Make processLine an object and use an object variable (code omitted)
>>
>> See https://www.magicsplat.com/articles/oo.html for a good description
>> of objects.

> Never used OO in Tcl, not a fan of OO, but I'm trying to build
> something around it.

Once you start to use it some, you'll be surprised as all the places
where you created a "partial object layer" to avoid global variables,
but didn't realize you were building part of an 'object' system at the
time.

> The two main variable inside_block and chan could be set for the
> instance to be visible all over the class and then I'd create
> instances of the object like so?
>
> oo::class create Parse {
> variable inside_block chan
> method parseLine {} {...}
> method processLine {} {...}
> }

Yes, that's the general template of what you'd want. Note you can also
have a 'constructor' method that can do extra work to 'create' the
object when you call 'new' or 'create' and you can have a 'destructor'
method that can cleanup (such as by being sure that 'chan' is closed
and the like).

> For the main part of the script:
> foreach id $exec_ids {
> set inst${id} [Parse new]
> }

Yes, and each 'new' call gives you an object with its own private
"inside_block" and 'chan' variabes, so you can have plural parses going
simultaneously with no conflict (at least no conflict at this level of
your project). And if you setup a 'destructor' then you can simply
call 'destroy' on the object, and have everything cleaned up ('chan'
channel closed, any memory allocated that does not get cleaned up by
Tcl's reference counting freed, etc.).

>> 4) You can redefine the callback as you go, so you 'could' pass the
>> value in via the callback by redefining the callback to contain the
>> value that should be present for the next call to processLine.
> What should I change in this line?
> chan event $chan readable [list parseLine $chan $inside_block]

You'd put that into the "process proc" such that just before you
existed 'process' you'd call that chan event, to attach the new value
of "$inside_block" to the handler.

Using a minimum amount of 'objects' (one class, for creating each
'parse' object and your parse procs as methods in those objects) is
likely the least complex of the various possibilties.

Then just attach the object to the event handler for each channel to
call the proper object method when the channel is readable.

On Tue, 16 Apr 2024 18:27:34 -0000 (UTC), Rich wrote:
>>> 3) Make processLine an object and use an object variable (code
>>> omitted)
>>>
>>> See https://www.magicsplat.com/articles/oo.html for a good description
>>> of objects.
>
>> Never used OO in Tcl, not a fan of OO, but I'm trying to build
>> something around it.
>
> Once you start to use it some, you'll be surprised as all the places
> where you created a "partial object layer" to avoid global variables,
> but didn't realize you were building part of an 'object' system at the
> time.
>
>> The two main variable inside_block and chan could be set for the
>> instance to be visible all over the class and then I'd create instances
>> of the object like so?
>>
>> oo::class create Parse {
>> variable inside_block chan method parseLine {} {...} method
>> processLine {} {...}
>> }
>
> Yes, that's the general template of what you'd want. Note you can also
> have a 'constructor' method that can do extra work to 'create' the
> object when you call 'new' or 'create' and you can have a 'destructor'
> method that can cleanup (such as by being sure that 'chan' is closed and
> the like).
>
>> For the main part of the script:
>> foreach id $exec_ids {
>> set inst${id} [Parse new]
>> }
>
> Yes, and each 'new' call gives you an object with its own private
> "inside_block" and 'chan' variabes, so you can have plural parses going
> simultaneously with no conflict (at least no conflict at this level of
> your project). And if you setup a 'destructor' then you can simply call
> 'destroy' on the object, and have everything cleaned up ('chan' channel
> closed, any memory allocated that does not get cleaned up by Tcl's
> reference counting freed, etc.).

I've created a minimum example that is not working.

First, a simple example that works:
=== procedural ===
#!/usr/local/bin/tclsh

proc parseLine {chan_r} {
set status [catch {chan gets $chan_r line} nchars]
puts "status: $status line: $line nchars: $nchars"
if {$status == 0 && $nchars >= 0} {
puts "received: $line"
# processLine
return
}
if {$status || [chan eof $chan_r]} {
chan event $chan_r readable {}
set ::exit_flag 1
chan close $chan_r
return
}
puts "incomplete line"
}

set chan_r [open |[list ls] r]
chan configure $chan_r -blocking 0 -buffering line
chan event $chan_r readable [list parseLine $chan_r]

vwait ::exit_flag
===============

=== object oriented ===
#!/usr/local/bin/tclsh

oo::class create Parse {
variable chan_r
constructor {comm} {
set chan_r [open |[list $comm] r]
chan configure $chan_r -blocking 0 -buffering line
chan event $chan_r readable [my parseLine]
}
method parseLine {} {
set status [catch {chan gets $chan_r line} nchars]
puts "status: $status nchars: $nchars chan eof: [chan eof
$chan_r] chan blocked: [chan blocked $chan_r] line: $line"
if {$status == 0 && $nchars >= 0} {
puts "received: $line"
#my processLine $line
return
}
if {$status || [chan eof $chan_r]} {
puts ">> All received from test!"
chan event $chan_r readable {}
set ::exit_flag 1
chan close $chan_r
return
}
puts "incomplete line"
}
}

set exec_id 0
set run_${exec_id} [Parse new ls]

vwait ::exit_flag_simple
==========

This OO example returns one line if -blocking is set to 1, than is waiting
forever.
If -blocking 0, then prints nothing.

It seems that placing `chan event $chan_r readable [my parseLine]` in the
constructor method makes that the context for the chan event is lost
somehow.
How can this work?

Another question.
What I'm building is supposed to read a log from an Ansible process, so
lines are being sent to stdout for some minutes, one shortly after another.
I'd like the script to parse the log of several Ansible processes running
in parallel, thus the approach to 'set run_${exec_id} [Parse new ...]'.
Does the option '-blocking' make a difference in this context?

Thanks,

Luís

Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
> On Tue, 16 Apr 2024 18:27:34 -0000 (UTC), Rich wrote:
>>>> 3) Make processLine an object and use an object variable (code
>>>> omitted)
>>>>
>>>> See https://www.magicsplat.com/articles/oo.html for a good description
>>>> of objects.
>>
>>> Never used OO in Tcl, not a fan of OO, but I'm trying to build
>>> something around it.
>>
>>>
>>> oo::class create Parse {
>>> variable inside_block chan method parseLine {} {...} method
>>> processLine {} {...}
>>> }
>>
>> Yes, that's the general template of what you'd want. Note you can also
>> have a 'constructor' method that can do extra work to 'create' the
>> object when you call 'new' or 'create' and you can have a 'destructor'
>> method that can cleanup (such as by being sure that 'chan' is closed and
>> the like).
>>
>>> For the main part of the script:
>>> foreach id $exec_ids {
>>> set inst${id} [Parse new]
>>> }
>>
>> Yes, and each 'new' call gives you an object with its own private
>> "inside_block" and 'chan' variabes, so you can have plural parses going
>> simultaneously with no conflict (at least no conflict at this level of
>> your project). And if you setup a 'destructor' then you can simply call
>> 'destroy' on the object, and have everything cleaned up ('chan' channel
>> closed, any memory allocated that does not get cleaned up by Tcl's
>> reference counting freed, etc.).
>
> I've created a minimum example that is not working.
>
>
> === object oriented ===
> #!/usr/local/bin/tclsh
>
> oo::class create Parse {
> variable chan_r
> constructor {comm} {
> set chan_r [open |[list $comm] r]
> chan configure $chan_r -blocking 0 -buffering line
> chan event $chan_r readable [my parseLine]
> }

If you do a quick test, you'll see what has gone wrong:

$ rlwrap tclsh
% oo::class create Parse {
constructor {} {
puts stderr "in constructor '[my parseLine]'"
}
method parseLine {} {
}
}
::Parse
% Parse new
in constructor ''
::oo::Obj12
%

The [my] call, in the constructor, returned nothing. Additionally,
[my] is defined only for use in methods, for calling other methods of
the same object. Event callbacks in Tcl register bits of script code
to execute later in time (i.e., not at the time of definition) and
almost allways execute the attached code in the global context (i.e.,
outside of your object).

You need to instead setup the event script as if you were calling the
object from the outside (which you in fact are doing) by calling the
desired method on the object's name):

$ rlwrap tclsh
% oo::class create Parse {
constructor {} {
puts stderr "in constructor '[self object]'"
}
}
::Parse
% Parse new
in constructor '::oo::Obj12'
::oo::Obj12
%

So your chan event needs to read:

chan event $chan_r readable [list [self object] parseLine]

And things should begin working (or at least the reason for not
receiving a callback will go away).

> It seems that placing `chan event $chan_r readable [my parseLine]` in the
> constructor method makes that the context for the chan event is lost
> somehow.

No, it never setup the event callback properly, because [my] in the
constructor returned an empty string.

> How can this work?

See above.

> Another question.
> What I'm building is supposed to read a log from an Ansible process,
> so lines are being sent to stdout for some minutes, one shortly after
> another. I'd like the script to parse the log of several Ansible
> processes running in parallel, thus the approach to 'set
> run_${exec_id} [Parse new ...]'. Does the option '-blocking' make a
> difference in this context?

Yes, if you want to use 'chan event' you have to set the channels to
non-blocking mode (-blocking 0). Otherwise you can't use 'chan event'
to have the channels handled in an event driven manner. And with
blocking, the first one where you try to gets or read and the channel
does not have enough data will block the entire process.

On 4/15/2024 12:54 PM, Luis Mendes wrote:
> Hi,
>
>
> I don't have the code to show up as it's in a corporate environment, but
> I'll explain where is my difficulty.
>
> I've already built a script to parse text output from a command.
> processLine does what I need, but I'd like to have it assyncrhonouly:
>
> proc processLine {line inside_block} {
> if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
> # block name or header
> set inside_block 1
> set block_name .....
> dict set parsed $block_name (simplified)
> return $inside_block
> }
> if {$inside_block && blank line..} {
> # it's a block separator
> set inside_block 0
> }
> if {$inside_block} {
> # lines after the block header and before the next blank line
> # do some parsing and fill in parsed dict
> }
> }
>
> proc parseLine {chan inside_block} {
> set status [catch {chan gets $chan line} nchars]
> if {$status == 0 && $nchars >= 0} {
> set inside_block [processLine $line $inside_block]
> return
> }
> if error or EOF
> return
> return $inside_block
> }
>
> And the main:
> set inside_block 0
> set chan [open |[list {*}shell_command] r]
>
> chan event $chan readable [list parseLine $chan $inside_block]
>
> vwait...
> show dict parsed
>
>
> So, in the first version I built, the synchronous one, the 'inside_block'
> value is passed back and forth so that 'processLine' know where does the
> context of the line.
>
> But how can I do this when using channels for an asynchronous version?
> It would be something like:
> when 'chan event $chan readable ...' detects a new line arriving,
> it should read the result of 'parseLine $chan $inside_block', so that for
> the next line the new value of inside_block could be passed as an argument
> to 'parseLine'.
>
> I'd appreciate some help on this.
> Thank you,
>
>
> Luís Mendes

I've only quickly read your post, so I didn't see what you want to do with the parsing output.

Monitoring of N other processes by creating OO objects each involving event driven code is more like trying to simulate separate processes or threads all in a single threaded program.

Assuming you need to correlate the output of each "parser" thread in a single place (otherwise just exec-ing processes would be enough), then tcl threads is what you really want. And if you're not a fan of OO, then you might as well do a crash course on threads instead of tclOO :)

The advantage of threads is that each thread can do simple sequential coding:

open channels etc. store in global variables
loop
read-with-block, parse, report
endloop

(I'm assuming you have a modern tcl, say 8.6.6 or later)

Each new thread would also have it's own private set of global variables (sort of like variable in a class).

To collect the parse data from all the threads, (the report step) you would thread::send from the parser threads back to the main thread a call to some collection proc, with the collected data as the arguments. The main thread would just do a vwait for-exit. The collection proc would be called fifo from each parser thread, and provided you don't do any updates or vwaits, could also run "sequentially".

How you intend to know when to exit or fire off some additional monitoring threads is a bookkeeping job, left as an exercise for the reader.

On Wed, 17 Apr 2024 19:06:08 -0000 (UTC), Rich wrote:
>>
>> === object oriented ===
>> #!/usr/local/bin/tclsh
>>
>> oo::class create Parse {
>> variable chan_r constructor {comm} {
>> set chan_r [open |[list $comm] r]
>> chan configure $chan_r -blocking 0 -buffering line chan event
>> $chan_r readable [my parseLine]
>> }
>
> If you do a quick test, you'll see what has gone wrong:
>
> $ rlwrap tclsh % oo::class create Parse {
> constructor {} {
> puts stderr "in constructor '[my parseLine]'"
> }
> method parseLine {} {
> }
> }
> ::Parse % Parse new in constructor ''
> ::oo::Obj12 %
>
> The [my] call, in the constructor, returned nothing. Additionally, [my]
> is defined only for use in methods, for calling other methods of the
> same object. Event callbacks in Tcl register bits of script code to
> execute later in time (i.e., not at the time of definition) and almost
> allways execute the attached code in the global context (i.e., outside
> of your object).
>
> You need to instead setup the event script as if you were calling the
> object from the outside (which you in fact are doing) by calling the
> desired method on the object's name):
>
> $ rlwrap tclsh % oo::class create Parse {
> constructor {} {
> puts stderr "in constructor '[self object]'"
> }
> }
> ::Parse % Parse new in constructor '::oo::Obj12'
> ::oo::Obj12 %
>
> So your chan event needs to read:
>
> chan event $chan_r readable [list [self object] parseLine]
>
> And things should begin working (or at least the reason for not
> receiving a callback will go away).

Thank you very much Rich, this was a show stopper and with your help and
knowledge I could move forward.
All the best to you!

Luís

On Wed, 17 Apr 2024 21:00:28 -0700, et99 wrote:

> On 4/15/2024 12:54 PM, Luis Mendes wrote:
>> Hi,
>>
>>
>> I don't have the code to show up as it's in a corporate environment,
>> but I'll explain where is my difficulty.
>>
>> I've already built a script to parse text output from a command.
>> processLine does what I need, but I'd like to have it assyncrhonouly:
>>
>> proc processLine {line inside_block} {
>> if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
>> # block name or header set inside_block 1
>> set block_name .....
>> dict set parsed $block_name (simplified)
>> return $inside_block
>> }
>> if {$inside_block && blank line..} {
>> # it's a block separator
>> set inside_block 0
>> }
>> if {$inside_block} {
>> # lines after the block header and before the next blank line # do
>> some parsing and fill in parsed dict
>> }
>> }
>>
>> proc parseLine {chan inside_block} {
>> set status [catch {chan gets $chan line} nchars]
>> if {$status == 0 && $nchars >= 0} {
>> set inside_block [processLine $line $inside_block]
>> return
>> }
>> if error or EOF
>> return
>> return $inside_block
>> }
>>
>> And the main:
>> set inside_block 0 set chan [open |[list {*}shell_command] r]
>>
>> chan event $chan readable [list parseLine $chan $inside_block]
>>
>> vwait...
>> show dict parsed
>>
>>
>> So, in the first version I built, the synchronous one, the
>> 'inside_block' value is passed back and forth so that 'processLine'
>> know where does the context of the line.
>>
>> But how can I do this when using channels for an asynchronous version?
>> It would be something like:
>> when 'chan event $chan readable ...' detects a new line arriving,
>> it should read the result of 'parseLine $chan $inside_block', so that
>> for the next line the new value of inside_block could be passed as an
>> argument to 'parseLine'.
>>
>> I'd appreciate some help on this.
>> Thank you,
>>
>>
>> Luís Mendes
>
>
> I've only quickly read your post, so I didn't see what you want to do
> with the parsing output.
>
> Monitoring of N other processes by creating OO objects each involving
> event driven code is more like trying to simulate separate processes or
> threads all in a single threaded program.
>
> Assuming you need to correlate the output of each "parser" thread in a
> single place (otherwise just exec-ing processes would be enough), then
> tcl threads is what you really want. And if you're not a fan of OO, then
> you might as well do a crash course on threads instead of tclOO :)
>
> The advantage of threads is that each thread can do simple sequential
> coding:
>
> open channels etc. store in global variables loop
> read-with-block, parse, report
> endloop
>
> (I'm assuming you have a modern tcl, say 8.6.6 or later)
>
> Each new thread would also have it's own private set of global variables
> (sort of like variable in a class).
>
> To collect the parse data from all the threads, (the report step) you
> would thread::send from the parser threads back to the main thread a
> call to some collection proc, with the collected data as the arguments.
> The main thread would just do a vwait for-exit. The collection proc
> would be called fifo from each parser thread, and provided you don't do
> any updates or vwaits, could also run "sequentially".
>
> How you intend to know when to exit or fire off some additional
> monitoring threads is a bookkeeping job, left as an exercise for the
> reader.

Hi et99,

Yes, that's a good idea and threads I'd like to play with threads.
Some time ago, I tried a very simple example and it was segfaulting
although the code run sequentially was not. And didn't considered threads
for this example but could fit very well for solving the problem.

Thank you,

Luís

On 4/18/2024 9:26 AM, Luis Mendes wrote:
> On Wed, 17 Apr 2024 21:00:28 -0700, et99 wrote:
>
>> On 4/15/2024 12:54 PM, Luis Mendes wrote:
>>> Hi,
>>>
>>>
>>> I don't have the code to show up as it's in a corporate environment,
>>> but I'll explain where is my difficulty.
>>>
>>> I've already built a script to parse text output from a command.
>>> processLine does what I need, but I'd like to have it assyncrhonouly:
>>>
>>> proc processLine {line inside_block} {
>>> if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
>>> # block name or header set inside_block 1
>>> set block_name .....
>>> dict set parsed $block_name (simplified)
>>> return $inside_block
>>> }
>>> if {$inside_block && blank line..} {
>>> # it's a block separator
>>> set inside_block 0
>>> }
>>> if {$inside_block} {
>>> # lines after the block header and before the next blank line # do
>>> some parsing and fill in parsed dict
>>> }
>>> }
>>>
>>> proc parseLine {chan inside_block} {
>>> set status [catch {chan gets $chan line} nchars]
>>> if {$status == 0 && $nchars >= 0} {
>>> set inside_block [processLine $line $inside_block]
>>> return
>>> }
>>> if error or EOF
>>> return
>>> return $inside_block
>>> }
>>>
>>> And the main:
>>> set inside_block 0 set chan [open |[list {*}shell_command] r]
>>>
>>> chan event $chan readable [list parseLine $chan $inside_block]
>>>
>>> vwait...
>>> show dict parsed
>>>
>>>
>>> So, in the first version I built, the synchronous one, the
>>> 'inside_block' value is passed back and forth so that 'processLine'
>>> know where does the context of the line.
>>>
>>> But how can I do this when using channels for an asynchronous version?
>>> It would be something like:
>>> when 'chan event $chan readable ...' detects a new line arriving,
>>> it should read the result of 'parseLine $chan $inside_block', so that
>>> for the next line the new value of inside_block could be passed as an
>>> argument to 'parseLine'.
>>>
>>> I'd appreciate some help on this.
>>> Thank you,
>>>
>>>
>>> Luís Mendes
>>
>>
>> I've only quickly read your post, so I didn't see what you want to do
>> with the parsing output.
>>
>> Monitoring of N other processes by creating OO objects each involving
>> event driven code is more like trying to simulate separate processes or
>> threads all in a single threaded program.
>>
>> Assuming you need to correlate the output of each "parser" thread in a
>> single place (otherwise just exec-ing processes would be enough), then
>> tcl threads is what you really want. And if you're not a fan of OO, then
>> you might as well do a crash course on threads instead of tclOO :)
>>
>> The advantage of threads is that each thread can do simple sequential
>> coding:
>>
>> open channels etc. store in global variables loop
>> read-with-block, parse, report
>> endloop
>>
>> (I'm assuming you have a modern tcl, say 8.6.6 or later)
>>
>> Each new thread would also have it's own private set of global variables
>> (sort of like variable in a class).
>>
>> To collect the parse data from all the threads, (the report step) you
>> would thread::send from the parser threads back to the main thread a
>> call to some collection proc, with the collected data as the arguments.
>> The main thread would just do a vwait for-exit. The collection proc
>> would be called fifo from each parser thread, and provided you don't do
>> any updates or vwaits, could also run "sequentially".
>>
>> How you intend to know when to exit or fire off some additional
>> monitoring threads is a bookkeeping job, left as an exercise for the
>> reader.
>
> Hi et99,
>
> Yes, that's a good idea and threads I'd like to play with threads.
> Some time ago, I tried a very simple example and it was segfaulting
> although the code run sequentially was not. And didn't considered threads
> for this example but could fit very well for solving the problem.
>
> Thank you,
>
> Luís

Hi Luis:

A while back I really didn't know much about the tcl threads package. But then I bought Ashok's book. I can't recommend it more highly. Whatever technique you decide on, threads, event driven, channels, tclOO, it's all in the book with great examples you can copy/paste right into your program (from the pdf version). As Rich pointed out, you can read the tclOO chapter for free on line.

-et

On Thu, 18 Apr 2024 19:52:42 -0700, et99 wrote:

> Hi Luis:
>
> A while back I really didn't know much about the tcl threads package.
> But then I bought Ashok's book. I can't recommend it more highly.
> Whatever technique you decide on, threads, event driven, channels,
> tclOO, it's all in the book with great examples you can copy/paste right
> into your program (from the pdf version). As Rich pointed out, you can
> read the tclOO chapter for free on line.
>
> -et

Hi et,

Yes, I've also bought the book and it has been my guide. Very good indeed.

Thank you,

Luís