As some here may recall, I'm in the midst of several years' work
scanning family documents, post-processing them, and then releasing them
either to the general public for generations sufficiently removed from
living people, or else just back to the source branch of the family.
Currently there are nearly 15,000 original documents with equal numbers
of post-processed versions arranged as below, but around 7,000 of them
completed in previous stages already have final documents, so there are
around 8,000 newly added documents left to do. What I need to do is use
a program, feh seems a likely choice, to work through those documents
that don't yet have a final version, choose one from five possible
versions described below, copy it to its corresponding final directory,
and move on to the next.

The documents are arranged as follows:

Family History
/_all/1o/subs Original scans
/_all/2n/subs Normalised versions of the originals
/_all/3t/subs Textcleaned versions ditto
/_all/4nt/subs Normalised then textcleaned versions ditto
/_all/5tn/subs Textcleaned then normalised versions ditto
/subs Final docs chosen from the best of the above

Example for one particular subdirectory:

Family History/_all/1o/Alston
Family History/_all/2n/Alston
Family History/_all/3t/Alston
Family History/_all/4nt/Alston
Family History/_all/5tn/Alston
Family History/Alston

I've tried the following command line for feh (sorry can't help any line
wrap):

user@computer:~# feh -Fdr -A 'echo cp -a \"%F\" \"$(echo %F | sed -E "{
s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family
History/_all/1o'

.... which results in ...

cp -a "Family History/_all/1o/Alston/Alston Descent - 2nd Earl Of
Montrose, William - 1 (o).png" "Family History/Alston/Alston Descent -
2nd Earl Of Montrose, William - 1.png"

.... which looks as though it should do the job, however removing the
echo ...

user@computer:~# feh -Fdr -A 'cp -a \"%F\" \"$(echo %F | sed -E "{
s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family
History/_all/1o'

.... results in ...

cp: target '1.png"' is not a directory

Can anyone suggest a magical incantation which will yield the desired
result?

Another problem is that I'm going to have order the files so that each
of the 5 possible versions are displayed successively so that I can use
the arrow keys to go back and forth between them to choose the best. I
think I might be able to use a file list to do this, but as yet I
haven't given much thought as to how to create it, it would have to be
of the form:

Family History/_all/1o/Alston/Document 1 (o).png
Family History/_all/2n/Alston/Document 1 (n).png
Family History/_all/3t/Alston/Document 1 (t).png
Family History/_all/4nt/Alston/Document 1 (nt).png
Family History/_all/5tn/Alston/Document 1 (tn).png
Family History/_all/1o/Alston/Document 2 (o).png
Family History/_all/2n/Alston/Document 2 (n).png
Family History/_all/3t/Alston/Document 2 (t).png
Family History/_all/4nt/Alston/Document 2 (nt).png
Family History/_all/5tn/Alston/Document 2 (tn).png
etc.

I could also use Windows if there's a better solution available there,
but that seems unlikely to me.

Any help gratefully received.

--

Fake news kills!

I may be contacted via the contact address given on my website:
www.macfh.co.uk

First problem solved ...

On 29/03/2024 22:54, Java Jive wrote:
> As some here may recall, I'm in the midst of several years' work
> scanning family documents, post-processing them, and then releasing them
> either to the general public for generations sufficiently removed from
> living people, or else just back to the source branch of the family.
> Currently there are nearly 15,000 original documents with equal numbers
> of post-processed versions arranged as below, but around 7,000 of them
> completed in previous stages already have final documents, so there are
> around 8,000 newly added documents left to do.  What I need to do is use
> a program, feh seems a likely choice, to work through those documents
> that don't yet have a final version, choose one from five possible
> versions described below, copy it to its corresponding final directory,
> and move on to the next.
>
> The documents are arranged as follows:
>
> Family History
>     /_all/1o/subs    Original scans
>     /_all/2n/subs    Normalised versions of the originals
>     /_all/3t/subs    Textcleaned versions ditto
>     /_all/4nt/subs    Normalised then textcleaned versions ditto
>     /_all/5tn/subs    Textcleaned then normalised versions ditto
>     /subs        Final docs chosen from the best of the above
>
> Example for one particular subdirectory:
>
> Family History/_all/1o/Alston
> Family History/_all/2n/Alston
> Family History/_all/3t/Alston
> Family History/_all/4nt/Alston
> Family History/_all/5tn/Alston
> Family History/Alston
>
> I've tried the following command line for feh (sorry can't help any line
> wrap):
>
> user@computer:~# feh -Fdr -A 'echo cp -a \"%F\" \"$(echo %F | sed -E "{
> s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family
> History/_all/1o'
>
> .... which results in ...
>
> cp -a "Family History/_all/1o/Alston/Alston Descent - 2nd Earl Of
> Montrose, William - 1 (o).png" "Family History/Alston/Alston Descent -
> 2nd Earl Of Montrose, William - 1.png"
>
> .... which looks as though it should do the job, however removing the
> echo ...
>
> user@computer:~# feh -Fdr -A 'cp -a \"%F\" \"$(echo %F | sed -E "{
> s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family
> History/_all/1o'
>
> .... results in ...
>
> cp: target '1.png"' is not a directory
>
> Can anyone suggest a magical incantation which will yield the desired
> result?

I solved this by doing the path conversion in a one-line shell script,
so the feh command line now reads ...

feh -Fdr -A '/home/Family\ History/_all/FHCopyIn.sh %F' 'Family
History/_all/1o'

.... while the one-line shell script reads ...

cp -a "$1" "$(echo $1 | sed -E '{ s~/_all/[1-5][ont]{1,2}~~ ; s~
+\([ont]{1,2}\)~~ }' )"

> Another problem is that I'm going to have order the files so that each
> of the 5 possible versions are displayed successively so that I can use
> the arrow keys to go back and forth between them to choose the best.  I
> think I might be able to use a file list to do this, but as yet I
> haven't given much thought as to how to create it, it would have to be
> of the form:
>
> Family History/_all/1o/Alston/Document 1 (o).png
> Family History/_all/2n/Alston/Document 1 (n).png
> Family History/_all/3t/Alston/Document 1 (t).png
> Family History/_all/4nt/Alston/Document 1 (nt).png
> Family History/_all/5tn/Alston/Document 1 (tn).png
> Family History/_all/1o/Alston/Document 2 (o).png
> Family History/_all/2n/Alston/Document 2 (n).png
> Family History/_all/3t/Alston/Document 2 (t).png
> Family History/_all/4nt/Alston/Document 2 (nt).png
> Family History/_all/5tn/Alston/Document 2 (tn).png
> etc.
>
> I could also use Windows if there's a better solution available there,
> but that seems unlikely to me.

Still to do.

> Any help gratefully received.

Stet

--

Fake news kills!

I may be contacted via the contact address given on my website:
www.macfh.co.uk

On 30/03/2024 01:04, Java Jive wrote:
>
> First problem solved ...
>
>> On 29/03/2024 22:54, Java Jive wrote:
>>
>> Another problem is that I'm going to have order the files so that each
>> of the 5 possible versions are displayed successively so that I can
>> use the arrow keys to go back and forth between them to choose the
>> best.  I think I might be able to use a file list to do this, but as
>> yet I haven't given much thought as to how to create it, it would have
>> to be of the form:
>>
>> Family History/_all/1o/Alston/Document 1 (o).png
>> Family History/_all/2n/Alston/Document 1 (n).png
>> Family History/_all/3t/Alston/Document 1 (t).png
>> Family History/_all/4nt/Alston/Document 1 (nt).png
>> Family History/_all/5tn/Alston/Document 1 (tn).png
>> Family History/_all/1o/Alston/Document 2 (o).png
>> Family History/_all/2n/Alston/Document 2 (n).png
>> Family History/_all/3t/Alston/Document 2 (t).png
>> Family History/_all/4nt/Alston/Document 2 (nt).png
>> Family History/_all/5tn/Alston/Document 2 (tn).png
>> etc.

I've solved this part now as well. The following command does it
(originally all one line):

find 'Family History/_all' -name "*.png" | sort | sed -E '{ h;
s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ ; G ; s/\n/\t/ }' |
sort | sed 's/^[^\t]\+\t//' > 'Family History/_all/FHBlinkList.txt'

The most complex part is the first sed command, so I will explain it:

h Copy input, the original full pathname, to the hold space
s 2x Remove the '/_all/*' and ' (*)' parts of the pathname
G Retrieve and append the original line from the hold space
s Replace the intervening new line character with a tab

The second sed command removes the first part of the line up to and
including the tab which was used as a key in the final sort immediately
previously.

However, although the problems as originally stated are now both solved,
I need to do this second stage better, as I need to exclude from the
list those files that already have a chosen final copy in the main part
of the archive.

>> I could also use Windows if there's a better solution available there,
>> but that seems unlikely to me.
>
> Still to do.
>
>> Any help gratefully received.
>
> Stet

--

Fake news kills!

I may be contacted via the contact address given on my website:
www.macfh.co.uk

On 2024-03-29 23:54, Java Jive wrote:
> I've tried the following command line for feh (sorry can't help any line
> wrap):
>
> user@computer:~# feh -Fdr -A 'echo cp -a \"%F\" \"$(echo %F | sed -E "{
> s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family
> History/_all/1o'

There is an addon for Thunderbird that allows writing long lines.

user@computer:~# feh -Fdr -A 'echo cp -a \"%F\" \"$(echo %F | sed -E "{ s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family History/_all/1o'

Unfortunately, it applies to the entire post or email, not to paragraphs.

Toggle Line Wrap
Toggle line wrapping in the message composition window.

Author Jan Kiszka

Toggle wrapping of long lines in the message composition window when writing plaintext messages. Wrapping can be controlled via a toolbar button and a keyboard shortcut (CTRL-SHIFT-W by default). The current state is visualized in the toolbar button. The shortcut can be adjusted via the Add-on Manager (click on the gear).

On order to use the add-on, turn off HTML composition (per account setting, Composition & Addressing → Composition → Compose messages in HTML format) and also make sure that "mailnews.send_plaintext_flowed" is disabled in the configuration (about:config). Furthermore, mailnews.wraplength needs to be non-zero, or the add-on will be deactivated (toolbar button grayed out).

Note that this add-on requires an WebExtension Experiment API, thus will request "full access" on installation.

--
Cheers, Carlos.

On 30/03/2024 13:06, Carlos E.R. wrote:
>
> On 2024-03-29 23:54, Java Jive wrote:
>>
>> I've tried the following command line for feh (sorry can't help any
>> line wrap):
>>
>> user@computer:~# feh -Fdr -A 'echo cp -a \"%F\" \"$(echo %F | sed -E
>> "{ s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family
>> History/_all/1o'
>
> There is an addon for Thunderbird that allows writing long lines.
>
> Unfortunately, it applies to the entire post or email, not to paragraphs.

[snip]

Thanks, but as the long code lines only wrapped at the edge of the
window anyway, I'll give that a miss.

--

Fake news kills!

I may be contacted via the contact address given on my website:
www.macfh.co.uk

On 3/29/2024 6:54 PM, Java Jive wrote:
> As some here may recall, I'm in the midst of several years' work scanning family documents, post-processing them, and then releasing them either to the general public for generations sufficiently removed from living people, or else just back to the source branch of the family. Currently there are nearly 15,000 original documents with equal numbers of post-processed versions arranged as below, but around 7,000 of them completed in previous stages already have final documents, so there are around 8,000 newly added documents left to do.  What I need to do is use a program, feh seems a likely choice, to work through those documents that don't yet have a final version, choose one from five possible versions described below, copy it to its corresponding final directory, and move on to the next.
>
> The documents are arranged as follows:
>
> Family History
>     /_all/1o/subs    Original scans
>     /_all/2n/subs    Normalised versions of the originals
>     /_all/3t/subs    Textcleaned versions ditto
>     /_all/4nt/subs    Normalised then textcleaned versions ditto
>     /_all/5tn/subs    Textcleaned then normalised versions ditto
>     /subs        Final docs chosen from the best of the above
>
> Example for one particular subdirectory:
>
> Family History/_all/1o/Alston
> Family History/_all/2n/Alston
> Family History/_all/3t/Alston
> Family History/_all/4nt/Alston
> Family History/_all/5tn/Alston
> Family History/Alston
>
> I've tried the following command line for feh (sorry can't help any line wrap):
>
> user@computer:~# feh -Fdr -A 'echo cp -a \"%F\" \"$(echo %F | sed -E "{ s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family History/_all/1o'
>
> ... which results in ...
>
> cp -a "Family History/_all/1o/Alston/Alston Descent - 2nd Earl Of Montrose, William - 1 (o).png" "Family History/Alston/Alston Descent - 2nd Earl Of Montrose, William - 1.png"
>
> ... which looks as though it should do the job, however removing the echo ...
>
> user@computer:~# feh -Fdr -A 'cp -a \"%F\" \"$(echo %F | sed -E "{ s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family History/_all/1o'
>
> ... results in ...
>
> cp: target '1.png"' is not a directory
>
> Can anyone suggest a magical incantation which will yield the desired result?
>
> Another problem is that I'm going to have order the files so that each of the 5 possible versions are displayed successively so that I can use the arrow keys to go back and forth between them to choose the best.  I think I might be able to use a file list to do this, but as yet I haven't given much thought as to how to create it, it would have to be of the form:
>
> Family History/_all/1o/Alston/Document 1 (o).png
> Family History/_all/2n/Alston/Document 1 (n).png
> Family History/_all/3t/Alston/Document 1 (t).png
> Family History/_all/4nt/Alston/Document 1 (nt).png
> Family History/_all/5tn/Alston/Document 1 (tn).png
> Family History/_all/1o/Alston/Document 2 (o).png
> Family History/_all/2n/Alston/Document 2 (n).png
> Family History/_all/3t/Alston/Document 2 (t).png
> Family History/_all/4nt/Alston/Document 2 (nt).png
> Family History/_all/5tn/Alston/Document 2 (tn).png
> etc.
>
> I could also use Windows if there's a better solution available there, but that seems unlikely to me.
>
> Any help gratefully received.
>

If cp had a --dry-run, you could obtain what it sees that way.
When it doesn't have such an option, you can debug another way.

Make yourself a "dumper" to dump the parameters seen on the command line.
You can put "dumper" in place of "cp" and get the parameters that way.

gcc -o dumper dumper.c

#include<stdio.h>
int main(int argc, char *argv[])
{ int i;
printf("Number of arguments = %d\n", argc-1);
for(i=0; i<argc; i++)
{
if (i == 0) {
printf("Program name = %s\n", argv[i]);
} else {
printf("%s\n",argv[i]);
}
}
}

$ ./dumper
Number of arguments = 0
Program name = dumper

$ ./dumper a b c
Number of arguments = 3
Program name = dumper
a b
c

Your problem will likely involve "escaping" and "how many
levels of shells, subshells, and interpreting are going on".
You know the drill by now :-) It is the "land of the
random number of backslashes" :-)

Echo is both a built-in for the bash shell, as well
as a /usr/bin/echo existing (maybe some other shell
might need it).

By inspection, I don't understand why your command
is failing, but shine a flashlight on it and
see if you can figure it out. The usage of "dumper"
should have the same "nested shell count" as using "cp",
that's my hope.

Paul

On 30/03/2024 16:06, Paul wrote:
> On 3/29/2024 6:54 PM, Java Jive wrote:
>> As some here may recall, I'm in the midst of several years' work scanning family documents, post-processing them, and then releasing them either to the general public for generations sufficiently removed from living people, or else just back to the source branch of the family. Currently there are nearly 15,000 original documents with equal numbers of post-processed versions arranged as below, but around 7,000 of them completed in previous stages already have final documents, so there are around 8,000 newly added documents left to do.  What I need to do is use a program, feh seems a likely choice, to work through those documents that don't yet have a final version, choose one from five possible versions described below, copy it to its corresponding final directory, and move on to the next.
>>
>> The documents are arranged as follows:
>>
>> Family History
>>     /_all/1o/subs    Original scans
>>     /_all/2n/subs    Normalised versions of the originals
>>     /_all/3t/subs    Textcleaned versions ditto
>>     /_all/4nt/subs    Normalised then textcleaned versions ditto
>>     /_all/5tn/subs    Textcleaned then normalised versions ditto
>>     /subs        Final docs chosen from the best of the above
>>
>> Example for one particular subdirectory:
>>
>> Family History/_all/1o/Alston
>> Family History/_all/2n/Alston
>> Family History/_all/3t/Alston
>> Family History/_all/4nt/Alston
>> Family History/_all/5tn/Alston
>> Family History/Alston
>>
>> I've tried the following command line for feh (sorry can't help any line wrap):
>>
>> user@computer:~# feh -Fdr -A 'echo cp -a \"%F\" \"$(echo %F | sed -E "{ s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family History/_all/1o'
>>
>> ... which results in ...
>>
>> cp -a "Family History/_all/1o/Alston/Alston Descent - 2nd Earl Of Montrose, William - 1 (o).png" "Family History/Alston/Alston Descent - 2nd Earl Of Montrose, William - 1.png"
>>
>> ... which looks as though it should do the job, however removing the echo ...
>>
>> user@computer:~# feh -Fdr -A 'cp -a \"%F\" \"$(echo %F | sed -E "{ s~/_all/[1-5][ont]{1,2}~~ ; s~ +\([ont]{1,2}\)~~ }" )\"' 'Family History/_all/1o'
>>
>> ... results in ...
>>
>> cp: target '1.png"' is not a directory
>>
>> Can anyone suggest a magical incantation which will yield the desired result?
>>
>> Another problem is that I'm going to have order the files so that each of the 5 possible versions are displayed successively so that I can use the arrow keys to go back and forth between them to choose the best.  I think I might be able to use a file list to do this, but as yet I haven't given much thought as to how to create it, it would have to be of the form:
>>
>> Family History/_all/1o/Alston/Document 1 (o).png
>> Family History/_all/2n/Alston/Document 1 (n).png
>> Family History/_all/3t/Alston/Document 1 (t).png
>> Family History/_all/4nt/Alston/Document 1 (nt).png
>> Family History/_all/5tn/Alston/Document 1 (tn).png
>> Family History/_all/1o/Alston/Document 2 (o).png
>> Family History/_all/2n/Alston/Document 2 (n).png
>> Family History/_all/3t/Alston/Document 2 (t).png
>> Family History/_all/4nt/Alston/Document 2 (nt).png
>> Family History/_all/5tn/Alston/Document 2 (tn).png
>> etc.
>>
>> I could also use Windows if there's a better solution available there, but that seems unlikely to me.
>>
>> Any help gratefully received.
>>
>
> If cp had a --dry-run, you could obtain what it sees that way.
> When it doesn't have such an option, you can debug another way.
>
> Make yourself a "dumper" to dump the parameters seen on the command line.
> You can put "dumper" in place of "cp" and get the parameters that way.
>
> gcc -o dumper dumper.c
>
> #include<stdio.h>
> int main(int argc, char *argv[])
> {
> int i;
> printf("Number of arguments = %d\n", argc-1);
> for(i=0; i<argc; i++)
> {
> if (i == 0) {
> printf("Program name = %s\n", argv[i]);
> } else {
> printf("%s\n",argv[i]);
> }
> }
> }
>
> $ ./dumper
> Number of arguments = 0
> Program name = dumper
>
> $ ./dumper a b c
> Number of arguments = 3
> Program name = dumper
> a
> b
> c
>
> Your problem will likely involve "escaping" and "how many
> levels of shells, subshells, and interpreting are going on".
> You know the drill by now :-) It is the "land of the
> random number of backslashes" :-)
>
> Echo is both a built-in for the bash shell, as well
> as a /usr/bin/echo existing (maybe some other shell
> might need it).
>
> By inspection, I don't understand why your command
> is failing, but shine a flashlight on it and
> see if you can figure it out. The usage of "dumper"
> should have the same "nested shell count" as using "cp",
> that's my hope.

Thanks Paul, I've copied the above into a folder to keep it. It's a
clever technique which I have used before with Windows BATch files - I
still have PrintCMD.COM in my Utilities folder! However, as regards the
current problems, as already described in a follow-up post, I solved it
another way, by putting the cp command in a one-line shell script.

In fact, I've now solved all the problems and have my list. It's 42,940
lines along representing at least 9,000 documents, but actually more
than that, because photos, maps, sketches, water-colours, etc will not
have textcleaned versions, only 'original' and 'normalised' versions.

Phew! Better get to it ...

Here is the FHBlinkList.sh script that created the list:

#!/bin/sh
ME="${0##*/}"
DIRY="${0%/*}"

# The Family History sub-directory
FH="~/Family History/_all"

OLDIFS=${IFS}
IFS=$'\n'
for A in $(find "${FH}" -name "*.png" | sort)
do
TARGET="$(echo $A | sed -E '{ s~/_all/[1-5][ont]{1,2}~~ ; s~
+\([ont]{1,2}\)~~ }' )"
if [ ! -f "${TARGET}" ] || [ "${A}" -nt "${TARGET}" ]
then
echo -e "${TARGET}\t${A}"
fi
done | sort | sed 's/^[^\t]\+\t//' > "${FH}/FHBlinkList.txt"
IFS=${OLDIFS}

--

Fake news kills!

I may be contacted via the contact address given on my website:
www.macfh.co.uk