Successful spamassassin tests were performed on commit:
fcaea2cee456de55d1a00c18e13ca8be183b9fdc.

Rslight appears to be working correctly with spamassassin. I
blacklisted some addresses in spamassassin configuration. The
blacklisted addresses and messages with clearly malformed headers are
filtered out and do not reach the spool.

The files 'banned_names.conf' and 'header_filters.conf' do not appear
to have any effect. It may be that I do not have the syntax correct for
entries in these files. Here is copypasta for correction if necessary:

$ cat header_filters.conf
From: DEADBEEF
From: /DEADBEEF/
From: Black Hole
From: /Black Hole/
From: \<black@hole\.url\>

$ cat banned_names.conf
DEADBEEF
/DEADBEEF/
Black Hole
/Black Hole/
\<black@hole\.url\>

Is the regex flavor the PHP/Apache regex specification?

Do the entries in header_filters.conf affect both locally posted and
remotely pulled articles? What about banned_names.conf?

Once I get them right I will try testing again.

--
CRYP7010G3R | Read between the signals. | Read between the signs.
<6ef76f5d853272d23e209476783618ed$1@sybershock.com>
3883@sugar.bug | sybershock.com | alt.sources.crypto

On Thu, 28 Dec 2023 12:07:19 -0600, SugarBug wrote:

> Successful spamassassin tests were performed on commit:
> fcaea2cee456de55d1a00c18e13ca8be183b9fdc.
>
> Rslight appears to be working correctly with spamassassin. I
> blacklisted some addresses in spamassassin configuration. The
> blacklisted addresses and messages with clearly malformed headers are
> filtered out and do not reach the spool.
>
> The files 'banned_names.conf' and 'header_filters.conf' do not appear
> to have any effect. It may be that I do not have the syntax correct for
> entries in these files. Here is copypasta for correction if necessary:

banned_names.conf is deprecated. It's no longer read.

header_filters.conf is the way to go.

> $ cat header_filters.conf
> From: DEADBEEF // Will not work (not regex format)
> From: /DEADBEEF/ // That should work
> From: Black Hole // Will not work (not regex format)
> From: /Black Hole/ // That should work
> From: \<black@hole\.url\> // Will not work (not regex format)

Start each with '/' and end with '/' Then escape '\' any special
characters. The value is passed to preg_match in php:
https://www.php.net/manual/en/function.preg-match.php

If it works at this site: https://regex101.com/ it "should" work.

> Do the entries in header_filters.conf affect both locally posted and
> remotely pulled articles? What about banned_names.conf?

Right now it will only work on remotely pulled articles, not locally
posted.

Retro Guy wrote:

> On Thu, 28 Dec 2023 12:07:19 -0600, SugarBug wrote:

>> Successful spamassassin tests were performed on commit:
>> fcaea2cee456de55d1a00c18e13ca8be183b9fdc.
>>
>> Rslight appears to be working correctly with spamassassin. I
>> blacklisted some addresses in spamassassin configuration. The
>> blacklisted addresses and messages with clearly malformed headers are
>> filtered out and do not reach the spool.
>>
>> The files 'banned_names.conf' and 'header_filters.conf' do not appear
>> to have any effect. It may be that I do not have the syntax correct for
>> entries in these files. Here is copypasta for correction if necessary:

> banned_names.conf is deprecated. It's no longer read.

> header_filters.conf is the way to go.

>> $ cat header_filters.conf
>> From: DEADBEEF // Will not work (not regex format)
>> From: /DEADBEEF/ // That should work
>> From: Black Hole // Will not work (not regex format)
>> From: /Black Hole/ // That should work
>> From: <black@hole.url> // Will not work (not regex format)

> Start each with '/' and end with '/' Then escape '' any special
> characters. The value is passed to preg_match in php:
> https://www.php.net/manual/en/function.preg-match.php

> If it works at this site: https://regex101.com/ it "should" work.

>> Do the entries in header_filters.conf affect both locally posted and
>> remotely pulled articles? What about banned_names.conf?

> Right now it will only work on remotely pulled articles, not locally
> posted.

Also, for your reference, it is designed currently to work only on the following headers:

From:
Subject:
Path:
Message-ID:

Other headers will be ignored.

--
Retro Guy

On Fri, 29 Dec 2023 13:14:18 +0000
retro.guy@rocksolidbbs.com (Retro Guy) wrote:

> Retro Guy wrote:
>
> > On Thu, 28 Dec 2023 12:07:19 -0600, SugarBug wrote:
>
> >> Successful spamassassin tests were performed on commit:
> >> fcaea2cee456de55d1a00c18e13ca8be183b9fdc.
> >>
> >> Rslight appears to be working correctly with spamassassin. I
> >> blacklisted some addresses in spamassassin configuration. The
> >> blacklisted addresses and messages with clearly malformed headers
> >> are filtered out and do not reach the spool.
> >>
> >> The files 'banned_names.conf' and 'header_filters.conf' do not
> >> appear to have any effect. It may be that I do not have the syntax
> >> correct for entries in these files. Here is copypasta for
> >> correction if necessary:
>
> > banned_names.conf is deprecated. It's no longer read.
>
> > header_filters.conf is the way to go.
>
> >> $ cat header_filters.conf
> >> From: DEADBEEF // Will not work (not regex format)
> >> From: /DEADBEEF/ // That should work
> >> From: Black Hole // Will not work (not regex format)
> >> From: /Black Hole/ // That should work
> >> From: <black@hole.url> // Will not work (not regex format)
>
> > Start each with '/' and end with '/' Then escape '' any special
> > characters. The value is passed to preg_match in php:
> > https://www.php.net/manual/en/function.preg-match.php
>
> > If it works at this site: https://regex101.com/ it "should" work.

Ok. So it looks like rslight is using preg_match?

I tested a few regexes for substrings and I think I got it right. Let
me know if this is wrong.

For instance I confirmed that matching: '/test/i'

to: 'TeSt_tEsT' or 'testing' or 'TESTING'

works for case insensitive substring 'test'.

I sent messages using one server and the sync server flagged them as
spam based on the substring being in the 'from' header in
header_filters.conf. So it works. There is no problem there.

<snip>

This leads me to another question. Take these paths for example:

news.spam.com!google-groups.googlegroups.com!not-for-mail
news.spam.com!google-groups.googlegroups.com!.POSTED!not-for-mail

I want to detect and ignore ".POSTED' and 'not-for-mail' and match
google in the path entry right next to it. So if google is in the
origin I want the match to be positive for spam flag. I can't use just
'$' for end of string regex because of 'not-for-mail' and other strings
like that that servers sometimes inject. How can I break this down? I
have no idea how to reference count the exclamation marks from string
tail in a preg_match regex string. I suspect I need to reference count
somehow and do a lookaround match between the last two '!' characters.

Before I start pulling a big feed of groups I want to ensure I am
pulling zero messages that originate with Google. I don't care if they
are spam or not, since I don't have time to sort them. I just want to
ensure that nothing originating with google is pulled into the spool
from remote.

--
1_|3_|7_|4_ 3883@sugar.bug | sybershock.com | alt.sources.crypto
1_|8_|7_|4_ sci.crypt | alt.random.noise | talk.politics.crypto
4_|8_|6_|7_ #CipherTag #WaffleTag #Cryptologer
4_|3_|6_|7_ fedi: @firefly@neon.nightbulb.net

On Tue, 2 Jan 2024 20:51:35 -0600, SugarBug wrote:

> On Fri, 29 Dec 2023 13:14:18 +0000
> retro.guy@rocksolidbbs.com (Retro Guy) wrote:
>
>> Retro Guy wrote:
>>
>>> On Thu, 28 Dec 2023 12:07:19 -0600, SugarBug wrote:
>>

> <snip>
>
> This leads me to another question. Take these paths for example:
>
> news.spam.com!google-groups.googlegroups.com!not-for-mail
> news.spam.com!google-groups.googlegroups.com!.POSTED!not-for-mail
>
> I want to detect and ignore ".POSTED' and 'not-for-mail' and match
> google in the path entry right next to it. So if google is in the
> origin I want the match to be positive for spam flag. I can't use just
> '$' for end of string regex because of 'not-for-mail' and other strings
> like that that servers sometimes inject. How can I break this down? I
> have no idea how to reference count the exclamation marks from string
> tail in a preg_match regex string. I suspect I need to reference count
> somehow and do a lookaround match between the last two '!' characters.

There is preg_split() in php. Similar to explode().

You can create an array with these functins using whatever you want as a
separator, so you could use '!'. Then count forward, backward, however you
want.

Not sure all that is necessary to just block gg, but it's available.

SugarBug wrote:

> On Fri, 29 Dec 2023 13:14:18 +0000
> retro.guy@rocksolidbbs.com (Retro Guy) wrote:

>> Retro Guy wrote:
>>
>> > On Thu, 28 Dec 2023 12:07:19 -0600, SugarBug wrote:
>>
<snip>
>>
>> > banned_names.conf is deprecated. It's no longer read.
>>
>> > header_filters.conf is the way to go.
>>
>> >> $ cat header_filters.conf
>> >> From: DEADBEEF // Will not work (not regex format)
>> >> From: /DEADBEEF/ // That should work
>> >> From: Black Hole // Will not work (not regex format)
>> >> From: /Black Hole/ // That should work
>> >> From: <black@hole.url> // Will not work (not regex format)
>>
>> > Start each with '/' and end with '/' Then escape '' any special
>> > characters. The value is passed to preg_match in php:
>> > https://www.php.net/manual/en/function.preg-match.php
>>
>> > If it works at this site: https://regex101.com/ it "should" work.

> Ok. So it looks like rslight is using preg_match?

Yes, that is how the headers are checked against the filters.

> I tested a few regexes for substrings and I think I got it right. Let
> me know if this is wrong.

> For instance I confirmed that matching: '/test/i'

> to: 'TeSt_tEsT' or 'testing' or 'TESTING'

> works for case insensitive substring 'test'.

It's correct :) (/i) makes it case insensitive.

> I sent messages using one server and the sync server flagged them as
> spam based on the substring being in the 'from' header in
> header_filters.conf. So it works. There is no problem there.

> <snip>

> This leads me to another question. Take these paths for example:

> news.spam.com!google-groups.googlegroups.com!not-for-mail
> news.spam.com!google-groups.googlegroups.com!.POSTED!not-for-mail

I've seen the top one, but not the one with 'POSTED' from gg, but I haven't looked at a gg header in a while (it's all automated).

> I want to detect and ignore ".POSTED' and 'not-for-mail' and match
> google in the path entry right next to it. So if google is in the
> origin I want the match to be positive for spam flag. I can't use just
> '$' for end of string regex because of 'not-for-mail' and other strings
> like that that servers sometimes inject. How can I break this down? I
> have no idea how to reference count the exclamation marks from string
> tail in a preg_match regex string. I suspect I need to reference count
> somehow and do a lookaround match between the last two '!' characters.

I do most of my regex testing here:
https://regex101.com/
Just paste your line in, then try to match it.

This seems to work ok with or without the .POSTED:
/groups.googlegroups.com(!.POSTED)?!not-for-mail/

> Before I start pulling a big feed of groups I want to ensure I am
> pulling zero messages that originate with Google. I don't care if they
> are spam or not, since I don't have time to sort them. I just want to
> ensure that nothing originating with google is pulled into the spool
> from remote.

Makes sense, and have fun!

I'll be around later today.

--
Retro Guy

On Wed, 3 Jan 2024 12:43:53 +0000, Retro Guy wrote:

> SugarBug wrote:
>
>> On Fri, 29 Dec 2023 13:14:18 +0000
>> retro.guy@rocksolidbbs.com (Retro Guy) wrote:
>
>>> Retro Guy wrote:
>>>
>>> > On Thu, 28 Dec 2023 12:07:19 -0600, SugarBug wrote:
>>>
> <snip>
>>>
>>> > banned_names.conf is deprecated. It's no longer read.
>>>
>>> > header_filters.conf is the way to go.
>>>
>>> >> $ cat header_filters.conf
>>> >> From: DEADBEEF // Will not work (not regex format)
>>> >> From: /DEADBEEF/ // That should work
>>> >> From: Black Hole // Will not work (not regex format)
>>> >> From: /Black Hole/ // That should work
>>> >> From: <black@hole.url> // Will not work (not regex format)
>>>
>>> > Start each with '/' and end with '/' Then escape '' any special
>>> > characters. The value is passed to preg_match in php:
>>> > https://www.php.net/manual/en/function.preg-match.php
>>>
>>> > If it works at this site: https://regex101.com/ it "should" work.
>
>> Ok. So it looks like rslight is using preg_match?
>
> Yes, that is how the headers are checked against the filters.
>
>> I tested a few regexes for substrings and I think I got it right. Let
>> me know if this is wrong.
>
>> For instance I confirmed that matching: '/test/i'
>
>> to: 'TeSt_tEsT' or 'testing' or 'TESTING'
>
>> works for case insensitive substring 'test'.
>
> It's correct :) (/i) makes it case insensitive.
>
>> I sent messages using one server and the sync server flagged them as
>> spam based on the substring being in the 'from' header in
>> header_filters.conf. So it works. There is no problem there.
>
>> <snip>
>
>> This leads me to another question. Take these paths for example:
>
>> news.spam.com!google-groups.googlegroups.com!not-for-mail
>> news.spam.com!google-groups.googlegroups.com!.POSTED!not-for-mail
>
> I've seen the top one, but not the one with 'POSTED' from gg, but I haven't looked at a gg header in a while (it's all automated).
>
>> I want to detect and ignore ".POSTED' and 'not-for-mail' and match
>> google in the path entry right next to it. So if google is in the
>> origin I want the match to be positive for spam flag. I can't use just
>> '$' for end of string regex because of 'not-for-mail' and other strings
>> like that that servers sometimes inject. How can I break this down? I
>> have no idea how to reference count the exclamation marks from string
>> tail in a preg_match regex string. I suspect I need to reference count
>> somehow and do a lookaround match between the last two '!' characters.
>
> I do most of my regex testing here:
> https://regex101.com/
> Just paste your line in, then try to match it.
>
> This seems to work ok with or without the .POSTED:
> /groups.googlegroups.com(!.POSTED)?!not-for-mail/

All the backslashes are stripped. I need to fix that at some point.

Anyway, the above regex should contain a number of backslashes.

Retro Guy wrote:

> SugarBug wrote:

>> On Fri, 29 Dec 2023 13:14:18 +0000
>> retro.guy@rocksolidbbs.com (Retro Guy) wrote:

>>> Retro Guy wrote:
>>>
>>> > On Thu, 28 Dec 2023 12:07:19 -0600, SugarBug wrote:
>>>
> <snip>
>>>
>>> > banned_names.conf is deprecated. It's no longer read.
>>>
>>> > header_filters.conf is the way to go.
>>>
>>> >> $ cat header_filters.conf
>>> >> From: DEADBEEF // Will not work (not regex format)
>>> >> From: /DEADBEEF/ // That should work
>>> >> From: Black Hole // Will not work (not regex format)
>>> >> From: /Black Hole/ // That should work
>>> >> From: <black@hole.url> // Will not work (not regex format)
>>>
>>> > Start each with '/' and end with '/' Then escape '' any special
>>> > characters. The value is passed to preg_match in php:
>>> > https://www.php.net/manual/en/function.preg-match.php
>>>
>>> > If it works at this site: https://regex101.com/ it "should" work.

>> Ok. So it looks like rslight is using preg_match?

> Yes, that is how the headers are checked against the filters.

>> I tested a few regexes for substrings and I think I got it right. Let
>> me know if this is wrong.

>> For instance I confirmed that matching: '/test/i'

>> to: 'TeSt_tEsT' or 'testing' or 'TESTING'

>> works for case insensitive substring 'test'.

> It's correct :) (/i) makes it case insensitive.

>> I sent messages using one server and the sync server flagged them as
>> spam based on the substring being in the 'from' header in
>> header_filters.conf. So it works. There is no problem there.

>> <snip>

>> This leads me to another question. Take these paths for example:

>> news.spam.com!google-groups.googlegroups.com!not-for-mail
>> news.spam.com!google-groups.googlegroups.com!.POSTED!not-for-mail

> I've seen the top one, but not the one with 'POSTED' from gg, but I haven't looked at a gg header in a while (it's all automated).

>> I want to detect and ignore ".POSTED' and 'not-for-mail' and match
>> google in the path entry right next to it. So if google is in the
>> origin I want the match to be positive for spam flag. I can't use just
>> '$' for end of string regex because of 'not-for-mail' and other strings
>> like that that servers sometimes inject. How can I break this down? I
>> have no idea how to reference count the exclamation marks from string
>> tail in a preg_match regex string. I suspect I need to reference count
>> somehow and do a lookaround match between the last two '!' characters.

> I do most of my regex testing here:
> https://regex101.com/
> Just paste your line in, then try to match it.

> This seems to work ok with or without the .POSTED:
> /groups.googlegroups.com(!.POSTED)?!not-for-mail/

Testing where I shouldn't be testing.

Do the backslashes remain?

/groups\.googlegroups\.com(\!\.POSTED)?|not-for-mail/

--
Unconfigured rslight site

Retro Guy wrote:

> Retro Guy wrote:

>> SugarBug wrote:

>>> On Fri, 29 Dec 2023 13:14:18 +0000
>>> retro.guy@rocksolidbbs.com (Retro Guy) wrote:

>>>> Retro Guy wrote:
>>>>
>>>> > On Thu, 28 Dec 2023 12:07:19 -0600, SugarBug wrote:
>>>>
>> <snip>
>>>>
>>>> > banned_names.conf is deprecated. It's no longer read.
>>>>
>>>> > header_filters.conf is the way to go.
>>>>
>>>> >> $ cat header_filters.conf
>>>> >> From: DEADBEEF // Will not work (not regex format)
>>>> >> From: /DEADBEEF/ // That should work
>>>> >> From: Black Hole // Will not work (not regex format)
>>>> >> From: /Black Hole/ // That should work
>>>> >> From: <black@hole.url> // Will not work (not regex format)
>>>>
>>>> > Start each with '/' and end with '/' Then escape '' any special
>>>> > characters. The value is passed to preg_match in php:
>>>> > https://www.php.net/manual/en/function.preg-match.php
>>>>
>>>> > If it works at this site: https://regex101.com/ it "should" work.

>>> Ok. So it looks like rslight is using preg_match?

>> Yes, that is how the headers are checked against the filters.

>>> I tested a few regexes for substrings and I think I got it right. Let
>>> me know if this is wrong.

>>> For instance I confirmed that matching: '/test/i'

>>> to: 'TeSt_tEsT' or 'testing' or 'TESTING'

>>> works for case insensitive substring 'test'.

>> It's correct :) (/i) makes it case insensitive.

>>> I sent messages using one server and the sync server flagged them as
>>> spam based on the substring being in the 'from' header in
>>> header_filters.conf. So it works. There is no problem there.

>>> <snip>

>>> This leads me to another question. Take these paths for example:

>>> news.spam.com!google-groups.googlegroups.com!not-for-mail
>>> news.spam.com!google-groups.googlegroups.com!.POSTED!not-for-mail

>> I've seen the top one, but not the one with 'POSTED' from gg, but I haven't looked at a gg header in a while (it's all automated).

>>> I want to detect and ignore ".POSTED' and 'not-for-mail' and match
>>> google in the path entry right next to it. So if google is in the
>>> origin I want the match to be positive for spam flag. I can't use just
>>> '$' for end of string regex because of 'not-for-mail' and other strings
>>> like that that servers sometimes inject. How can I break this down? I
>>> have no idea how to reference count the exclamation marks from string
>>> tail in a preg_match regex string. I suspect I need to reference count
>>> somehow and do a lookaround match between the last two '!' characters.

>> I do most of my regex testing here:
>> https://regex101.com/
>> Just paste your line in, then try to match it.

>> This seems to work ok with or without the .POSTED:
>> /groups.googlegroups.com(!.POSTED)?!not-for-mail/

> Testing where I shouldn't be testing.

> Do the backslashes remain?

> /groups\.googlegroups\.com(\!\.POSTED)?|not-for-mail/

Appears so.

--
Unconfigured rslight site

Retro Guy wrote:

> Retro Guy wrote:

>> Retro Guy wrote:

>>> SugarBug wrote:

>>>> On Fri, 29 Dec 2023 13:14:18 +0000
>>>> retro.guy@rocksolidbbs.com (Retro Guy) wrote:

>>>>> Retro Guy wrote:
>>>>>
>>>>> > On Thu, 28 Dec 2023 12:07:19 -0600, SugarBug wrote:
>>>>>
>>> <snip>
>>>>>
>>>>> > banned_names.conf is deprecated. It's no longer read.
>>>>>
>>>>> > header_filters.conf is the way to go.
>>>>>
>>>>> >> $ cat header_filters.conf
>>>>> >> From: DEADBEEF // Will not work (not regex format)
>>>>> >> From: /DEADBEEF/ // That should work
>>>>> >> From: Black Hole // Will not work (not regex format)
>>>>> >> From: /Black Hole/ // That should work
>>>>> >> From: <black@hole.url> // Will not work (not regex format)
>>>>>
>>>>> > Start each with '/' and end with '/' Then escape '' any special
>>>>> > characters. The value is passed to preg_match in php:
>>>>> > https://www.php.net/manual/en/function.preg-match.php
>>>>>
>>>>> > If it works at this site: https://regex101.com/ it "should" work.

>>>> Ok. So it looks like rslight is using preg_match?

>>> Yes, that is how the headers are checked against the filters.

>>>> I tested a few regexes for substrings and I think I got it right. Let
>>>> me know if this is wrong.

>>>> For instance I confirmed that matching: '/test/i'

>>>> to: 'TeSt_tEsT' or 'testing' or 'TESTING'

>>>> works for case insensitive substring 'test'.

>>> It's correct :) (/i) makes it case insensitive.

>>>> I sent messages using one server and the sync server flagged them as
>>>> spam based on the substring being in the 'from' header in
>>>> header_filters.conf. So it works. There is no problem there.

>>>> <snip>

>>>> This leads me to another question. Take these paths for example:

>>>> news.spam.com!google-groups.googlegroups.com!not-for-mail
>>>> news.spam.com!google-groups.googlegroups.com!.POSTED!not-for-mail

>>> I've seen the top one, but not the one with 'POSTED' from gg, but I haven't looked at a gg header in a while (it's all automated).

>>>> I want to detect and ignore ".POSTED' and 'not-for-mail' and match
>>>> google in the path entry right next to it. So if google is in the
>>>> origin I want the match to be positive for spam flag. I can't use just
>>>> '$' for end of string regex because of 'not-for-mail' and other strings
>>>> like that that servers sometimes inject. How can I break this down? I
>>>> have no idea how to reference count the exclamation marks from string
>>>> tail in a preg_match regex string. I suspect I need to reference count
>>>> somehow and do a lookaround match between the last two '!' characters.

>>> I do most of my regex testing here:
>>> https://regex101.com/
>>> Just paste your line in, then try to match it.

>>> This seems to work ok with or without the .POSTED:
>>> /groups.googlegroups.com(!.POSTED)?!not-for-mail/

>> Testing where I shouldn't be testing.

>> Do the backslashes remain?

>> /groups\.googlegroups\.com(\!\.POSTED)?|not-for-mail/

> Appears so.

Great! But it actually should be:

/groups\.googlegroups\.com(\!\.POSTED)?\!not-for-mail/

--
Retro Guy

On Fri, 5 Jan 2024 14:04:18 +0000
retro.guy@rocksolidbbs.com (Retro Guy) wrote:

> >> Do the backslashes remain?
>
> >> /groups\.googlegroups\.com(\!\.POSTED)?|not-for-mail/
>
> > Appears so.
>
> Great! But it actually should be:
>
> /groups\.googlegroups\.com(\!\.POSTED)?\!not-for-mail/

Is the backend stripping them?

--
No man is free who is not master of himself. ~ Reinhard Ponty

SugarBug wrote:

> On Fri, 5 Jan 2024 14:04:18 +0000
> retro.guy@rocksolidbbs.com (Retro Guy) wrote:

>> >> Do the backslashes remain?
>>
>> >> /groups\.googlegroups\.com(\!\.POSTED)?|not-for-mail/
>>
>> > Appears so.
>>
>> Great! But it actually should be:
>>
>> /groups\.googlegroups\.com(\!\.POSTED)?\!not-for-mail/

> Is the backend stripping them?

It was in post.php. Backslashes were being stripped at the 'textarea' where the post is entered. That included quoted text.

Posts sent / received using the nntp server not affected.

Should be all fixed now.

--
Retro Guy